Headers are inexpensive means to connect wires to Arduinos or other circuit boards, while still being able to disconnect a pcb from your prototype. Always use plugs if you can. It keeps things modular, and you can exchange an Arduino with another if you have doubts that your Arduino is broken.

A header can be used for low voltages, and currents under 1 Ampère.