Battery holders are a good practice when using batteries rather than wall wart power supplies, or when you need to power your prototype wirelessly. For starters in electronics, regular batteries are safer to use than Li-Ion or Li-Po batteries.
Batteries can be connected in series to increase the voltage in increments of 1.5 volts. Arduinos need at least 7 volts input, with a maximum of 12 volts.
Battery holders exist for various types of batteries. Most used are AA and AAA batteries for which holders can be bought that hold 1, 2, 3, or 4 batteries. Some holders use the same battery clips for connecting as those used for 9V block cell batteries, whereas others connect through wires.