Replacements are used mainly to recover losses, and I think to add regiments to some brigades or fortress garrisons when they start under strength in some scenarios. There are some rather obscure rules as to how this actually works - putting a stack in passive posture (green flag on the top row of the red-orange-blue-green display for the stack) gives it priority in receiving replacements, though this may not be a good thing to do in a battle area.
Reinforcements are entirely new units, appearing on the turns after they are paid for...
Union:
+1 Militia, supply wagons.
+2 Infantry, sharpshooters, cavalry, mixed brigades containing these, sailors.
+3 Field artillery (all weights), marines, zouaves, mixed brigades containing artillery. Brigs, transports, river transports, steam frigates.
+4 Siege and coastal artillery. Support (signallers, engineers, hospitals etc). All other ship types.
CSA:
+1 Milita, supply wagons.
+2 Infantry, cavalry, zouaves, mixed brigades containing these.
+3 Artillery, mixed brigades with artillery, marines, sharpshooters, TX Rangers, support units except naval engineers, Army HQs, ships except frigates.
+4 Naval engineers, frigates.
Caveat: these numbers are a bit provisional as they vary by what and how many units are being built in which cities.
Militia cost resources to build and extra when they randomly convert in time to regulars, though rather poor quality regulars. I believe the next game upgrade will add to the conversion cost, which will make building masses of militia a risky option

.