Thats the pat answer. As the batteries charge their voltages rise again and this time the smaller battery charges faster.
Second Solution Circuitlab
Charging 12v batteries in series. Make sure that your 12v charger is rated for the type of batteries youre using like sealed vs. Then connect the negative end of the battery to the positive end of the next one and continue to do so for the rest of your batteries. Assuming nothing has exploded but the 12 volt battery eventually dropped in voltage to a point where the device cut off the supply you are left with a fairly flat 12 volt battery and a very flat 6 volt battery. Charging 12 volt batteries wired in parallel requires a 12 volt battery charger. For the final battery connect the negative end of the battery to the negative output in black of the charger. Two 12 volt batteries wired in parallel continue to produce 12 volts but the capacity doubles.
Charge all the batteries at the same time. Two 12 volt batteries wired in series produce 24 volts but the capacity remains the same as one for a 12 volt battery. To charge two 12 volt batteries in series youd need a 24 volt charger. This means you have a fully charged 48v battery pack with each battery at the same state of charge. One basic configuration for charging batteries in series is to connect the positive charger output in red to the positive end of one of the batteries. You need to overcome the total voltage of the series batteries 12 v 12 v 24 v to drive electrons.
Charging only some of the batteries will result in batteries attempting to equal out the power and charging each other. Charge more than five batteries by connecting one 12 volt battery charger across each battery in the series as if each battery were the only one being charged. After your 4 12v batteries are all topped off take them out of parallel and wire them in series. 147v total and then float around 135v. Note that a 12v battery will charge to approx. Batteries may be wired in parallel or in series.