Charging voltage, Bulk charging voltage, trickle charge voltage, battery voltage without load and battery voltage with load are all going to be different.
A full charged battery without load will be 13-13.5V for a 12Volt battery. Giving you a total of 26+ volts when connected in series with no Load
Upon load application via UPS, the voltage will instantly drop down to 12.5-12.8 volt for each battery, resulting in a total of 25.x Volts for a full charged battery with load.
A discharged (but healthy battery with correct specific gravity) should read 12 Volt without load and 11.x with load. (never apply a load to a discharged battery)
A discharged battery with weak electrolyte/low specific gravity can drop down to 9.x Volt.
Some heat is expected when charging the battery. Constantly warm batteries can indicate that UPS is trying to full up the charge but somehow the battery is not reaching the charge.
A battery with on or more shorted cells will get charged very quickly. Like within 15-20 minutes UPS will stop charging for a battery or set of battery that have one or more shorted cells.
If your batteries are healthy and and they are warming up even in room temperature that make sure that UPS charging voltage is 26.x Volts. If your charging voltage is 28.x Volts and you have standard batteries for your UPS then it means there is a problem with UPS (or its software settings).
I have an NS 3000 Alfa UPS which allows to specify a custom battery setting with custom Voltage and current configuration for the bulk charging mode and custom voltage for trickle charging. If your UPS allows software configuration, please make sure your bulk charging voltage is right.
Since I have custom built battery, I set bulk charging voltage to 28.x for 2-3 days and then change that to 26.x Volt after that because between load sheddings, the batteries take 2-3 to get fully charged.
A standard battery bulk charging voltage should not be more than 26.x (for two batteries connected in series with the UPS).
For custom built batteries, the voltage may need to be adjusted for a few days to as high as 28.x Volts. But this is only for custom built desi batteries. I get my batteries custom built once their warranty expires and they become useless. Custom built batteries store more current, last longer and are more reliable in high load shedding scenarios.
Warm battery in room temperature indicates over-charging. You may notice that the size battery holes with caps may look wet and smelly with Acid. The top or bodies may also indicate electrolyte spill. This will be in the case when the electrolyte/water in your lead acid batteries is full and your UPS is overcharging for some reason.
Warming = over-charging = problem with software or hardware configuration of the UPS (assuming your batteries are healthy and you have proper means to check your battery with and without load)