Yes. I have noticed this with my solar charger and 3.7V LiPo battery - but mine is only about 0.3V higher when there is sun. I think of it as the charger having a higher voltage to push the power into the battery.
I found the power module for my hobby greenhouse to be the most involved part. Trying to work out how big a battery I will need to last 3 days with no sun in winter, how big a solar panel to keep it charged, and how to reduce the power consumed by the rest of the system. So many factors that are variable.
How long does the battery last ? Is it the charge which runs out, or is the battery physically dead (unable to charge) ? What battery and charger ?
First thing to check is how much power is your solar panel actually generating. Most panels specifications are "theoretical maximum" - not what anyone gets in the real world. Consider also that every day is not bright sunshine, so won't generate as much power; and your panel is almost certainly fixed, so can't track the sun for optimal angle of reception throughout the day/year.
Next major issue is to adjust how often your measuring device is doing the checks ... though if you use a device like a Shelly you may not have any control over this. If your battery is running down slowly, checking once per day may still give you enough warning to do something about it. In my case I discovered I can plug a powerbank into the USB port on the solar charger to give it a boost if needed.
In my case I'm using an ESP32-S3 as my main microcontroller, with a DFRobot Solar Power manager board, 5000mAh LiPo battery and had to add an INA3221 to measure power to/from the battery, and from the solar panel. I also had to upgrade to a bigger solar panel. Much of my findings in Notes on ESPHome deep_sleep