Disclaimer: I haven't checked your wiring - the diagram is incomplete and truncated. I'm going by your description of what happens.
Go back to the design stage and component choices.
Is the 9v battery able to satisfy the solenoid current requirements? As the voltage goes down, the current goes up. They are usually specified as 24V with a VA rating. Use Ohms law to work out the current requirements of your solenoid at 24v, 12v, and 9V. AC or DC operation (it usually doesn't matter)? Are you using that 9v battery because your rechargeable battery doesn't have enough grunt?
Is your wattmeter recording and reporting peaks, or just sampling? You may not be getting the whole picture from it. An analog (non-sampling) oscilloscope is the more appropriate tool for this where you can see short spikes in real time that sampling or a multimeter (analog or digital) can often miss.
Look at the data sheets of each and every component to check maximum current draw. You can then decide if it will work in circuit.
Are your power supplies connected correctly? Common where needed and not where shorts develop, especially with relays. Is your 9v battery life long enough to keep the solenoid closed?
Does your latching relay have dry contacts? Does it need to be powered at high current draw on both on and off modes? Is the pulse to toggle coming from the correct GPIO port? You havent showed any code.
Are you using your ESP32 in WiFi mode? Are you allowing for the extra current requirements during session times?
Is the ESP32 suitable for battery operation at all? See the discussion elsewhere in these forums regarding the letterbox monitor, and other threads regarding ESP sleep modes and low power operations.