I made the integration and I don’t have batteries.
There’s really nothing to investigate within the integration as far as connection errors like that go. When you try to send a command it tries to open a Modbus/TCP connection using the pymodbus library and if it can’t connect it will return the “connection failed” message.
You could go lower level and use wireshark to sniff packets to see what’s happening on the wire between Home Assistant and the inverter.
There is a difference between reads and writes that I have some retry logic on reads to try and avoid unavailable state, but writes only get one attempt and immediately return error on failure. I could extend retries to writes and see how that goes, my concern that it could cause UI problems in HA with delayed responses if there is ultimately an error. Or maybe it will be fine, i don’t know since I can only test writes with my simulator which is more well behaved than the real inverters are.
I also recommend running HA 2023.11.0 or higher or Release v2.4.7-pre.1, either of which will load dependencies for the latest version of pymodbus (3.5.4 as of this writing).
That’s a great work, thank you! I have recently installed SE8K-RWB48 with a battery and have a quick question on AC_POWER values.
Does Inverter AC_POWER (40083 40084 1 I_AC_Power int16 Watts AC Power value) ever go into a negative value same as Inverter DC_POWER (40100 40101 1 I_DC_Power int16 Watts DC Power value) does when charging battery from grid and there is 0 PV production?
It doesn’t seem to be the case for me. While charging battery from grid, AC POWER stays at 0, while DC power is negative. I can see some conditions assume it can go negative: https://github.com/ryanm101/hasolarcfg/blob/2b86032adfff5115df272fdf7fc3184d5c5e7821/packages/electricity.yaml#L162
I have the latest software on the inverter. I had previously a regular SE8K without battery so couldn’t observe a negative value due to lack of battery.
Okay, thanks for clarifying. I guess I need to work out why it’s not connecting when a command is sent on those two specific occasions. Is there any way of finding out what the command was that was trying to be sent?
I suppose that’s my other option, in case it’s the ethernet cable causing the issue. I did try changing to a different port in the switch, but that made no difference.
I’m largely able to predict when it’s most likely to happen based on historical data, so I just try to avoid sending any commands around those predicted times.
I shall investigate further.
Hi @WillCodeForCats,
this all would be no problem if the sensors would return to their normal state as soon as the read/write becomes available again, but as soon as they become unavailable, they will stay (because the sensor cannot convert “unavailable” to a number I guess). I am not good enough to provide a solution, but would it work to catch that “unavailable” case within the sensor definition and then just set the sensor to it’s most recent value?
Write errors never cause an unavailable state.
Reads have to fail multiple retry attempts where each error increases the retry time before the next attempt before being declared unavailable. So if it’s failing an update cycle it’s really failing multiple read attempts.
If you want to try changing this behavior you can edit the file solaredge_modbus_multi/const.py and increasing the values in class RetrySettings:
class RetrySettings(IntEnum):
"""Retry settings when opening a connection to the inverter fails."""
Time = 800 # first attempt in milliseconds
Ratio = 3 # time multiplier between each attempt
Limit = 4 # number of attempts before failing
You can start with changing Time to 1000 or changing Limit to 6 or higher. Restart Home Assistant after saving the changes to the file. Debug logging will show the values being applied during retry attempts. If you figure out values that work better for you then we can see how far off they are from my defaults. The defaults are based on observed results for my dual inverter system while trying to avoid excessively high values.
2 Likes
Hi, also trying to diagnose the problem above, since I also have on average two connection problems a day. I have the configuration you mention now as follows:
Time = 1500 # first attempt in milliseconds
Ratio = 3 # time multiplier between each attempt
Limit = 6 # number of attempts before failing
The last two attempts log as follows in the debug logs:
2023-11-15 04:21:32.996 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 04:21:37.500 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 04:21:42.014 WARNING (MainThread) [pymodbus.logging] Failed to connect [Errno 111] Connect call failed ('192.168.178.28', 1502)
And:
2023-11-15 18:18:27.998 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 18:18:32.502 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 18:18:40.007 WARNING (MainThread) [pymodbus.logging] Failed to connect
I did restart Home Assistant - but I don’t really see 6 attempts being made - and also the retry-times don’t really appear to match. Anything I am doing wrong?
I am interested in this because I am trying to set my Battery Discharge Power to 0 so my heat-pump does not suck my home battery dry in the morning while my power prices are still cheap - and then has no power left when we get into the expensive time.
Even though to be honest, my suspicion is now that the inverter resets itself between 4:00 and 5:00 - and with it looses the parameters I have set before - so rather than fixing the connection problems, I would rather need to work around the reset (e.g. by triggering an action when the battery starts discharging when it should not according to my automation).
Those aren’t debug logs for the integration, those are pymodbus log entries.
Debug log entries for retry attempts look like this:
2023-11-15 11:11:13.740 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 20.4
2023-11-15 11:11:16.742 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 11:11:16.743 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #1
2023-11-15 11:11:16.743 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 800 ms before data refresh attempt #2
2023-11-15 11:11:17.545 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 20.4
2023-11-15 11:11:20.546 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 11:11:20.547 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #2
2023-11-15 11:11:20.547 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 2400 ms before data refresh attempt #3
2023-11-15 11:11:22.949 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 20.4
2023-11-15 11:11:25.950 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 11:11:25.950 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #3
2023-11-15 11:11:25.950 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 7200 ms before data refresh attempt #4
2023-11-15 11:11:33.152 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 20.4
2023-11-15 11:11:36.153 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-15 11:11:36.153 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] No more data refresh attempts (maximum 4)
2023-11-15 11:11:36.154 ERROR (MainThread) [custom_components.solaredge_modbus_multi] Error fetching SolarEdge Coordinator data: Modbus/TCP connect to 10.0.0.97:1502 failed.
2023-11-15 11:11:36.154 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Finished fetching SolarEdge Coordinator data in 22.414 seconds (success: False)
Yes, I was hoping I could spare my poor SD-card all the logs, since the pymodbus log entries are caused by your integration and do contain a timestamp. Oh well, I have now enabled the logging, I think tomorrow morning I will know more.
Interestingly, your disconnection times are quite close to my own. Mine happen at around 1800 and 0400 plus or minus 10 minutes. I’ve actually created a few template sensors to “predict” when this is most likely to happen based on when it last happened.
I’ll be interested to see what comes up in your debug logs.
My inverters go offline at least once overnight, sometimes twice. They will refuse connection, eventually succeed on one of the retries and show status OFF, then go back to status SLEEPING.
Also, on timing, pymodbus itself has a 3 second timeout on connect. So there’s my retry timing and then there’s the pymodbus timeout on top of that. That is set in class ModbusDefaults in the same file.
Here are the results of my log-experiments. At the beginning of the experiment, around 00:30, I set the Storage Discharge limit of my battery to 0 - this also shows up in the logs:
2023-11-16 00:30:31.505 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.number] set SE10K-RWS48BEN4_7E062AB0_storage_discharge_limit to 0.0
This is also confirmed in the regular debug output:
2023-11-16 00:30:32.678 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] Inverter 1: discharge_limit 0x0
At 4:19, the connection problems happen:
2023-11-16 04:19:02.937 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 10.8
2023-11-16 04:19:05.939 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-16 04:19:05.940 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #1
2023-11-16 04:19:05.940 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 1500 ms before data refresh attempt #2
2023-11-16 04:19:07.442 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 10.8
2023-11-16 04:19:10.445 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-16 04:19:10.446 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #2
2023-11-16 04:19:10.446 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 4500 ms before data refresh attempt #3
2023-11-16 04:19:14.948 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 10.8
2023-11-16 04:19:17.951 WARNING (MainThread) [pymodbus.logging] Failed to connect
2023-11-16 04:19:17.952 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #3
2023-11-16 04:19:17.952 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 13500 ms before data refresh attempt #4
2023-11-16 04:19:31.453 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 10.8
2023-11-16 04:19:31.456 WARNING (MainThread) [pymodbus.logging] Failed to connect [Errno 111] Connect call failed ('192.168.178.28', 1502)
2023-11-16 04:19:31.457 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Failed data refresh attempt #4
2023-11-16 04:19:31.457 DEBUG (MainThread) [custom_components.solaredge_modbus_multi] Waiting 40500 ms before data refresh attempt #5
2023-11-16 04:20:11.959 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] coordinator timeout is 10.8
2023-11-16 04:20:12.000 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] len rcount : 40 40
2023-11-16 04:20:12.028 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] len rcount : 4 4
2023-11-16 04:20:12.162 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] len rcount : 2 2
2023-11-16 04:20:12.181 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] len rcount : 4 4
2023-11-16 04:20:12.202 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] len rcount : 2 2
Interestingly and unexpectedly for me, pymodbus reports failure before the retry loop is even finished, no idea if this is expected behaviour. Connection is back at 4:20.
And of course, immediately afterwards in the first status message after the connection works again, my discharge settings are already gone:
2023-11-16 04:20:12.252 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] Inverter 1: discharge_limit 0x459c4000
Just to make sure, I checked the last log before the connection failure - and it is still set to 0 before:
2023-11-16 04:18:33.105 DEBUG (MainThread) [custom_components.solaredge_modbus_multi.hub] Inverter 1: discharge_limit 0x0
My hypothesis now: the inverter resets itself twice daily. During that time, the connection cannot be established, until the reset is finished. And: the inverter drops some settings that it considers non-essential to default settings (such as the discharge limit).
I talked to the Solaredge support today and they also confirmed that the inverter resets twice daily at those times - and that this is expected behaviour (even though I don’t trust this 100%, because I had asked the guy about my other problem before, which clearly went over his head and he was not capable to answer any Modbus-related questions - and was probably trying to get rid of me at this point).
Last finding: I had one more warning in the log afterwards that I had not seen before:
2023-11-16 05:00:42.677 WARNING (MainThread) [homeassistant.helpers.entity] Updating state for sensor.solaredge_b1_state_of_energy (<class 'custom_components.solaredge_modbus_multi.sensor.SolarEdgeBatterySOE'>) took 0.434 seconds. Please create a bug report at https://github.com/WillCodeForCats/solaredge-modbus-multi/issues
My working assumption: if my hypothesis is correct and the connection drops are caused by the inverter resetting, of course I can try fine-tuning connection-values all I want but the maximum I will be able to achieve is get rid of the error message in the logs. The loss of my discharge-setting is however caused by the inverter-reset and not by the loss of connection - and I would need to react accordingly in my automations, e.g. by trying to catch the reset in some kind of trigger that then resets the discharge-value to my desired values. Unless of course the integration could also somehow notice the reset (e.g. by finding out that its internal state of certain states and the values it gets from the inverter is no longer consistent - which appears to cause the last warning).
Thanks for listing to my TED-talk 
- and sorry for putting this in this thread, but it appears to be the most active thread for discussions on this Modbus-integration.
1 Like
I think it’s actually the reverse. The connection drops due to the inverter resetting.
That’s really interesting to read. I share your scepticism on this being expected behaviour, but as it is happening to more than just your inverter, it seems a plausible explanation. I have 2 inverters in my system and both reset at the same time twice a day.
I read through the Solaredge Power Control Open Protocol for Inverters document today to see if I could find any other indicators that might explain this behaviour. Based on my own observations from the logbook, all I could find in relation to Inverter Grid Control was this:
Power Control Block
This block enables the grid control functionality. The power control settings are operative only if the grid control functionality is enabled.
This functionality is disabled by default and should be set by your installer. This in itself is not earth-shatteringly important in determining the reason for inverter reset, but, the Power Control Block describes the range of settings in relation to inverter active power, reactive power, power factor and grid power frequency. In my own case, our grid voltage is quite high and ranges from 246V - 253V.
Grid frequency also fluctuates quite a bit
Finally, power factor also varies quite a bit
Now this might be perfectly normal behaviour, but I suspect these fluctuations might be the cause of the inverter reset. Of course, I have nothing to compare this against so I’d be interested to see how your grid supply behaves, if only to discount this from the investigation.
For now, I do exactly as you suggest above and try to pre-empt this behaviour by avoiding predicted reset times.
These are straight timeout failures; you can see they’re 3 seconds after the coordinator timeout is X message. The inverter is not responding at all so the connection attempt times out.
This is an active rejection of the TCP connection; it happens without delay after the coordinator timeout message. At this point the inverter is “up” but not accepting connections.
Last year I was having an issue where my inverter would reset (crash) every time a threshold was exceeded when I had set an export limit.
Some of these ‘advanced’ features don’t get broad usage so you can identify corner cases that slip through testing.
Anyway after a firmware upgrade the problem hasn’t reappeared for the last 12 months so it seemed to have fixed it, but of course nothing mentioned in the release notes.
You might want to try different firmware version?
1 Like
Worth a try I guess.
There’s another constant in class ModbusDefaults in const.py that anyone with write failures might want to try, although it will only work with “short” errors not the very long 60+ second inverter resets.
For values of ReconnectDelay:
ReconnectDelay = 0 # Minimum in seconds.milliseconds before reconnecting.
Try setting a value other than 0 such as ReconnectDelay = 0.1
However keep in mind that I disabled (set to 0) this feature of pymodbus because it didn’t work properly with Home Assistant async and so I’ve left it disabled ever since.