Well, let’s take some more time to see how the socket.timeout will help. If it doesn’t result in incomplete data or other weird stuff, it would be an option to wait a while and if the device is too busy just try it again in the next two minutes. Aha,… and that’s the reason why being in service mode,no data is sent to EMA.
Ok another code update. This adds some new binary_sensors: ecu_querying_enabled, and ecu_using_cached_data.
The ecu_querying_enabled indicates ‘on’ if the integration will actually try and open a socket and query the ECU, if it’s ‘off’ then it will just return cached data.
The ecu_using_cached_data will be ‘off’ when it got data from querying the ECU, but ‘on’ if it had to pull from cache. ‘on’ could indicate either the ECU is down, or you have told it to stop querying.
Which leads to the 2 new services that the integration has stop_query and start_query
You can now call these in your own automations if you would like it to start/stop querying at sunset or sunrise. Up to you since you can just create an automation as you wish.
You can now also now create some kinda alert, or pro-active power cycle. If binary_sensor.ecu_querying_enabled = on and binary_sensor.ecu_using_cached_data = on then you know there is some kind of error condition from the ECU. You may want to put in a time condition, that if both are on for 10 minutes or something - otherwise you could get in a power-cycle loop.
ooehh… sexy solution.
i had the sesnor for data age to check indeed for functioning properly over time. Now i almost have 24hours without gaps:
the peak in night is at 2AM CET… will keep this to track if there is patterns.
But since you last update, no error in log either, just warning lines. Completely empty ‘issues’ section, very happy
2021-01-13 02:05:25 WARNING (MainThread) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Error: Incomplete data from ECU after 3 attempts, cmd=‘APS1100280002216000061728END’ data=b’APS110015000202\n’
2021-01-13 02:06:40 WARNING (MainThread) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Error: Incomplete data from ECU after 3 attempts, cmd=‘APS1100280002216000061728END’ data=b’’
The previous timeouts I mentioned only happen when the inverter is online (when Zigbee signal is being received). I’ve had a connection refused around 02:45 CET, ECU unit connected with ecu2.apsema. com, ecuna.apsema. com and ecueu.apsema. com at that moment (look at my btw!). Just installed the new release. Does services.yaml belong in the same folder or do I put it somewhere else?
Ps. Early in the morning the Zigbee signal appears then disappears again to return permanent later maybe due to insufficient power from the panels?
Btw. omg! firmware version on ECU now shows 1.2.14009 (previously 1.2.13009) APSystems: show me the release notes!
ow… versions just bump? need to start looking into dns blocking those sites now 
good you still sniff the traffic, how you do that now btw to capture traffic thats not to your capturing host?
Same way you do, PiHole. Because ecu2 is not approached regularly I expect this is polling for new versions and firmware is then being pulled from the two other domains. I blocked them now. Hope WiFi on the ECU does not shut down because one off those domains is not reachable anymore.
opening that domain in browser doesnt do anything. would it attempt to ftp to it you reckon?
it does respond on ftp connect indeed. If we could trace the TCP traffic on that, username and password are easy to get.
Must be a pull from apsema not a push from their part to the ECU because ports are not open and forwarded to my ECU. What firmware do you have now?
just checked: ECU_R_1.2.14009
ftp is requiring encryption, username / password would be not easy to retrieve
User (ecu2.apsema.com:(none)): guest
530 Non-anonymous sessions must use encryption.
I am now very curious about what data is sent and how. For regular PV updates, I have such an idea that it goes via AT commands with which a TCP connection is established. But that is not my main focus, I’d like to get freed from the apsema cloud. Personally I’m not interessed in getting into their systems (and we’re way off-topic ). With at least these domains blocked let’s see if ECU’s WiFi remains active and updates are blocked untill we can see what efforts have been made to improve the product.
off topic indeed, but i’m always interested to get to internal workings of a black box. Having some visiblity in firmware would definitely help.
now with good stability on the integration handling, i again will try service mode again
service mode, that went well for 15 minutes
HA restart required to get it up again.
I see my ECU upgraded too.
Make sure to update the thread on how the domain blocking is working.
Edwin, on your question about the services.yaml, all files go into the same directory.
Sander, are you saying HA locks up in service mode? That shouldn’t happen. If so I’ll try and fix whatever bug is causing that.
My automation worked this morning and just turned on the querying with the sunrise. So that’s nice and maybe will make the ECU more stable long term (i can hope).
We use PiHole (adblocker). DNS request are being logged and passed (or not to avoid advertising). This way you are also able to block domains you don’t want to be reached. Thank you for answering my question about the file placements. Haven’t had any errors today and (great job) btw WiFi is still in the air, so blocking these three domains does no harm to functionality.
Yup, I’m familiar with PiHole, but currently not using it. Guess I have a reason to now.
I’m assuming it prevents it from logging data to the EMA site though? Do we know for sure firmware is coming via FTP? If so, I’m thinking of trying to just block FTP traffic at my firewall for that source IP. So the ECU can’t do any FTP activity at all, but still send data to EMA.
not sure on that, but it points to that if i look at it logically. I might do some tcp tracking later to see if i can get that sorted.
No, I just blocked the three four domains: ecu2.apsema. com, ecuna.apsema. com, www.apsystemsema. cn and ecueu.apsema. com. Regular updates are done to ecu.apsema .com.
If you block ecu.apsema .com then the ECU-R thinks there’s no Internet connection and disables WiFi so the integration won’t work anymore. When inverters are “on” DNS queries on ecu.apsema are made every 5 minutes, when “off” every 15 minutes. There’s AT+ traffic happening at these moments.
I haven’t had any problems so far but my experience is that ecu2 is being polled at night time round 02:00 o’clock in the early morning.
OT: apsema site is broken, that robustness is also bad
(saw something on too many files open in the stack trace… this is really bad chineze shizzels)
Everything is still working,… around three this morning the blocked domains where accessed in the same pattern as before. Still on 1.2.14009. Sent some error logging to you Kyle.
Update: No new log messages this morning, service stop/start works great. Feature request would be a “Max. Power Today”. No more wishes or remarks, all works well and seems stable now. Still very happy with this integration and work you put into it, thank you!
Just for that sake, i’m running too pretty stable. I still keep track of ‘how old is my sensor data’ and it only hiccups in the morning and recovers by itself without intervention. Didnt have to restart anything (no ECU nor HA) . I’m fine with current state of this integration.