Hassio 0.102.1 just froze. How I can troubleshoot such issues?

vukisz · November 27, 2019, 5:09am

Hi,
my hassio 0.102.1 just froze before a night and didn’t switch off a heating.
How I can troubleshoot / prevent such issues? I am a bit worried, as winter is coming.

Details in 3rd post

myle · November 27, 2019, 5:14am

questions

1 is on a pi

2 is on a real PC

3 is in a VM

4 what do the logs say

5 what was the last thing you did to HA

6 we need more info the AI is still been written we need YOUR HELP to HELP you

vukisz · November 27, 2019, 5:36am

Sorry was in a panic and didn’t provide much more information.
TL;DR version:

Running on Raspberry Pi3 b+
On previous night (~11 PM) lot’s of sonoff sensors went to unavailable
Zigbee sensors also on previous night stopped reporting temperature
Yr temperature reported allright all through the nigh
I was unable to connect to Hassio until power off / power on restart this morning ~6:30
No changes was carried out on Hassio config in a last few weeks.
A week ago did additional google wifi setup, but older wifi network (which Pi and sonoffs are conected is still intact).

Details:

Yes it’s on a Raspberry Pi3 b+.
No, not on a real PC
Neither on VM
As in Hass.io->System->System log I can only see logs after a restart. In general logbook I can see, that it had some strange activity at arround 11:10-11:17PM yesterday. Lot’s of sonoff switches reported as unavailable, then turned on of and again and finnaly changed to unavailable. Then at 12:00AM date sensor changed to 2019-11-27. At 3:30AM heating towel turned on. And no more entries in logbook. Zigbee temperature sensors stopped updating at 10:54PM yesterday (reported same temperature over all night before I did a manual restart). From thermostat it reported as no temperature at all. Any other loogs worth analyzing?
I have not done any additional configuration for a couple of weeks now. Did an update to 0.102.1 (also deconz+configurator plugin updates) on 2019-11-24. But it froze only on a night from 2019-11-24 to 2019-11-25.
Yep I did a mistake with not providing information in advance. Sorry again.

I have conbee module with deconz plugin and have 4 zigbee temperature sensors, ~6 zigbee on off siwtches, ~5 sonoff switches with tasmota FW.
Installed google wifi at home a week before, but Raspberry is still on older router.
Any additional information needed?

Just saw 0.102.2 update and applied it.

Temperature sensor (zigbee):

Thermostat (I am adding +1.5 for thersmostat set temperature at 23:00 every day from automations):

Reality of heating switch (zigbee):

Sonoff switch:

Meanwhile outside yr temperature sensor was reporting allright:

Memory use percent (at 6:38 I did manual restart):

bart · November 27, 2019, 5:27pm

I am having the same issue. Already did a few clean installs but after one (or) two reboots, the sd card seems to be corrupted. bought a new sd card. Same thing happend. Don’t know what now… Has is to do with the conbee?

vukisz · November 27, 2019, 5:52pm

How often do you need to reboot?
I didn’t have such freeze for a few months till yesterday

Are your rpi connected through wifi, or lan cable?
My rpi is connected through wifi, but thinking about switching to lan cable or at least changing router lease times. But I don’t have recurrent and consistent freezes to test this yet.

Sprint · November 27, 2019, 5:59pm

I’m having the same issue with hassio on a USB plugged SSD (to avoid a corrupt SD card). It’s installed on a rpi 3b with a LAN connection to a AVM fritzbox 7490 router. I’ve already shrinked the log file to avoid huge amount of data. The rpi isn’t strong enough for home assistant I think.
The strange thing is, if I can’t go to the frontend interface, I also can’t connect via SSH. Firstly I rebooted the raspberry and all worked again.
But then I figured out, that if I reboot the router, all worked then again.
But I have no idea what is the problem because the most people say home assistant is stable over weeks, months, years in 24/7 service but for me every after max. 2 days I get no connection to it.

vukisz · November 27, 2019, 6:16pm

Interesting. During today’s research, I found that somebody fixed similar issues with changing routers dhcp lease time. See this post:

Sprint · November 27, 2019, 6:57pm

The lease time in my router is 10 days (for guest network 6 hrs) but I can’t connect to the raspberry every single day.
A reboot helps. And the rpi gets everytime the same IP address so it can’t be an DHCP issue at least for me.

Valentino_Stillhardt · November 27, 2019, 7:06pm

From what I’ve saw, when HA “freezes”, it just somehow loses contact to the outter world, which is basically any device you have and therefore don’t do what it should.

A way to temporarily prevent this is to add a “ping sensor” which pings your own router every 30 seconds or so and a automation that detects if connectivity has been lost for X amount of time, which then reboots the host (your entire Pi, NUC, cat, whatever you use to run HA on).

# automations.yaml

  - alias: Check router connectivity
    trigger:
      platform: state
      entity_id: binary_sensor.ping_router
      from: "on"
      to: "off"

      # If given, will trigger when state has been the to state for X time.
      for: "00:01:00"
  action:
    - service: shell_command.backup_ha_log
    - service: hassio.host_reboot

For convenience, you can add the shell_command too, which makes a copy of the log before rebooting. That way you’ll see what has probably failed.

# configuration.yaml
binary_sensor:
  - platform: ping
    host: 192.168.2.1
    count: 5
    scan_interval: 30
    name: Ping router

shell_command:
  backup_ha_log: cp home-assistant.log home-assistant-backup.log

I’ve just implemented this, 'cause my setup also crashed out of nowhere despite running fine for literal months.
Untested, but should work

EDIT: On a further analysis I think I answered too early. It hasn’t crashed, as you can see your interface and stuff. Strange.

Sorry.

EDIT2: Code corrected

bart · November 27, 2019, 7:44pm

I have a lan connection. router is a huawei LTE router. Now I formatted the sd card (again…) I do a complete new installation now. nothing from a snapshot

Sprint · November 27, 2019, 8:05pm

Yes I can’t open the web gui after a day or so. If that is the case, I even can’t get a connection with ssh to the pi.
Only two things help me out. 1. Restart the hole pi or 2. Restart the router. Then everything works like a charm for 1 day.

vukisz · November 28, 2019, 5:37am

Try increasing lease time for 2 days for both networks. You ar having very similar issue as author (@Chris8837 ) of my linked thread above. And he did have static ip already (getting same ip address), but lease time increase helped with his issue of recurrent freeze.
As for my case it didn’t freeze last night, so my case is not recurrent.

vukisz · November 28, 2019, 5:41am

Marvelous! That’s exactly a peace of solution I was looking for!
So many new things! ping platform, and shell_command! Thanks again I will copy paste it to my HA asap.
One question from me as I don’t have any Linux experience: is it hard to make shell command destination file name with date/teime home-assistant-backup_YYYYMMDD_HHMM.log or something similar?

Valentino_Stillhardt · November 28, 2019, 11:22am

Try something like

cp "home-assistant.log" "home-assistant-backup-"`date +"%Y-%m-%d"`.log

~~%D: Display date as mm/dd/yy.~~ ← Avoid slashes, as it interferes with the filename!
%d: Display the day of the month (01 to 31).
%a: Displays the abbreviated name for weekdays (Sun to Sat).
%A: Displays full weekdays (Sunday to Saturday).
%h: Displays abbreviated month name (Jan to Dec).
%b: Displays abbreviated month name (Jan to Dec).
%B: Displays full month name(January to December).
%m: Displays the month of year (01 to 12).
%y: Displays last two digits of the year(00 to 99).
%Y: Display four-digit year.
%T: Display the time in 24 hour format as HH:MM:SS.
%H: Display the hour.
%M: Display the minute.
%S: Display the seconds.

References:

Tested, it works: