Hi all. After years of great stability I’m experiencing crashes (unresponsiveness) daily or every couple days. I have tried to troubleshoot rending logs at DEBUG but I don’t see anything “interesting” happening at the time when HA becomes unresponsive.
What I’m hoping for help with is any other logging configurations I could try that would turn up something causing this?
It’s worth noting that the container is running on a very powerful Intel NUC with plenty of CPU and RAM and nothing anomalous shows at the time I lose responsiveness.
This is how I have my logging set:
# Configure Logging
logger:
default: info
logs:
homeassistant.core: debug
homeassistant.components: debug
Here is an excerpt of the logs with the crash coinciding with the log entry for influxdb. I suppose the camera stream handling (I have 3 Nest cameras) runs in another thread and it will continue to throw the errors like you see at the bottom:
2022-04-26 02:17:33 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection] [139830382487680] Sending {"id":9,"type":"event","event":{"event_type":"state_changed","data":{"entity_id":"sensor.asuswrt_download_speed","old_state":{"entity_id":"sensor.asuswrt_download_speed","state":"0.67","attributes":{"state_class":"measurement","hostname":"192.168.51.1","unit_of_measurement":"Mbit/s","icon":"mdi:download-network","friendly_name":"Asuswrt Download Speed"},"last_changed":"2022-04-26T07:16:33.039537+00:00","last_updated":"2022-04-26T07:16:33.039537+00:00","context":{"id":"31777531246c10e62bc4776d2bebd5cf","parent_id":null,"user_id":null}},"new_state":{"entity_id":"sensor.asuswrt_download_speed","state":"0.56","attributes":{"state_class":"measurement","hostname":"192.168.51.1","unit_of_measurement":"Mbit/s","icon":"mdi:download-network","friendly_name":"Asuswrt Download Speed"},"last_changed":"2022-04-26T07:17:33.040823+00:00","last_updated":"2022-04-26T07:17:33.040823+00:00","context":{"id":"6a85db6a5c99c71a6c69ef49f745cb56","parent_id":null,"user_id":null}}},"origin":"LOCAL","time_fired":"2022-04-26T07:17:33.040823+00:00","context":{"id":"6a85db6a5c99c71a6c69ef49f745cb56","parent_id":null,"user_id":null}}}
2022-04-26 02:17:33 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection] [139830382487680] Sending {"id":20,"type":"event","event":{"event_type":"state_changed","data":{"entity_id":"sensor.asuswrt_download_speed","old_state":{"entity_id":"sensor.asuswrt_download_speed","state":"0.67","attributes":{"state_class":"measurement","hostname":"192.168.51.1","unit_of_measurement":"Mbit/s","icon":"mdi:download-network","friendly_name":"Asuswrt Download Speed"},"last_changed":"2022-04-26T07:16:33.039537+00:00","last_updated":"2022-04-26T07:16:33.039537+00:00","context":{"id":"31777531246c10e62bc4776d2bebd5cf","parent_id":null,"user_id":null}},"new_state":{"entity_id":"sensor.asuswrt_download_speed","state":"0.56","attributes":{"state_class":"measurement","hostname":"192.168.51.1","unit_of_measurement":"Mbit/s","icon":"mdi:download-network","friendly_name":"Asuswrt Download Speed"},"last_changed":"2022-04-26T07:17:33.040823+00:00","last_updated":"2022-04-26T07:17:33.040823+00:00","context":{"id":"6a85db6a5c99c71a6c69ef49f745cb56","parent_id":null,"user_id":null}}},"origin":"LOCAL","time_fired":"2022-04-26T07:17:33.040823+00:00","context":{"id":"6a85db6a5c99c71a6c69ef49f745cb56","parent_id":null,"user_id":null}}}
2022-04-26 02:17:33 DEBUG (MainThread) [homeassistant.components.alexa.state_report] Not exposing sensor.asuswrt_download_speed because filtered by config
2022-04-26 02:17:33 DEBUG (MainThread) [homeassistant.components.nest.camera_sdm] Failed to extend stream: Error connecting to API: Cannot connect to host smartdevicemanagement.googleapis.com:443 ssl:default [Connect call failed ('142.251.45.42', 443)]
2022-04-26 02:17:34 DEBUG (influxdb) [homeassistant.components.influxdb] Wrote 3 events.
2022-04-26 02:19:12 ERROR (stream_worker) [libav.tls] The specified session has been invalidated for some reason.
2022-04-26 02:19:12 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Stream ended; no additional packets
2022-04-26 02:19:12 DEBUG (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Restarting stream worker in 10 seconds: rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
2022-04-26 02:19:22 ERROR (stream_worker) [libav.tls] The specified session has been invalidated for some reason.
2022-04-26 02:19:32 ERROR (stream_worker) [libav.rtsp] method DESCRIBE failed: 404 Not Found
2022-04-26 02:19:32 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Error opening stream (HTTP_NOT_FOUND, Server returned 404 Not Found) rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
2022-04-26 02:19:32 DEBUG (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Restarting stream worker in 20 seconds: rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
2022-04-26 02:20:22 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Error opening stream (INVALIDDATA, Invalid data found when processing input) rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
2022-04-26 02:20:22 DEBUG (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Restarting stream worker in 30 seconds: rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
2022-04-26 02:21:05 ERROR (stream_worker) [libav.rtsp] method DESCRIBE failed: 404 Not Found
2022-04-26 02:21:05 ERROR (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Error from stream worker: Error opening stream (HTTP_NOT_FOUND, Server returned 404 Not Found) rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
2022-04-26 02:21:05 DEBUG (stream_worker) [homeassistant.components.stream.stream.camera.back_door] Restarting stream worker in 40 seconds: rtsps://stream-us1-charlie.dropcam.com:443/sdm_live_stream/CiUA2vuxr2dx3ToRqH4y3wdyfdCIvmLmzuv3F6OyAjju7Fu_RluHEnEAMPcxGPr0F-RQRRIc0Go8hQuMkm8vd9qQgiGp84lE3CGpCa62unpVMmO1IZGE3M5-ygJy0Cy_U9aVv3Unw_5iP-WKkvirDmfjzaz4r2X-z1VjmvykEvYy0A5A7klpzfdd-Jch_54i_aG1ogrHJQ_9Bw?auth=****
Netdata for container at the time of the “crash” (2:17a):
