HA crashing with enormous log files

Hi guys :slight_smile:
I have noticed that my HA is restarting randomly, not sure when it started, but it does multiple restarts a day. Always creating huge log files, 10GB+ multiple times a day so I cant even upload the full log.

There is one Error that repeats 193,661,828 times in the last log file.

2024-07-24 11:54:27.749 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback None() (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
TypeError: 'NoneType' object is not callable
type or paste code here

I have tried multiple ways to find the cause like Service profiler.set_asyncio_debug + debugpy: in configuration.yaml + dump_log_object type: RenderIngo. I have been trying to follow all the errors, which were mostly described as not being important. But there are several ones, that I cannot find much about :confused:

One critical error is this:


Logger: homeassistant.components.profiler
Source: components/profiler/__init__.py:174
integration: Profiler (documentation, issues)
First occurred: 19:17:47 (163 occurrences)
Last logged: 19:17:47

RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.load_l1')|float + states('sensor.back_up_l1_power')|float) }}) renders=260> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.load_l1', 'sensor.back_up_l1_power'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.load_l3')|float + states('sensor.back_up_l3_power')|float) }}) renders=278> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.load_l3', 'sensor.back_up_l3_power'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.load_l1')|float + states('sensor.back_up_l1_power')|float) + (states('sensor.load_l2')|float + states('sensor.back_up_l2_power')|float) + (states('sensor.load_l3')|float + states('sensor.back_up_l3_power')|float) }}) renders=800> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.back_up_l3_power', 'sensor.back_up_l2_power', 'sensor.load_l1', 'sensor.back_up_l1_power', 'sensor.load_l3', 'sensor.load_l2'}) rate_limit=None has_time=False exception=None is_static=False>
RenderInfo object in memory: <RenderInfo Template<template=({{ (states('sensor.load_l2')|float + states('sensor.back_up_l2_power')|float) }}) renders=270> all_states=False all_states_lifecycle=False domains=frozenset() domains_lifecycle=frozenset() entities=frozenset({'sensor.back_up_l2_power', 'sensor.load_l2'}) rate_limit=None has_time=False exception=None is_static=False>

I dont see anything wrong with the template, it works, all entities are working.

Logger: homeassistant
Source: /usr/src/homeassistant/homeassistant/runner.py:148
First occurred: 18:29:42 (735 occurrences)
Last logged: 20:20:31

Error doing job: Error on transport creation for incoming connection (None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 235, in _accept_connection2
    await waiter
  File "/usr/local/lib/python3.12/asyncio/sslproto.py", line 578, in _on_handshake_complete
    raise handshake_exc
ConnectionResetError

There is also this weird error I could not find anything about :frowning:
While the HA is running I can download smaller log which I uploaded here.
A would appreciate any help!!!

Greetings xxgmxx,

You said something about asyncio logs, so you may have seen this post from bdraco, but this is the recommended way to find problems like this.
2024.5+: Tracking down instability issues caused by integrations.

Also there is an updated version of asyncio coming in this months release. I heard it mentioned in another chat. That has the potential of fixing or at least better identifying your issue when it’s released. Guessing on that, BTW.

The first thing I would do is restart in safe mode. That way you can see if it is a third party integration that is the issue (these are not loaded in safe mode).

Developer Tools → YAML → Restart → Advanced Options → Restart in safe mode

Restarting again normally will re-enable your third party integrations.

Thank you guys for suggestions :slight_smile: I apreciate it a lot. The safemode made my HA running without any crash for 2 day so far. I would probably need to uninstall all the HACS components and start adding it one by one. Or is there any easier way of doing that?

I have noticed a weird pattern with the issue I am facing. If I do not interact with my HA it works. The pattern is editing my ESPHome .yaml´s, while doing that, I get the most restarts. If I don’t do anything with ESPhome it could stay without restart for days. As soon as I start editing my HA starts crashing like crazy even in safe mode. A long way to go :confused: not sure if there is something other that plays a role in that.
In case anyone would have some time to look at the log, I uploaded it here. It’s not that big (which is a nice change for now) home-assistant.log.1_new_2024_08_06
I am still not that sure what to do next, nothing seems to be showing any obvious cause.

Editing or compiling?

That is very resource intensive. You may not have enough resources.

What hardware are you using?

HA was runnimg for few days just fine and crashed 5x during editting ESPHome .yaml. I have no Idea if its just a coincidence or not.
I did not even saved the file and I could see the messace in the left bottom corner, that HA is down, and shortly after that it is starting.

I kind of feel, that ESPHome can’t be the problem but I remember, that most of the crashes happened during me being on my PC and having ESPHome opened, or it just may be because it was easy to see the error message.

I added a automation that sends me a notification when HA restarst, so I could better track it. At least the last x log files were not size of 20+GB. But I still think no-one will download file from random person, but I dont have other way of sharing bigger files.

The HW I am using is actually 3 different systems, to rule out the hardware. I tested I7-7100 + 32GB ram 500GB SSD, I7-7700 + 32GB ram 1000GB SSD, I5-5100 16GB ram 150GB SSD. Using both VMware and VirtualBox. The startup is super snappy, CPU + RAM usage is really low. Hopefully HW is not the problem.

Yeah, those are more than capable of handling ESPHome.

My system was stable for a long time, I changed nothing at all. I could not find the problem so I kept waiting for solution which somehow came by itself (Problably me not having time to do anything new, especially ESPHome stuff). Now when I decided to update some ESPHome device I noticed HA crashing every 5-10 minutes if ESPHome addon was opened in my browser.

That is crazy I might just avoid ESPHome on my HA. I have no idea why is that happening and I cant find anything related in the logs.

Fix this and see if it helps:

[homeassistant.config] Key whitelist_external_dirs has been replaced with allowlist_external_dirs. Please update your config