Wow !
Pretty damning
(Edit: AND a well documented effect !)
1 Like
So I just checked out my cpu level for the last month and itās been static @ 1% on average. You can check out my config to compare to yours. Iām also using esphome but I only have 2 devices with ~20 entities. The event loop is tight when performing automations on it. I only have 15 template sensors and about 10 or so template entitites outside of that. Most of my stuff comes from appdaemon. My memory dropped 2k from 15.2 to 15.3. But I removed tensorflow 1.0 and added tensorflow 2.0.
Thereās a link posted by Bdraco for installing py-spy. It can reveal what is occupying Home Assistantās time and potentially help the development team to fix it. Someone suggested in a WTH that it be included with Home Assistant to make it easier for users to provide its reports when logging a GitHub Issue.
Direct link: GitHub - benfred/py-spy: Sampling profiler for Python programs
1 Like
ooh, I didnāt think about that.
Iāll see if I can get that running and maybe itāll help narrow it down.
At this point I donāt even know what to put into a bug report. Iām sure that āIām getting a bad latency on a few switchesā wonāt give them much to go on.
Iād Upvote that to become an official Add-On and a well documented addition to other install methods so that such diagnostic information being available to all (if running supervised (ie able to be monitored by the devs) and enabled)
yep, especially for those amongst us using Homeassistant OSā¦ rather difficult to setup in that.
I donāt know if it can be used as an Add-on (i.e. its own docker container). I may be wrong but it may need to run in the same context as the python program it is profiling (so in the homeassistant docker container).
Itās coded in Rust, not python, so that means it needs Rust-related resources. I donāt know how much space all of this takes but it may be a consideration when deciding whether to include it by default.
EDIT
If I have understood the following blog post correctly, it creates a docker container for py-spy and a separate container for a python test program. It then proceeds to use the containerized py-spy to profile the containerized test program. If this is true then, theoretically, py-spy could be packaged as an Add-on.
Full disclosure: Iām making a lot of assumptions ā¦
In integration then?
Either way, Frenkās your man !
Edit:
So you could āchooseā to install it (or not)
And if you could then āchooseā to allow remote monitoring, just of Py-Spy date (or not)
That way everyone gets what they want.
Long term you could look at what processor hit it causes / data bandwidth it consumes and change accordingly.
In case someone has more spare time than I do this weekend. 
If you do have a spare moment, do check this https://github.com/home-assistant/core/issues/40621#issuecomment-699510926
Seems rather relatedā¦
No, If we want this as an āOfficial Add-Onā I think it has to be undertaken by a member of the āCore-Teamā - I willing to be educated to the contrary though.
As this āmayā provide useful data to the same ā¦
Iād run it that way anyway
It certainly seemed that way but ultimately it was decided that it wasnāt the proper place to discuss it.
Moreover, the multiple posts about restoring entity_id were seen as harassing the developer (and hidden). In addition, anyone who brings up entity_id again will be punished by having their account banned from the project.
Obviously I wonāt be participating in that thread anymore.
Well how unfriendly.
My contribution was hidden too . Unjustly so, since it was spot on topic of the issue adding an extra example of a template killing the instance in the dev template editor.
Iāve asked to unhide because of that.
My ability to continue using my GitHub account to post in the project is more important to me than this one issue. Whether the decision was āunfriendlyā or not, I have no further interest in that thread.
I think āthe devsā (whichever ones they areā¦) are a bit thin skinned and like playing the āharassmentā card a bit too easily.
Along with playing that card too easily too.
2 Likes
Iāve run up against a failure to install the py-spy package in my HA Container system following those instructions.
The error Iām getting is the same as apop did in that thread. He said he fixed it but didnāt say how.
Any idea how to over come that error?
And while Iām here I see that I need to run the ātopā command to find the PID number. Can you give a full example on what the command is to get the info I need to run py-spy in the HA container? Everything Iāve found seems to already assume you know what the PID is before you run top.
Awesome thanks!
finally got it to work after the above and doing a bit more finagling.
Where would you recommend adding the .svg file I created? Here and/or in some other bug report? Thereās a few of them around so Iām not sure which one would be the best
@bdraco any suggestions?
Also, Just as a follow up in case anyone else comes here looking for the same info:
I had to first add āācap-add SYS_PTRACEā to my docker run command to enable py-spy to run without permission errors.
Next, exec into your container by either using the console in portainer or via SSH.
docker exec -it <your_container_name> /bin/bash
Once inside the container then run the following to install py-spy:
echo 'manylinux1_compatible = True' > /usr/local/lib/python3.8/site-packages/_manylinux.py
pip install py-spy
After that run top to find the PID of HA. You should see something like the following:
the one you want is the python3 program for HA (first one in the list there).
then using the PID from above run the following (replacing my PID of 205 with your own):
py-spy record --pid 205 --output /config/www/py-spy.svg --duration 120
This will run a sample and eventually write a py-spy.svg to your www folder.
The only thing I couldnāt get to work is the recoprding didnāt stop automatically after 120 seconds. I had to stop it manually by hitting ctrl-c.
Also I kept getting a continous string of warnings that the sdampling was behind and to reduce the sampling rate. I donāt know how to do that so I just let it run. Iām not sure if it impacted the accuracy of the recording or not tho TBH.