cool, I’ve updated the instructions above with this method. (left the original as reference, because as you said, not being sure this works for all platforms)
Great contribution, thanks!
I get “no such file or directory” for this command.
Added “wget” before the URL.
1 Like
Thank you. I think I may have solved my issue though. Will see if I need it in the next 24 hrs.
thanks,
out of curiosity: how could you edit my post?
is this because of it being a community guide anyone can edit?
Yes. Read the category description:
Community Guides
The Community Guides section is a place to share guides/tutorials with our community. Every post/topic in this section works like a Wiki and can be edited and improved by anybody.
Please note, guides provided in this section may be outdated/broken and are not supported by Home Assistant. Use these at your own risk.
1 Like
So that saves py_spy-0.3.14-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl in /config, now what?
I see no compressed file.
The whl-File is the compressed file
mkdir pyspy
cd pyspy
unzip ../py_spy-0.3.14-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl
cd py_spy-0.3.14.data/scripts
from the scripts folder you can execute pyspy with
./pyspy
Armin
1 Like
Thanks Armin.
I did not actually solve my issue and it occurred again just now. So just successfully ran py-spy for 120s.
Following Marius’s instructions it created an SVG file and dumped a bunch of info to my screen.
From memory the last time I did this it produced data files that Nick could interpret. I can’t find those anywhere. What should I send to Nick for analysis?
Perhaps you are looking for the “–format” parameter of the record command? Did you produce raw format?
py-spy record --help
I guess it depends on your problem? When I used py-spy I had a something leaking threads, used “pyspy dump” to monitor the threads and a command line entity to monitor the number of threads, my system was getting unstable after having 100s of threads created … at the end a custom integration was identified.
Armin
Maybe I was thinking of the Profiler integration?
Another option is to use gProfiler. It uses py spy also. Instructions are here: GitHub - Granulate/gprofiler: gProfiler is a system-wide profiler, combining multiple sampling profilers to produce unified visualization of what your CPU is spending time on.
You can generate different reports of the running threads, processes, …
Example:
I’m trying to diagnose high CPU issue on an RPi4.
I’ve disabled almost all integrations, and still seeing a solid ~65% cpu usage. (very stable - CPU usage stays at that level)
I have installed py-spy, but can’t find the PID for home assistant. top shows nothing with “homeassistant” or python or any other obvious process.
I note that for both top and htop, all the visible processes are ~0% CPU usage, yet I can see all four cores are running about 65% each. i.e I can’t see any processes actually using any CPU…yet the CPU is very busy
It appears as if whatever process is using all the CPU is hidden from me?
pgrep -u homeassistant shows unknown user.
I’m pretty lost here…I cant py-spy a process if I can’t see that process?
Here is the output from ps aux
PID USER TIME COMMAND
1 root 0:00 /package/admin/s6/command/s6-svscan -d4 – /run/service
17 root 0:00 s6-supervise s6-linux-init-shutdownd
19 root 0:00 /package/admin/s6-linux-init/command/s6-linux-init-shutdownd -d3 -c /run/s6/basedir -g 3
26 root 0:00 s6-supervise s6rc-oneshot-runner
27 root 0:00 s6-supervise ttyd
28 root 0:00 s6-supervise sshd
29 root 0:00 s6-supervise s6rc-fdholder
37 root 0:00 /package/admin/s6/command/s6-ipcserverd -1 – /package/admin/s6/command/s6-ipcserver-acc
310 root 0:00 ttyd -d1 -i hassio --writable -p 62211 tmux -u new -A -s homeassistant zsh -l
313 root 0:00 sshd: /usr/sbin/sshd -D -e [listener] 0 of 10-100 startups
415 root 0:00 sshd: redacted [priv]
417 julz 0:00 sshd: redacted@pts/0
418 root 0:00 sudo -i
420 root 0:00 sudo -i
421 root 0:16 -zsh
1013 root 0:00 ps aux
Update: I couldn’t figure out how to see the processes directly from the terminal (I guess it’s running ina docker that is hidden from me, and I don’t know how to access).
But I installed Glances, and found it is the hassio_dns process that is using 162% cpu.
I checked my DNS entry, it was correct (8.8.8.8). So I changed it to point at my local DNS server, (that all my DHCP clients use), and that still had the same issue. I switched again to 1.1.1.1, same problem.
I note the dns logs have many lines like:
[ERROR] plugin/errors: 2 . NS: dial tcp 1.1.1.1:853: connect: connection refused
Anyhow, this is no longer a py-spy issue, so I’ll refrain from commenting further in this thread, and go try and figure out this DNS issue. Just leaving this info here in case it helps anyone else track down their CPU usage issues.
This thread looks promising regarding High CPU usage from Hassio DNS
You can build it manually from this PR fix line number for python 3.12 by penguin-wwy · Pull Request #666 · benfred/py-spy · GitHub
I tried building above mentioned py-spy PR for python3.12 myself (forked dev312 branch on GitHub, and build with GitHub Actions: simplify workflow push · TimSoethout/py-spy@dcdec48 · GitHub, see wheel.zip download at the bottom.)
I used the py_spy-0.3.14-py2.py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64 build for my Odroid C2.
This results in error when run in my homeassistant docker on hassio:
$ ./py-spy top --pid 67
thread 'Error: <unnamed>' panicked at 'Failed to get process executable name. Check that the process is running.called `Result::unwrap_err()` on an `Ok` value: ()
', Reason: src/sampler.rs:59No such file or directory: (os error 492
)note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Reason: No such file or directory (os error 2)
Any pointers?
(I’m debugging this to find out a memory leak that reboots my whole device every ~3h when memory get to ~90%.)
I’m running into the same error as @TimmyBankers above, did you (or any else) manage to solve this problem?
Could really use py-spy to hunt down what is causing high CPU usage on my system…
For me downloading the latest release from Releases · benfred/py-spy · GitHub for my architecture seems to work. Unpack the .whl file using wheel unpack ....
The py-spy executable is found some folders deep, and I had to copy it to the home dir instead of /tmp in order to execute it.
(Now py-spy top hangs my hass container. 
The next think to look into.)
1 Like
Still no luck, running HAOS (latest release) on a RPI4, I downloaded py_spy-0.4.0-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl and followed the guide. However, when running
./py-spy top --pid 67 [which is the correct PID :-)] I get the following
Error: Invalid argument (os error 22). Anyone knows what it means/why it occurs?