Instructions to install Py-spy on HAOS

Community Guides
1

originally landed at How to install Py spy on a HA OS instance, please instruct - #44 by Mariusthvdb but this might be valuable to post as a standalone community guide. thanks to the relentless support by @bdraco, I can post the solution to original request for

instructions to install py-spy on HAOS (you need to run it on the host):

2 ways to install and run the required files:

1 The easiest way to run docker commands required for the py-spy is to install the ssh & web terminal add-on and disable protection mode then you can ( get into the host and) access the docker cli and follow the rest of the steps described below.

Important: this is the ssh add-on in the community add-ons repo, not the ssh add-on in the official add-ons repo. You cannot access the docker cli from the ssh add-on in the official add-ons repo.

2 On the other hand, you won’t be able to c&p commands (and results) from those terminal windows.
thats the main reason I run ssh in external terminal: to be able to c&p results

For that use the Debugging the Home Assistant Operating System | Home Assistant Developer Docs
method to create authorized keys and copy that to your instance via
ha os import on an attached usb

So you can shell into the Host, which is required for these readings.

ssh [email protected] -p 22222

After having settled the above, you can now continue with the actual setup process of the py-spy files:

docker exec -it homeassistant /bin/bash

next, download and install the correct image for py-spy for your hardware:

For the RPI4:

wget https://github.com/benfred/py-spy/releases/download/v0.3.14/py_spy-0.3.14-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
unzip py_spy-0.3.14-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

For the intel based use:

wget https://github.com/benfred/py-spy/releases/download/v0.3.14/py_spy-0.3.14-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl

instead.

There should be a py-spy binary once you call unzip in one of the directories it creates

run top (to get pid for homeassistant) (used below as --pid 60)

go to the created dir in your /config folder using

cd py_spy-0.3.14.data

cd scripts

Run the next commands for the actual py-spy readings to be analyzed by dev’s:

(in below commands, replace ‘60’ with the actual pid you got above using top)

./py-spy top --pid 60

./py-spy record --duration 120 --rate 100 -o homeassistant_$(date +"%Y_%m_%d_%H_%M").svg --pid 60

./py-spy record --duration 300 --rate 200 -o homeassistant_$(date +"%Y_%m_%d_%H_%M").svg --pid 60

./py-spy dump --pid 60

Thanks Nick!

alternative commands

as pointed out by @armin-gh in this post, we can also directly issue the command including the correct PID automagically, by using:

py-spy dump --pid $(pgrep -u homeassistant hass )

this is even easier, as it doesn’t not require the extra steps of running top first, noting the pid, en stopping top again.

just replace the actual number (so 60 in my examples above) by $(pgrep -u homeassistant hass )

3 Likes
2 
bash-5.1# wget https://github.com/benfred/py-spy/releases/download/v0.3.12/py_spy-0.3.12-py2.py3-non
unzip py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Connecting to github.com (140.82.121.4:443)
Connecting to objects.githubusercontent.com (185.199.111.133:443)
saving to 'py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl'
py_spy-0.3.12-py2.py   4% |******                                                                   py_spy-0.3.12-py2.py 100% |****************************************************| 2714k  0:00:00 ETA
'py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl' saved
Archive:  py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
  inflating: py_spy-0.3.12.dist-info/METADATA
  inflating: py_spy-0.3.12.dist-info/WHEEL
  inflating: py_spy-0.3.12.dist-info/license_files/LICENSE
  inflating: py_spy-0.3.12.data/scripts/py-spy
  inflating: py_spy-0.3.12.dist-info/RECORD
bash-5.1# cd py_spy-0.3.12.data
bash-5.1# cd scripts
bash-5.1# ./py-spy top --pid 63
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: ParseIntError { kind: PosOverflow }', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-maps-0.2.1/src/linux_maps.rs:81:65
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: receiving on a closed channel

bash-5.1#

need help

3

@morpheus8888 I can confirm I’m also getting this error.

Have you come across a solution?

Thanks

4

A few notes:

  • with the community add-ons repo it’s possible to remotely log in from an SSH client also with username: and password: if they are specified in the add-on config. To disable protection it’s not enough to just toggle the switch, you also need to restart the addon.
  • it’s worth getting the latest version of py_spy from Releases ¡ benfred/py-spy ¡ GitHub
  • to get the correct pid of the HA process you should run first ps or top and look for the pid of python3 -m homeassistant --config /config. In my case this was 66, so I had to use the commands above with 66 not 60.
  • if the funky-named record file 66-2023-03-02T20:37:26+01:00.svg is not visible on your smb share you should rename it to something more natural, with cp '66-2023-03-02T20:37:26+01:00.svg' 66-1.svg or better add an output filename option like -o homeassistant_$(date +"%Y_%m_%d_%H_%M").svg to the command.
  • to continuously grab data in 120s chunks while waiting for something to happen you could do something like while true; do ./py-spy record --pid 66 -o homeassistant_$(date +"%Y_%m_%d_%H_%M").svg --duration 120 --rate 100; sleep 1; done but keep in mind you need to press Ctrl+C very quickly twice in order to escape from the endless loop.
5

on the community add-on:

I don’t use that, as mentioned, because we can not copy and paste, which is a real nuisance. Hence my preference to use an external Terminal. Much better solution, also, if your instance might not be reachable via the frontend for some reason, you can still use the terminal window (if setup properly ofc, for port 22222, see linked instructions to do so)

thanks for your additional observations. havent run it in a long time. the pid check was already in the above guide:

top (to get pid for homeassistant)

not sure if that should be emphasized a bit more.

anyways, cheers!

6

thanks,
Ive edited the main post a bit, so this is in the commands people can simply c&p

7

not sure if this is possible with all installation methods, but if the pgrep-utility is available you can query thd pid dynamically in the command for py-spy like:
py-spy dump --pid $(pgrep -u homeassistant hass )
here looking for processes named “hass” run by user homeassistant

Armin

8

thanks, I’ll give that a try.

but, how is that easier than checking top ?`

or, would this incorporate the correct pid directly into the other commands, is that what this does

9

yes, it puts the correct pid directly into the py-spy command ( or any other command).
pgrep -u homeassistant hass
just returns the pid, perhaps a little bit better than using top, as you do not have to know where in top to look for the pid of the process
using this with $(…) puts the output of the command directly as a parameter into any command in the shell

py-spy dump --pid $(pgrep -u homeassistant hass )
dumps all threads for the homeassistant process and you don’t need to know how to find the pid

Armin

1 Like
10

cool, I’ve updated the instructions above with this method. (left the original as reference, because as you said, not being sure this works for all platforms)

Great contribution, thanks!

11

I get “no such file or directory” for this command.

12

Added “wget” before the URL.

1 Like
13

Thank you. I think I may have solved my issue though. Will see if I need it in the next 24 hrs.

14

thanks,
out of curiosity: how could you edit my post?
is this because of it being a community guide anyone can edit?

15

Yes. Read the category description:

Community Guides

The Community Guides section is a place to share guides/tutorials with our community. Every post/topic in this section works like a Wiki and can be edited and improved by anybody.

Please note, guides provided in this section may be outdated/broken and are not supported by Home Assistant. Use these at your own risk.

1 Like
16

So that saves py_spy-0.3.14-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl in /config, now what?

I see no compressed file.

17

The whl-File is the compressed file

mkdir pyspy
cd pyspy
unzip ../py_spy-0.3.14-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl
cd py_spy-0.3.14.data/scripts

from the scripts folder you can execute pyspy with
./pyspy

Armin

1 Like
18

Thanks Armin.

I did not actually solve my issue and it occurred again just now. So just successfully ran py-spy for 120s.

Following Marius’s instructions it created an SVG file and dumped a bunch of info to my screen.

From memory the last time I did this it produced data files that Nick could interpret. I can’t find those anywhere. What should I send to Nick for analysis?

19

Perhaps you are looking for the “–format” parameter of the record command? Did you produce raw format?
py-spy record --help
I guess it depends on your problem? When I used py-spy I had a something leaking threads, used “pyspy dump” to monitor the threads and a command line entity to monitor the number of threads, my system was getting unstable after having 100s of threads created … at the end a custom integration was identified.

Armin

20

Maybe I was thinking of the Profiler integration?