How to install Py spy on a HA OS instance, please instruct

Configuration
39

From your top:

The google assistant issue is not infrequent as I had hoped. Its pretty clear its the number 2 cpu/runtime time usage past JSON.
10.00% 10.00% 5.88s 5.93s supported (homeassistant/components/google_assistant/trait.py)
10.00% 20.00% 2.96s 9.00s <listcomp> (homeassistant/components/google_assistant/helpers.py

YAML is a close 3rd but thats just waiting on the new python version (really the JSON PR is as well)

Sadly we are back to me not having a google assistant so there isn’t much I can do for that.

40

Nabucasa should throw one at you! they’re cheap (the Google Minis), and the importance is huge :wink:

41

Might be time for some new py-py recordings now the json is lowered?

image
image1125×2436 186 KB

Btw, the listing of Powercalc here is a mystery still. I talked to the dev about this, and after some testing and debug mode we can’t see it cross the 1second startup time in the log… all entities are then up and running. Really strange.-

also seeing this:

2022-06-23 07:18:34 ERROR (SyncWorker_3) [homeassistant.util.json] Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at 
2022-06-23 07:18:34 ERROR (MainThread) [homeassistant.helpers.storage] Error writing config for core.entity_registry: Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at

and details

Logger: homeassistant.helpers.storage
Source: helpers/storage.py:280 
First occurred: 07:01:21 (1 occurrences) 
Last logged: 07:01:21

Error writing config for core.entity_registry: Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at

unfortunately, and maybe thats the reason config check only takes 1 second…? giving another restart now, #crossing fingers
update
Nope, still happening, see Issue.

Also, and not sure if this is coincidence: the Lovelace refresh doesnt work anymore…

I did find this:

Logger: homeassistant.components.websocket_api.messages
Source: components/websocket_api/messages.py:199 
Integration: Home Assistant WebSocket API (documentation, issues) 
First occurred: 07:52:43 (3 occurrences) 
Last logged: 08:34:14

Unable to serialize to JSON. Bad data found at $.event.result={'type': 'custom:template-entity-row', 'entity': 'button.symfonisk_dining_ota_update', 'state': 'BUTTON', 'secondary': '20211109-130223/v1.11.7-g682a0db'},{'type': 'custom:template-entity-row', 'entity': 'button.tester_ota_update', 'state': 'BUTTON', 'secondary': '20211109-130223/v1.11.7-g682a0db'},(<class 'homeassistant.helpers.template.TupleWrapper'>

which has never complained before, so commented that out in the view and need a Lovelace refresh, which it wont. All Json issues?

update2

wrapping those templates in [ ] makes the errors on the specific views disappear. Lovelace refresh does work, it is so FAST!!, I couldn’t believe it worked…

42

New py-spys and profiles would be great.

We should know the cause of the entity registry with tonight’s nightly. If its invalid JSON it should point you where to fix. If the parser is too strict, we can likely relax it a bit, however I suspect there is something bad that crept in that the previous parser didn’t catch.

43

consider those sent

44

thanks to the relentless support by @bdraco, I can post the solution to original request for instructions to install py-spy on HAOS (you need to run it on the host):

2 ways to install and run the required files:

1 The easiest way to run docker commands required for the py-spy is to install the ssh & web terminal addon and disable protection mode then you can ( get into the host and) access the docker cli and follow the rest of the steps described below.

Important: this is the ssh addon in the community addons repo, not the ssh addon in the official addons repo. You cannot access the docker cli from the ssh addon in the official addons repo.

2 On the other hand, you won’t be able to c&p commands (and results) from those terminal windows.
thats the main reason I run ssh in external terminal: to be able to c&p results

For that use the Debugging the Home Assistant Operating System | Home Assistant Developer Docs
method to create authorized keys and copy that to your instance via
ha os import on an attached usb

So you can shell into the Host, which is required for these readings.

ssh [email protected] -p 22222

After having settled the above, you can now continue with the actual setup process of the py-spy files:

docker exec -it homeassistant /bin/bash

next, download and install the correct image for py-spy for your hardware:

For the RPI4:

wget https://github.com/benfred/py-spy/releases/download/v0.3.12/py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
unzip py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

For the intel based use:

https://github.com/benfred/py-spy/releases/download/v0.3.12/py_spy-0.3.12-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl

instead.

There should be a py-spy binary once you call unzip in one of the directories it creates

top (to get pid for homeassistant)

go to the created dir in your /config folder using

cd py_spy-0.3.12.data

cd scripts

Run the next commands for the actual py-spy readings to be analyzed by dev’s:

./py-spy top --pid 60

./py-spy record --pid 60 --duration 120 --rate 100

./py-spy record --pid 60 --duration 300 --rate 200

./py-spy dump --pid 60

Thanks Nick!

3 Likes
45

That looks amazingly better!

Here is what is left:

Screen_Shot_2022-06-23_at_16_23_06
Screen_Shot_2022-06-23_at_16_23_062510×1150 449 KB

In order of time spent:
Unifi: https://github.com/Kane610/aiounifi/pull/145
Executor: Lots of various integrations, no standouts
Templates: https://github.com/home-assistant/core/pull/73728 :white_check_mark:
Google Assistant: I ordered one, but I think Paulus is going solve it before it shows up based on the shipping date :smiley: https://github.com/home-assistant/core/pull/73936 :white_check_mark:
Database: Looks very good, no standouts

There is normal python overhead we likely cannot reduce. Which will be better in Python 3.11 - Python 3.11 Performance Benchmarks Are Looking Fantastic - Phoronix

1 Like
46

2nd py-spy is more of the same except aio_georss_client's data parser is slow

Screen_Shot_2022-06-23_at_16_29_51
Screen_Shot_2022-06-23_at_16_29_512540×1320 457 KB

Screen Shot 2022-06-23 at 16.30.03
Screen Shot 2022-06-23 at 16.30.031920×1006 194 KB

47

hmm, what is aio_georss_client ?

might that be feedreader?

48

Looks like it comes from gdacs Global Disaster Alert and Coordination System (GDACS) - Home Assistant

49

a yes, I read that with the feedreader:

feedreader:
  urls:
    - https://alerts.home-assistant.io/feed.xml
    - https://www.home-assistant.io/atom.xml
    - https://hasspodcast.io/feed/podcast
    - https://www.gdacs.org/xml/rss.xml
  scan_interval:
    minutes: 15
  max_entries: 5

and I also have

Schermafbeelding 2022-06-23 om 23.38.07

50

No additional insights from the callgrind/profile as the py-spy has enough resolution now to show whats going on since most of the JSON overhead is gone (sans unifi which is still number 1)

1 Like
51

well has been a ride this week.

yaml check: from 30 secs to max 2!

lovelace refresh: from 5-10 secs to practically immediately

without an open browser: startup: from 2:20 to 1:10…! (and this was already so much speedier because I had a Pi4 before taking 4 min, which came down to that 2:20 on a mini pc)

any single reload option is so much faster, some even so fast I click it twice to be sure :wink:

1 thing remains a mystery:

integrations startup listing. As said, checking eg Powercalc with debug reveals it is setup and running its 236 sensors in 1 sec… yet I always see it in the top of the list:

Schermafbeelding 2022-06-23 om 23.46.11
Schermafbeelding 2022-06-23 om 23.46.111204×894 43 KB

Now Ive noticed restarts are so quick, I dont think this is a real issue, but still, its not very clear.

but first, there’s a night and tomorrow 's new nightly, so hoping we can catch those json errors in the entity_registry.

cheerio!

52

I’m traveling tomorrow so I might miss your response but feel free to ping me if it’s not bad data being injected somehow

53

Ping:

$.data.entities[1187].capabilities.available_tones<key: 1>=1(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 2>=2(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 3>=3(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 4>=4(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 5>=5(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 7>=7(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 8>=8(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 9>=9(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 10>=10(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 11>=11(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 12>=12(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 13>=13(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 14>=14(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 15>=15(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 16>=16(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 17>=17(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 18>=18(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 19>=19(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 20>=20(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 21>=21(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 22>=22(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 23>=23(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 24>=24(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 25>=25(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 26>=26(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 27>=27(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 28>=28(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 29>=29(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 30>=30(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 255>=255(<class 'int'>, $

I also see this now:

2022-06-24 09:43:20 ERROR (MainThread) [homeassistant.helpers.storage] Error writing config for core.entity_registry: Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at $.data.entities[1187].capabilities.available_tones<key: 0>=0(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 1>=1(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 2>=2(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 3>=3(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 4>=4(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 5>=5(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 7>=7(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 8>=8(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 9>=9(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 10>=10(<class 'int'>, $

and I havent got a clue what that would indicate…

found them to be the Aeotec siren endpoints, and Ive updated the issue with that info.
Apparently the core Z-wave.js integration writes incorrect Json tuples in the registry?

54

https://github.com/home-assistant/core/pull/73936 Is merged so we should be able to see a difference with tonight’s dev

1 Like
55

exciting news! thanks, (and also Paulus ofc)

will let you know and send new py-spy profiles in the morning :wink:

56

Template speed ups made it in as well https://github.com/home-assistant/core/pull/73728

The only thing outstanding is the aiounifi changes

57

sent in my latest profile/py-spy files, (though the .svg are untraceable …)

must admit startup feels really swift nowadays, but the listing isn’t that improved really and fluctuates enormously

Schermafbeelding 2022-06-25 om 10.56.27
Schermafbeelding 2022-06-25 om 10.56.271354×1198 64.4 KB

startup started at exactly 10:54, and flies by, until this:

2022-06-25 10:54:32 INFO (MainThread) [homeassistant.setup] Setup of domain script took 0.3 seconds
2022-06-25 10:54:36 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform command_line is taking over 10 seconds.
2022-06-25 10:55:00 INFO (MainThread) [homeassistant.setup] Setting up google_assistant
2022-06-25 10:55:00 INFO (MainThread) [homeassistant.setup] Setup of domain google_assistant took 0.0 seconds
2022-06-25 10:55:01 WARNING (MainThread) [custom_components.sun2.binary_sensor] Astral Above Astronomical Dawn: Sun elevation will not reach -18.000000 again until 2022-07-22
2022-06-25 10:55:04 WARNING (MainThread) [custom_components.sun2.binary_sensor] Astral Above Astronomical Dawn: Sun elevation will not reach -18.000000 again until 2022-07-22
2022-06-25 10:55:08 INFO (MainThread) [homeassistant.setup] Setting up ffmpeg
2022-06-25 10:55:08 INFO (MainThread) [homeassistant.setup] Setup of domain ffmpeg took 0.1 seconds

and then is finished.

apparently the Command-line takes longer and causes delay?

58

Strange that you can’t get a py-spy recording but can get a top. I’ve never seen that happen before.

From your top:

0.00% 0.00% 1.87s 1.96s update (homeassistant/components/file/sensor.py)

How large are the files you are using for the sensor?