How to install Py spy on a HA OS instance, please instruct

Mariusthvdb · June 23, 2022, 5:14am

Might be time for some new py-py recordings now the json is lowered?

Btw, the listing of Powercalc here is a mystery still. I talked to the dev about this, and after some testing and debug mode we can’t see it cross the 1second startup time in the log… all entities are then up and running. Really strange.-

also seeing this:

2022-06-23 07:18:34 ERROR (SyncWorker_3) [homeassistant.util.json] Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at 
2022-06-23 07:18:34 ERROR (MainThread) [homeassistant.helpers.storage] Error writing config for core.entity_registry: Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at

and details

Logger: homeassistant.helpers.storage
Source: helpers/storage.py:280 
First occurred: 07:01:21 (1 occurrences) 
Last logged: 07:01:21

Error writing config for core.entity_registry: Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at

unfortunately, and maybe thats the reason config check only takes 1 second…? giving another restart now, #crossing fingers
update
Nope, still happening, see Issue.

Also, and not sure if this is coincidence: the Lovelace refresh doesnt work anymore…

I did find this:

Logger: homeassistant.components.websocket_api.messages
Source: components/websocket_api/messages.py:199 
Integration: Home Assistant WebSocket API (documentation, issues) 
First occurred: 07:52:43 (3 occurrences) 
Last logged: 08:34:14

Unable to serialize to JSON. Bad data found at $.event.result={'type': 'custom:template-entity-row', 'entity': 'button.symfonisk_dining_ota_update', 'state': 'BUTTON', 'secondary': '20211109-130223/v1.11.7-g682a0db'},{'type': 'custom:template-entity-row', 'entity': 'button.tester_ota_update', 'state': 'BUTTON', 'secondary': '20211109-130223/v1.11.7-g682a0db'},(<class 'homeassistant.helpers.template.TupleWrapper'>

which has never complained before, so commented that out in the view and need a Lovelace refresh, which it wont. All Json issues?

update2

wrapping those templates in [ ] makes the errors on the specific views disappear. Lovelace refresh does work, it is so FAST!!, I couldn’t believe it worked…

bdraco · June 23, 2022, 7:40pm

New py-spys and profiles would be great.

We should know the cause of the entity registry with tonight’s nightly. If its invalid JSON it should point you where to fix. If the parser is too strict, we can likely relax it a bit, however I suspect there is something bad that crept in that the previous parser didn’t catch.

Mariusthvdb · June 23, 2022, 9:26pm

consider those sent

Mariusthvdb · June 23, 2022, 9:27pm

thanks to the relentless support by @bdraco, I can post the solution to original request for instructions to install py-spy on HAOS (you need to run it on the host):

2 ways to install and run the required files:

1 The easiest way to run docker commands required for the py-spy is to install the ssh & web terminal addon and disable protection mode then you can ( get into the host and) access the docker cli and follow the rest of the steps described below.

Important: this is the ssh addon in the community addons repo, not the ssh addon in the official addons repo. You cannot access the docker cli from the ssh addon in the official addons repo.

2 On the other hand, you won’t be able to c&p commands (and results) from those terminal windows.
thats the main reason I run ssh in external terminal: to be able to c&p results

For that use the Debugging the Home Assistant Operating System | Home Assistant Developer Docs
method to create authorized keys and copy that to your instance via
ha os import on an attached usb

So you can shell into the Host, which is required for these readings.

ssh [email protected] -p 22222

After having settled the above, you can now continue with the actual setup process of the py-spy files:

docker exec -it homeassistant /bin/bash

next, download and install the correct image for py-spy for your hardware:

For the RPI4:

wget https://github.com/benfred/py-spy/releases/download/v0.3.12/py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
unzip py_spy-0.3.12-py2.py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

For the intel based use:

https://github.com/benfred/py-spy/releases/download/v0.3.12/py_spy-0.3.12-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl

instead.

There should be a py-spy binary once you call unzip in one of the directories it creates

top (to get pid for homeassistant)

go to the created dir in your /config folder using

cd py_spy-0.3.12.data

cd scripts

Run the next commands for the actual py-spy readings to be analyzed by dev’s:

./py-spy top --pid 60

./py-spy record --pid 60 --duration 120 --rate 100

./py-spy record --pid 60 --duration 300 --rate 200

./py-spy dump --pid 60

Thanks Nick!

bdraco · June 23, 2022, 9:28pm

That looks amazingly better!

Here is what is left:

In order of time spent:
Unifi: https://github.com/Kane610/aiounifi/pull/145
Executor: Lots of various integrations, no standouts
Templates: https://github.com/home-assistant/core/pull/73728
Google Assistant: I ordered one, but I think Paulus is going solve it before it shows up based on the shipping date https://github.com/home-assistant/core/pull/73936
Database: Looks very good, no standouts

There is normal python overhead we likely cannot reduce. Which will be better in Python 3.11 - Python 3.11 Performance Benchmarks Are Looking Fantastic - Phoronix

bdraco · June 23, 2022, 9:31pm

2nd py-spy is more of the same except aio_georss_client's data parser is slow

Mariusthvdb · June 23, 2022, 9:32pm

hmm, what is aio_georss_client ?

might that be feedreader?

bdraco · June 23, 2022, 9:34pm

Looks like it comes from gdacs Global Disaster Alert and Coordination System (GDACS) - Home Assistant

Mariusthvdb · June 23, 2022, 9:38pm

a yes, I read that with the feedreader:

feedreader:
  urls:
    - https://alerts.home-assistant.io/feed.xml
    - https://www.home-assistant.io/atom.xml
    - https://hasspodcast.io/feed/podcast
    - https://www.gdacs.org/xml/rss.xml
  scan_interval:
    minutes: 15
  max_entries: 5

and I also have

Schermafbeelding 2022-06-23 om 23.38.07

bdraco · June 23, 2022, 9:40pm

No additional insights from the callgrind/profile as the py-spy has enough resolution now to show whats going on since most of the JSON overhead is gone (sans unifi which is still number 1)

Mariusthvdb · June 23, 2022, 9:49pm

well has been a ride this week.

yaml check: from 30 secs to max 2!

lovelace refresh: from 5-10 secs to practically immediately

without an open browser: startup: from 2:20 to 1:10…! (and this was already so much speedier because I had a Pi4 before taking 4 min, which came down to that 2:20 on a mini pc)

any single reload option is so much faster, some even so fast I click it twice to be sure

1 thing remains a mystery:

integrations startup listing. As said, checking eg Powercalc with debug reveals it is setup and running its 236 sensors in 1 sec… yet I always see it in the top of the list:

Now Ive noticed restarts are so quick, I dont think this is a real issue, but still, its not very clear.

but first, there’s a night and tomorrow 's new nightly, so hoping we can catch those json errors in the entity_registry.

cheerio!

bdraco · June 23, 2022, 11:35pm

I’m traveling tomorrow so I might miss your response but feel free to ping me if it’s not bad data being injected somehow

Mariusthvdb · June 24, 2022, 7:45am

Ping:

$.data.entities[1187].capabilities.available_tones<key: 1>=1(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 2>=2(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 3>=3(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 4>=4(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 5>=5(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 7>=7(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 8>=8(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 9>=9(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 10>=10(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 11>=11(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 12>=12(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 13>=13(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 14>=14(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 15>=15(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 16>=16(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 17>=17(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 18>=18(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 19>=19(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 20>=20(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 21>=21(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 22>=22(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 23>=23(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 24>=24(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 25>=25(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 26>=26(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 27>=27(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 28>=28(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 29>=29(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 30>=30(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 255>=255(<class 'int'>, $

I also see this now:

2022-06-24 09:43:20 ERROR (MainThread) [homeassistant.helpers.storage] Error writing config for core.entity_registry: Failed to serialize to JSON: /config/.storage/core.entity_registry. Bad data at $.data.entities[1187].capabilities.available_tones<key: 0>=0(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 1>=1(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 2>=2(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 3>=3(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 4>=4(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 5>=5(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 7>=7(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 8>=8(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 9>=9(<class 'int'>, $.data.entities[1187].capabilities.available_tones<key: 10>=10(<class 'int'>, $

~~and I havent got a clue what that would indicate…~~

found them to be the Aeotec siren endpoints, and Ive updated the issue with that info.
Apparently the core Z-wave.js integration writes incorrect Json tuples in the registry?

bdraco · June 24, 2022, 10:09pm

https://github.com/home-assistant/core/pull/73936 Is merged so we should be able to see a difference with tonight’s dev

Mariusthvdb · June 24, 2022, 10:25pm

exciting news! thanks, (and also Paulus ofc)

will let you know and send new py-spy profiles in the morning

bdraco · June 25, 2022, 1:23am

Template speed ups made it in as well https://github.com/home-assistant/core/pull/73728

The only thing outstanding is the aiounifi changes

Mariusthvdb · June 25, 2022, 9:01am

sent in my latest profile/py-spy files, (though the .svg are untraceable …)

must admit startup feels really swift nowadays, but the listing isn’t that improved really and fluctuates enormously

startup started at exactly 10:54, and flies by, until this:

2022-06-25 10:54:32 INFO (MainThread) [homeassistant.setup] Setup of domain script took 0.3 seconds
2022-06-25 10:54:36 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform command_line is taking over 10 seconds.
2022-06-25 10:55:00 INFO (MainThread) [homeassistant.setup] Setting up google_assistant
2022-06-25 10:55:00 INFO (MainThread) [homeassistant.setup] Setup of domain google_assistant took 0.0 seconds
2022-06-25 10:55:01 WARNING (MainThread) [custom_components.sun2.binary_sensor] Astral Above Astronomical Dawn: Sun elevation will not reach -18.000000 again until 2022-07-22
2022-06-25 10:55:04 WARNING (MainThread) [custom_components.sun2.binary_sensor] Astral Above Astronomical Dawn: Sun elevation will not reach -18.000000 again until 2022-07-22
2022-06-25 10:55:08 INFO (MainThread) [homeassistant.setup] Setting up ffmpeg
2022-06-25 10:55:08 INFO (MainThread) [homeassistant.setup] Setup of domain ffmpeg took 0.1 seconds

and then is finished.

apparently the Command-line takes longer and causes delay?

bdraco · June 25, 2022, 10:49am

Strange that you can’t get a py-spy recording but can get a top. I’ve never seen that happen before.

From your top:

0.00% 0.00% 1.87s 1.96s update (homeassistant/components/file/sensor.py)

How large are the files you are using for the sensor?

Mariusthvdb · June 25, 2022, 11:08am

I see them in the Vsc editor but not in my samba share on the Mac …

I’ll send them over soon

I am sorry, but what does that mean…? you referring to the sensor.py file in the docker?
Ive just updated to todays dev20220625 so I guess thats the file I am using…?

about the py-spy files not showing: I mean not showing in Mac Finder where they were visible before:

but they are in Studio code server:

so I copied the content and pasted into handmade .svg’s

bdraco · June 25, 2022, 3:18pm

For the file size I’m asking about the actual size (in bytes) of the files for the file sensor

ie file_path: /home/user/.homeassistant/sensor-data.txt

The underlying code reads the whole file to the last line and gives you the result. If the file is growing without bounds it will get slower and slower over time since there is more data to read through until it gets to the line at the end.

github.com

home-assistant/core/blob/e67f8720e81e8618370aaf3e6a36b8b1553b9e9a/homeassistant/components/file/sensor.py#L80

    
      
          ) -> None:
              """Initialize the file sensor."""
              self._attr_name = name
              self._file_path = file_path
              self._attr_native_unit_of_measurement = unit_of_measurement
              self._val_tpl = value_template
          
          
def update(self):
              """Get the latest entry from a file and updates the state."""
              try:
                  with open(self._file_path, encoding="utf-8") as file_data:
                      for line in file_data:
                          data = line
                      data = data.strip()
              except (IndexError, FileNotFoundError, IsADirectoryError, UnboundLocalError):
                  _LOGGER.warning(
                      "File or data not present at the moment: %s",
                      os.path.basename(self._file_path),
                  )
                  return