I think “the devs” (whichever ones they are…) are a bit thin skinned and like playing the “harassment” card a bit too easily.
Along with playing that card too easily too.
2 Likes
I’ve run up against a failure to install the py-spy package in my HA Container system following those instructions.
The error I’m getting is the same as apop did in that thread. He said he fixed it but didn’t say how.
Any idea how to over come that error?
And while I’m here I see that I need to run the “top” command to find the PID number. Can you give a full example on what the command is to get the info I need to run py-spy in the HA container? Everything I’ve found seems to already assume you know what the PID is before you run top.
Awesome thanks!
finally got it to work after the above and doing a bit more finagling.
Where would you recommend adding the .svg file I created? Here and/or in some other bug report? There’s a few of them around so I’m not sure which one would be the best
@bdraco any suggestions?
Also, Just as a follow up in case anyone else comes here looking for the same info:
I had to first add “–cap-add SYS_PTRACE” to my docker run command to enable py-spy to run without permission errors.
Next, exec into your container by either using the console in portainer or via SSH.
docker exec -it <your_container_name> /bin/bash
Once inside the container then run the following to install py-spy:
echo 'manylinux1_compatible = True' > /usr/local/lib/python3.8/site-packages/_manylinux.py
pip install py-spy
After that run top to find the PID of HA. You should see something like the following:
the one you want is the python3 program for HA (first one in the list there).
then using the PID from above run the following (replacing my PID of 205 with your own):
py-spy record --pid 205 --output /config/www/py-spy.svg --duration 120
This will run a sample and eventually write a py-spy.svg to your www folder.
The only thing I couldn’t get to work is the recoprding didn’t stop automatically after 120 seconds. I had to stop it manually by hitting ctrl-c.
Also I kept getting a continous string of warnings that the sdampling was behind and to reduce the sampling rate. I don’t know how to do that so I just let it run. I’m not sure if it impacted the accuracy of the recording or not tho TBH.
And for anyone interested here is the bug report I just submitted containing the relevant info (py-spy, screenshots)
EDIT:
And because I can’t post more than three consecutive posts in a row (silly rule…) I need to edit this post to add…
I finally ran a proper test to see what the expected latency “should be”.
I created a variable to store the last 10 results of the delta-T from the time HA sees the physical relay flip till the time that the light gets turned on.
Here are the results in v114.2:
And here are the results after upgrading back to v115.5:
Like you said - it’s pretty damning.
But I thought it would be good info to answer your question about expectations.
so, the option to control the update frequency is gone with this: https://github.com/home-assistant/core/pull/41147#issuecomment-703250497 ?
feels a bit odd that we had a fine way to control the updating of the templates, that was taken out explicitly, and adding it back is described as feature overload ? How could an option for user control be called feature overload at all? In general, but all the more now, when the system takes over (breaking) control so abundantly, since 114 versions…
3 Likes
agreed.
I wonder how many people need to report problems to meet the “until we can show that we have a larger set of affected users” requirement?
Alas, since most people don’t post those kinds of problems to the discord or github but only on the forums and the devs don’t monitor the forums very often to see the users reporting stuff (as observed by the huge number of “WTH?” threads) I don’t ever see this ever getting implemented.
1 Like
Edit: I think I was having issues because of this coupled with my old template below. The fix was to pipe the state_attr to float instead of using it as a function. See comment for old, uncomment for new.
I wonder if this change is causing the issues I am seeing with two of my templates? Although when I try them in the web-ui they say they are listening for any changes. After boot they are stuck on unknown.
My templates are:
house_power_generation:
friendly_name: "House Power Generation"
unit_of_measurement: "W"
#value_template: "{{ float(state_attr('sensor.aeon_labs_dsb28_home_energy_meter_2nd_edition_power_3', 'power_consumption'))*2.0 }}"
value_template: "{{ (state_attr('sensor.aeon_labs_dsb28_home_energy_meter_2nd_edition_power_3', 'power_consumption')|float)*2.0 }}"That’s an interesting observation. Theoretically, there should be no difference in behavior. You may want to post that as an Issue in the Core GitHub repo.
Yes, Frenck appears to have squelched development of what would be a substitute for entity_id: sensor.date (and offer even more scheduling flexibility). However, although one hand taketh, the other hand giveth. He’s currently working on a PR to allow templates to return value types other than just string.
That’s a game changer. It will make it possible to dynamically generate a list (such as for rgb_color). If implemented, it will probably be a Breaking Change but the benefit will be worth the trouble (of reviewing one’s templates).
I still want entity_id back to reduce needless processing.
3 Likes
I agree but based on the response from the development side, that’s never gonna happen.
The convenience and control provided by entity_id is gone. The currently suggested substitutes are less convenient and more complex. The proposed enhancement(s) still won’t be able to do what could be done with entity_id. It seems like a stiff price to pay for improved entity detection but it appears to be acceptable to the project’s gatekeepers.
1 Like
What’s really puzzling is that we could have had both worlds so easily. There’s been no true explanation why the current template engine cant exist with the former and suddenly abandoned entity_id. While at the same time, devs have gone into some real trouble and effort to replace it with newly created options that could not be named the same but aimed at the same goal.
Creating true python types might be mighty indeed, but that’s rather irrelevant to this issue isn’t it? Sure, it will bring us a lot of new options. But we still won’t be able to control the updating of our templates manually.
True. Just trying to brighten this gloomy discussion; some good news to look forward to (hopefully with no drawbacks)
I still believe all the consequences of the template changes were not foreseen during the design phase (it happens). With full confidence in the new method’s superiority, entity_id was probably deemed to be superfluous and so it was deprecated.
Then the proverbial $hit hit the fan and affected users’ system were crippled by excessive template evaluations. Suggestions to expediently fix the problem by restoring entity_id were dismissed. The reason offered (by Bdraco) is the new method doesn’t only touch upon Template Sensors but other things as well such as Template Triggers and Template Conditions. Therefore re-establishing entity_id for Template Sensors won’t help Template Triggers and Template Conditions.
So that brings us to the present where we await 0.116 to allow us to throttle the number of evaluations but not quite as comprehensively as with entity_id. A template that employs states and requires to be updated at the start of every day will need to be either redesigned, to exclude the use of states, or refreshed using an automation (that runs once a day, at the start of the day).
Agreed.
only hope the latest merges I saw fly by, don’t take away these options even more.
But yes, awaiting 116 eagerly, (Bdraco did a heck of a lot more than only try to accommodate need for the throttling options), and we’ll see what happens…
In the end, if we still need it, maybe a CC including the former entity_id: would be thinkable…
this the right place to thumbs up the reinstatement of the entity id? hope so. thanks.
seriously though. shouldn’t be an hour of reading to reinvent what isn’t broken.
I guess I don’t see how this will work.
If you have any entities listed in a template or all entities (by using states…) then the sensor will still update automatically on every state change that has a listener assigned.
So yes you can force an update to be a minimum of once a day by using a once-a-day automation but you can’t cause update at a maximum of no more than once a day by any automation unless the listeners themselves update less than once a day.
Unless I’m not getting something?
1 Like
Yeah I phrased that poorly. Your observation is what I have been saying all along about the deprecation of entity_id (loss of control of how and what determines template refreshes). If the template contains states an automation with a Time Trigger won’t prevent the template from being refreshed when any entity changes state. The automation will have to include the template outright and be used to set the value of a sensor (like how Marius has redesigned the “unavailable entities” sensor).
However, Bdraco is one of the most prolific contributors to the project, resolving long-standing performance problems and adding many enhancements. Based on bdraco’s investigation of this particular issue, I have every confidence it will eventually be resolved as well. It may take a version or three before it happens but the end-result is likely to be better than what we had. There’s a PR in 0.116 that performs rate-limiting for templates that use states so it should reduce/prevent the performance penalty users have reported for 0.115.
2 Likes
Yeah, I thought that was the case.
I just wanted to make sure I wasn’t thinking about it wrong since I’ve been up all night. 
1 Like