This is not a good solution. At least when the forecast was polled users could not set the polling interval to 1 second. And now that they can someone will do this.
5 Likes
I donāt know where youāre getting that number, but that is definitely not the limit on the length of a string that is held in memory
2 Likes
They are confusing āstringā with āstate valueā.
2 Likes
I stand corrected. Regardless, what would be the solution for an entity longer than 255 characters?
Good point.
Iām aware of one weather integration that had set its default polling interval to a small value. The user could increase it but most did not.
It wasnāt long before the weather provider (who offers free access to its weather data) determined who was overwhelming their servers with frequent requests, tracked down the integrationās author and explained that they will block all requests from the integration until something is done to throttle them. The integrationās default polling interval was increased.
For users with complete control of the polling interval in a Time Pattern Trigger, and set it too short, they may quickly exceed their allotment of free API requests, in the case of a provider that offers tiered free and paid services, and get cut off (or automatically billed for the overage). In the case where a provider only offers free access, they may overwhelm the providerās services causing operational problems and management headaches for the provider.
tl;dr
Please poll your weather provider responsibly.
4 Likes
So perhaps this would be the right order of actions! Have possibility to create āservice templatesā first, then start to move towards retrieving detailed information from integrations via service calls.
Forcing users to move to unconvinient automations just to offer them more elegant solution laterā¦ not the plan Iād stick toā¦
Not sure if this is what you meant, but Iāll throw it in just in case. Calling the service will not poll the forecast. It will just return the latest valid data from an in memory cache. It will not trigger any cloud connection.
The actual cloud API polling is completely independent of this change, at least for now.
2 Likes
That sounds better than the way I thought it worked. 
Ideally, the Trigger-based Template Sensor should update only when the weather integration gets fresh data instead of periodically with a Time Pattern Trigger. Perhaps a State Trigger that monitors the weather entityās last_updated or last_changed property.
What is the point of caching the values?
Just update the state/attribute if you are polling the API without the need for a trigger.
The database has been improved to the point where this should not be an issue if users donāt exclude the forecast entity.
You always need some form of local cache when dealing with remote APIs, so to decouple internal state queries from external cloud query API calls. Thatās not only for HA, but generally. Imagine a local consumer thread polling the data state every millisecond, for whatever reason. You wouldnāt want to trigger a cloud call for every query. Another reason are push APIs that do not require polling, but that are internally exposed the same way as poll APIs (the weather domain supports many different implementations over the same interface).
Previously that cache was the entity attribute itself. Now it is a separate variable that is used under the hood for the service call. While this additional indirection may seem wasteful, it actually makes things a lot more flexible. For example, it allows on-demand cloud calls, so the cloud API is only queried for data you actually use. No weather imementation I (briefly) checked seems to do this yet, but itās now possible.
In additiom to that, thereās the whole getting rid of attributes and consistency / orthogonality thing. Thatās obviously a lot more subjective.
As asked before/above. So it is as before updated with the weather enetiy but only now invisible data? And you have not to call the service to get fresh data but to update the entity before calling the service to call the hidden data into a visible entity afterwards? For me too many steps for being where we are and no improvement. Or I I only donāt see the tree in the woods.
See above my questions/thoughts above. Can you perhaps answer it to get more clarifications/insights?
Not too sure how to explain this in an easily understandable way, itās pretty technical. Itās a bit like virtualization: instead of natively executing code directly on your CPU, and put an additional indirection layer (the VM) between both, so to essentially do - the same thing ? Well yes, but with a lot more flexibility !
I think there are two points that come into play here:
-
Policy. They want to get rid of attributes. This has been decided a long time ago now (I remember discussions from Frenck years ago about this when I tried to add attributes in a fast way into the history explorer, and failed) and thereās probably an ADR about it somewhere. Might be related to the fact the the recorder used to handle attributes in a terrible way (and while itās much better now, itās still far from ideal) and it may be around consistency too. This is all debatable of course, but thatās the way they decided to go and we all know what that means.
-
Flexibility. Now this is where things get interesting. Virtual indirection to get data over calls rather than directly from a variable is a concept that has been used for a long time now and itās at the very heart of many object oriented languages like C++, Java, ObjectiveC, etc. Itās a very powerful concept.
When calling functions rather than just dumping data into attributes, a function (method, service call, whatever you call it) is actively informed about your attempt to get this data and can react to it. This opens a lot of possibilities, like on demand cloud queries I mentioned above. Or other things, like delaying heavy calculations (maybe some weather integration that locally calculates forecast models ? Who knows) for when theyāre actually needed.
Kinda. It depends on how the integration implements this. Thatās the flexible part of it - weather provider integrations can now do this in the most efficient way for their own cloud API, rather than simply dumping everything into attributes.
That depends on how the integration is implemented 
The way met.no seems to be doing it is - call the service to get your forecast. Call it as many times as you want, it wonāt trigger the cloud API. The actual cloud API calls are handled the same way as before, over an interval.
But the point is - thatās only one way of doing this. It allows a lot of flexibility for future optimizations.
If this feature is well implemented by the devs, most (or all) of that complexity will be hidden from you. Now obviously I donāt know what their future plans are on this, how exactly things will look in templates, etc, but this is more about structural decisions on their part. Itās not about the attribute vs service calls.
In Python (I donāt know about Jinja, I tend to stay away from that monstrosity), the difference between both could be as simple as this:
forecast = weather.attributes.forecast
vs
forecast = weather.forecast()
The only difference is the function call. But as I said, no idea if this is going to be the way they choose to do it.
You could even go further to avoid all breaking changes. I donāt know if this is possible in Python, but in polymorphic OO languages like C++, it could simply look like this:
forecast = weather.attributes.forecast
Both of them. The service call would be entirely hidden and āmagicallyā be done behind the scenes as you access the (virtual) attribute. Again, not sure if this is possible in Python / Jinja and/or if the devs want to do things that way.
Really ? , so do you mean i canāt poll a remote API and āstoreā the requested data , And let my script/templates/functions read this data from the āstoreā ? , No system should entirely depend upon a cache, unless the owner/administrator of this system/server is careless , ANY Unnecessary additional request to an remote API is waste of resources, An Integration/App should only request Data from an remote API when required, SO, if script/template/whatever request data the procedure should be, Cache/Storage/Remote-API , ā¦ anything else a big failure in common data communication-structure ( and a more fragile system, maybe even more costly if each API request cost money, which even a free might eventually turn into, because some multi-user system only use a cache, and NO internal backup(like DB/file storage for the already requested data) ā¦ We are NOT talking about 1 HA server, Met.no is cording to statistic used by roughly 25% of the installation, thats +50000 installations, ā¦ What if they ālooseā the cached data ( im SURE they WILL quite often, i.e how is it with a reboot, after an update etc. etc. etcā¦ ā¦ Having the approach of saying/thinking , Well itās just a āfastā API call away, who cares ā¦ I Donāt get it
That āstoreā is a cache.
Ahh You mean a disk-cache ? ( which wont break, or fail , OK, so now i feel safe and secure)
Nonono. A disk cache is just another very specific implementation of a general cache concept. Again, and donāt take this badly lol, a cache is not what you think it is 
If you query your cloud API, you will want to put that result somewhere so that your scripts/templates/whatever can later read this result from this somewhere and use it without having to re-query the data from the cloud everytime. That somewhere is a cache. It doesnāt matter where it is stored. In a CPU register, in main memory (thatās what the weather integration does), on a file on a disk (slow), or printed on a piece of paper and flown in a capsule to Mars. As long as this data can be retrieved (read) again without re-requesting it (eg from the cloud API), then you have a cache.
THAT was an ironic answer ! ,gee, try to read and grep the context of what people are writing, and get rid of you bad rabbit of picking out sentence, just to pick(BTW that was the only reason that i ask you above question, bc you statement is so utterly wrong) ā¦ i have worked with commercial server solutions for +15 years, I DO KNOW WHAT A CACHE IS, You might have have some more experience than me, but you definitely, have hard to try to read and understand the context of what people write
Huh ? There was nothing ironic or sarcastic there. All I tried is to explain the concept of a cache on its most basic level in the context of this weather integration. Okay then. You can always Google it I guess.
My god, I mend MY answer was Ironic
It might be a language barrier thing going on here, but Iām having a really hard time following your technical argumentation. I did not mean to offend you in any way, sorry if you had that impression. The type of caches you know (server and database related, disk caches, etc) are not the type of cache weāre talking about here. Theyāre something completely different.
Anyway I think we should leave it at this and not bring this thread further offtopic.