Well, I posted some other ideas too. I don’t have strong convictions about this particular one, but without a full development of this idea or any other, it remains opinions and ideas. I’m just trying to get a conversation going that can lead to an actual solution instead of complaining.
I think I have a good reason why it’s a bad idea to eliminate the forecast attribute. It’s the limit on the number of characters in a string, which is 255. While it’s good for most forecasts, it’s insufficient for the more important ones.
My favorite weather integration will often add additional detail the more near-term the forecast is. As stormy weather approaches, the amount of detail increases and many times the forecast for the current day or night is over the entity limit. The attribute has no limit I’m aware of.
1 Like
PR merged 
Example in that PR:
template:
- trigger:
- platform: time_pattern
hours: /1
action:
- service: weather.get_forecast
data:
type: hourly
target:
entity_id: weather.home
response_variable: hourly
sensor:
- name: Weather Forecast Hourly
unique_id: weather_forecast_hourly
state: "{{ now().isoformat() }}"
attributes:
forecast: "{{ hourly.forecast }}"
1 Like
Yeah, I’m all in favor of a discussion, and if I came off as dismissive of your idea, that was definitely not my intention. Quite the contrary - my suggestion builds on top of yours.
1 Like
I’m all for finding solutions for the changes to come, but in this case it is hard to see the added value.
Weather api’s should not be called too often, so forecasts need caching as they will likely end op on a dashboard. Cynical me would say, the attribute we already have is that cache. As long as it is not stored in the db, all is fine. Users and components already should know how to get to attributes. If you replace it service calls, both users and UI components need to learn new tricks.
Requiring the integrations to add caching in favor of a cleaner weather platform design adds complexity. There are multiple weather integrations but there is just one weather platform. They will all implement a form of caching.
Services hide what information is available in the documentation. Entities and attributes data is there for any one to see. Chances are people won’t know the forecast data is there to use if you need to call a service to get it.
And guess what: some users will create a template to store the (cached) service call data in… an attribute, so they can get to it more easily. Those who don’t understand how to use services will copy such a solution form this forum, because it will get the info in a place they understand. And then we cache it twice. And if they don’t configure the recorder it it also lands in the database.
I agree we should avoid attributes when we can, but the rule should not be dogmatic and architecture choices should be made in service of user experience. In this case, we seem to have endless discussions how users should work around the new design. Users do not benefit from ever growing ways to retrieve data. That just adds confusion.
2 Likes
Yup, I had copied that example from the PR when it was still a ‘request for comment’ (up until about 2 days ago when its development was rekindled and expedited).
Now it will be included in 2023.9.0’s documentation for the Template integration:
A big thanks to allenporter and emontnemery for implementing this feature in time for the September release; it will simplify the task of getting calendar events and weather forecasts (and other things).
BTW, this comment intrigues me:
Perhaps, as a side-effect, this PR also fulfills a long-standing Feature Request:
4 Likes
Based on that, here’s a proposal:
- 1 entity for the current weather
- Each type of forecast gets an entity
- If a forecast is viewed from the UI and its data is stale, a service is called to update it
- If you want to use the forecast in a template/automation, you need to call the service to update it
The con is that the user has to create an automation to call the service, and if they don’t they might be like “why is my forecast only updating when I have HA open?”
2 Likes
I don’t think this is the proper solution (it’s something extra to set up, it pollutes the recorder, it’s just reproducing the previous intuitive behavior), and I’m scared that this will cause the HA team to reject any other solutions.
I believe it does. As a sidebar, adding a template that calls a service is not off the table, it just requires a complete rewrite of the templating engine to use async. It’s a bit harder to do at the moment because of that.
2 Likes
This is not a good solution. At least when the forecast was polled users could not set the polling interval to 1 second. And now that they can someone will do this.
5 Likes
I don’t know where you’re getting that number, but that is definitely not the limit on the length of a string that is held in memory
2 Likes
They are confusing “string” with “state value”.
2 Likes
I stand corrected. Regardless, what would be the solution for an entity longer than 255 characters?
Good point.
I’m aware of one weather integration that had set its default polling interval to a small value. The user could increase it but most did not.
It wasn’t long before the weather provider (who offers free access to its weather data) determined who was overwhelming their servers with frequent requests, tracked down the integration’s author and explained that they will block all requests from the integration until something is done to throttle them. The integration’s default polling interval was increased.
For users with complete control of the polling interval in a Time Pattern Trigger, and set it too short, they may quickly exceed their allotment of free API requests, in the case of a provider that offers tiered free and paid services, and get cut off (or automatically billed for the overage). In the case where a provider only offers free access, they may overwhelm the provider’s services causing operational problems and management headaches for the provider.
tl;dr
Please poll your weather provider responsibly.
4 Likes
So perhaps this would be the right order of actions! Have possibility to create ‘service templates’ first, then start to move towards retrieving detailed information from integrations via service calls.
Forcing users to move to unconvinient automations just to offer them more elegant solution later… not the plan I’d stick to…
Not sure if this is what you meant, but I’ll throw it in just in case. Calling the service will not poll the forecast. It will just return the latest valid data from an in memory cache. It will not trigger any cloud connection.
The actual cloud API polling is completely independent of this change, at least for now.
2 Likes
That sounds better than the way I thought it worked. 
Ideally, the Trigger-based Template Sensor should update only when the weather integration gets fresh data instead of periodically with a Time Pattern Trigger. Perhaps a State Trigger that monitors the weather entity’s last_updated or last_changed property.
What is the point of caching the values?
Just update the state/attribute if you are polling the API without the need for a trigger.
The database has been improved to the point where this should not be an issue if users don’t exclude the forecast entity.
You always need some form of local cache when dealing with remote APIs, so to decouple internal state queries from external cloud query API calls. That’s not only for HA, but generally. Imagine a local consumer thread polling the data state every millisecond, for whatever reason. You wouldn’t want to trigger a cloud call for every query. Another reason are push APIs that do not require polling, but that are internally exposed the same way as poll APIs (the weather domain supports many different implementations over the same interface).
Previously that cache was the entity attribute itself. Now it is a separate variable that is used under the hood for the service call. While this additional indirection may seem wasteful, it actually makes things a lot more flexible. For example, it allows on-demand cloud calls, so the cloud API is only queried for data you actually use. No weather imementation I (briefly) checked seems to do this yet, but it’s now possible.
In additiom to that, there’s the whole getting rid of attributes and consistency / orthogonality thing. That’s obviously a lot more subjective.
As asked before/above. So it is as before updated with the weather enetiy but only now invisible data? And you have not to call the service to get fresh data but to update the entity before calling the service to call the hidden data into a visible entity afterwards? For me too many steps for being where we are and no improvement. Or I I only don’t see the tree in the woods.
See above my questions/thoughts above. Can you perhaps answer it to get more clarifications/insights?