If you read this, what is it that gives you the impression that i don’t know what kind of cache you a talking about, and that we refer to in this topic ?
What happens to your cache if you reboot your system ? ( or 50000 met.no users does ) after an update or similar ? … i know what happens to “my” kind of cache
Pretty much everything in that paragraph. And please don’t take this as an offense again (it isn’t), but you are really misunderstanding what caching means here in the context of this service call. It is completely unrelated to what you are thinking of. The type of cache we’re talking about here cannot break, there cannot be a failure in communication protocol (because there isn’t any) and the cache is always 100% reliable (it’s literally just a bunch of memory). The only way things can fail is if there’s a bug in the integration code (duh).
Alright, I’m going to stop discussing this now, as this is completely OT. There are tons of resources about local data caching strategies on the web, if you are interested.
Fair enough, i will definitely be aware of not taking advises or believing it what you write in future, you dont seems to be well informed or have much experience in “high” reliable server solutions ( atleast not when it comes to the use of cache)
I see here 1 likely schenario (not the answer to the question i asked you) but the 50000 native met.no instances , which is a free service, could become a paid service, or HA-Users… get banned
Beside automations, template wont work, even cards shows noting etc. before a new API call is made to the Provider. With the data stored(stored is not cached), the integration-script-template will work as it can re-fecth it from “storage” until it’s either “stalled” data ( API-call is required) , else the Integration don’t have to make an API-call before next “regular” “scheduled-update”
EDIT: Sorry , i mixed up met.no with weather-integration 
, but try to understand the context, and read advises upon the use of cache in systems like HomeAutomation or similar systems which requires/should have a “backup” strategy, when using cache, a strategy which DONT involve API call as only choice( it’s not a backup ) “diskstorage/DB storage” of Data is, to avoid to many API calls, in mentioned scenarios, and i can come up with a few more ( but i doubt you will understand )
The Meteorologisk institutt (Met.no) integration was introduced in Home Assistant 0.79,
and it's used by **73.2% of the active installations**. Its IoT class is Cloud
Wonder where this number comes from, dont seems right ( must mean “it’s included in the 2 solutions which uses Supervisor” )
EDIT2: Fore some reason i had to reboot, maybe a thunderstorm cause some “interference” (PS: it’s just 1 out of many scenarios) When i got HA up again, it turns out the internet-line is “dead” , and obvious my cached data is gone, However some “important” stuff is entirely “Local” controlled (Hint the reason i felt for HA).
But the cached Weather.data is gone ( I mean 24 hours ahead/7 days ahead ) because someone decided that this is not worth “storing” Local ( or whatever reason ) … before or maybe still some only saw weather.forecasts as "“nice info”, but More and More people using these forecast and mentioned alot in this Topic, So one way or the other quite some people Have To, find a solution to store this anyway,
What would be more Appropriate is to build in an option , or make the system/function so that such data is/can be stored and access easily It’.s very little data we are talking about for the 2 arrays “hourly/days” ( likely way less than 1MB) and it’s updated regularly (if things work) Meaning the Database wont grow, because of this.
And thing depended upon this data, wont break/fail as easy as if it’s only cached data, and in many cases can feel safe ( for awhile ) for the functionally which can run without cloud.connection.
So one way or the other alot, AND alot more People in the future, needs a solution to store this data local , so this should be buildin in the HA-system and the functionality of the integration … As mentioned (if you or other missed it “Cache-Storage-API Call” , integration request the data and store it locally and why not, in cache at the same time 
, so after an ie reboot or for other reasons, integrations will load it into cache, obviously it will notice that “some” data are "stalled/over due " ( maybe could dump the stalled ) and try to perform an API-call, but as there is no internet, it failed ( warning ), if it keeps fail ( warning warning ) < PS: Just as example , … but this way the system buys the user/system some time, maybe even days, as the data was recently updated, so still i.e 23hours/ 7days stored in DB/File, and things that in other ways are not depended upon cloud, will still works ( as it actually does for many things in My HA )
My Cameras, some automations/templates etc, All my Zigbee devices, the “important Switches” etc.
Obviously not for the automation/template, related to “weather-forecasts” like outside temperatures/current temperatures(cording to weather-integration)
but as i said “Zigbees” and important switches works, but i really thought/hoped that weather-integration would be a “reliable” integration some day, as met.no is quite accurate here
At the end of the day it is not relevant what is discussed in this thread or if we want winter as summer or free beer for everyone in the world. If the core team wants it that way, it will change as they have planned and/or decided. The decisions are made somewhere else. Sometimes visible inside ADRs sometimes somewhere else.
So it’s doing the exact same polling (not just using the last response if it’s recent). As in: It doesn’t mean the API is called less.
In fact I think it might be called more. Right now we can disable types of forecasts and the integration doesn’t have to fetch them, but now it has to constantly poll every type of forecast. And of course, now you need to use a less intuitive way to get it.
Welp.
Right now, yes. But this could be changed with a trivial update to make them on-demand. Which wasn’t possible before.
No, why ? Disabling them would yield the exact same result as before. The service call would just return an empty data set if called with disabled forecast.
As noted above, this can be made very intuitive if done right.
Sadly this is so true. Anything we discuss here goes directly into a black hole either way. Oh well.
No. We can’t disable forecasts anymore because the weather entity only represents the current weather. This change completely removes forecasts from the entity registry.
Sure you can, the configuration can be made part of the integration configuration like it already is for some of the weather integrations.
This topic’s discussion attracted the attention of emontnemery who went on to assist allenporter to implement actions in a Trigger-based Template Sensor. This new feature simplifies the process of getting a weather forecast (and calendar events) into a Template Sensor.
In version 2023.8.0, the naming of entities created via MQTT Discovery was modified. The change also appeared as a warning message. Discussion of the messages, and what should/shouldn’t be done in response, caused so much confusion that the warning messages were removed in 2023.8.4.
The announcement of the discontinuation of official support for the Home Assistant Supervised installation method caused a long and heated discussion that ultimately led to its continued support but exclusively for one operating system (Debian).
There are other examples but the point is that discussions here may not reverse decisions but they can influence them, to the point of making compromises and accomodations.
Having said that, I imagine members of the development team skip, or skim, the posts involving bickering or ‘wall of text’.
There’s quite a lot of chat in this thread by now, and it’s hard to keep up with it all.
What weather forecast related use cases are still significantly more cumbersome, or perhaps downright impossible, to implement without weather forecast as a state attribute with the merge of @allenporter’s PR?
@FloatingBoater has an example of building a natural language string from a weather forecast, but it seems to me it’s not less difficult to build such a string if the weather forecast is in an entity state attribute compared to if it’s in the response of a service call.
I need to have a good look at that PR and how it works, but here’s my main concern. I use weather data for a couple of critical automations so most of my weather entities are template sensors that are layered with redundancy to ensure continuity in the data. So if one (or more) weather providers go down my sensors just default to the next preferred provider.
- name: "Outdoor Temperature"
unique_id: outdoor_temperature
device_class: temperature
state_class: measurement
unit_of_measurement: °C
state: >
{% if is_state('binary_sensor.weatherflow_cloud_connected','on')
and is_number(states('sensor.weatherflow_air_temperature')) %}
{{ states('sensor.weatherflow_air_temperature')|float }}
{% elif is_state('binary_sensor.weatherflow_local_connected','on')
and is_number(states('sensor.tempest_st_00057689_temperature')) %}
{{ states('sensor.tempest_st_00057689_temperature')|float }}
{% elif is_state('binary_sensor.envcan_connected','on')
and is_number(states('sensor.windsor_temperature')) %}
{{ states('sensor.windsor_temperature')|float }}
{% elif is_state('binary_sensor.pirateweather_connected','on')
and is_number(states('sensor.pirateweather_temperature')) %}
{{ states('sensor.pirateweather_temperature')|float }}
{% endif %}
attributes:
provider: >
{% if is_state('binary_sensor.weatherflow_cloud_connected','on')
and is_number(states('sensor.tempest_st_00057689_temperature')) %}
WeatherFlow Cloud
{% elif is_state('binary_sensor.weatherflow_local_connected','on')
and is_number(states('sensor.tempest_st_00057689_temperature')) %}
WeatherFlow Local
{% elif is_state('binary_sensor.envcan_connected','on')
and is_number(states('sensor.windsor_temperature')) %}
Environment Canada
{% elif is_state('binary_sensor.pirateweather_connected','on')
and is_number(states('sensor.pirateweather_temperature')) %}
Pirate Weather
{% endif %}
availability: >
{{ (is_state('binary_sensor.weatherflow_cloud_connected','on')
and is_number(states('sensor.tempest_st_00057689_temperature')))
or (is_state('binary_sensor.weatherflow_local_connected','on')
and is_number(states('sensor.tempest_st_00057689_temperature')))
or (is_state('binary_sensor.envcan_connected','on')
and is_number(states('sensor.windsor_temperature')))
or (is_state('binary_sensor.pirateweather_connected','on')
and is_number(states('sensor.pirateweather_temperature'))) }}
Of concern, is I do the same with my main weather entity and it’s forecast attribute.
- platform: template
name: "Home Daily"
unique_id: home_daily
condition_template: "{{ states('sensor.current_condition') }}"
temperature_template: "{{ states('sensor.outdoor_temperature')|float(-1) }}"
apparent_temperature_template: "{{ states('sensor.outdoor_apparent_temperature')|float(-1) }}"
dew_point_template: "{{ states('sensor.dew_point')|float(-1) }}"
humidity_template: "{{ states('sensor.outdoor_humidity')|float(-1) }}"
pressure_template: "{{ states('sensor.barometric_pressure')|float(-1) }}"
wind_speed_template: "{{ states('sensor.wind_speed')|float(-1) }}"
wind_gust_speed_template: "{{ states('sensor.wind_gust')|float(-1) }}"
wind_bearing_template: "{{ states('sensor.wind_bearing')|float(-1) }}"
visibility_template: "{{ states('sensor.visibility')|float(-1) }}"
cloud_coverage_template: "{{ states('sensor.cloud_cover')|float(-1) }}"
ozone_template: "{{ states('sensor.ozone')|float(-1) }}"
temperature_unit: °C
pressure_unit: hPa
wind_speed_unit: km/h
visibility_unit: km
precipitation_unit: mm
forecast_template: >
{% if is_state('binary_sensor.weatherflow_cloud_connected','on')
and state_attr('weather.weatherflow_day_based_forecast','forecast') != none %}
{{ state_attr('weather.weatherflow_day_based_forecast','forecast') }}
{% elif is_state('binary_sensor.weatherflow_local_connected','on')
and state_attr('weather.weatherflow_local_daily','forecast') != none %}
{{ state_attr('weather.weatherflow_local_daily','forecast') }}
{% elif is_state('binary_sensor.envcan_connected','on')
and state_attr('weather.windsor','forecast') != none %}
{{ state_attr('weather.windsor','forecast') }}
{% elif is_state('binary_sensor.pirateweather_connected','on')
and state_attr('weather.pirateweather','forecast') != none %}
{{ state_attr('weather.pirateweather','forecast') }}
{% endif %}
attribution_template: >
{% if is_state('binary_sensor.weatherflow_cloud_connected','on')
and state_attr('weather.weatherflow_day_based_forecast','forecast') != none %}
WeatherFlow Cloud
{% elif is_state('binary_sensor.weatherflow_local_connected','on')
and state_attr('weather.weatherflow_local_daily','forecast') != none %}
WeatherFlow Local
{% elif is_state('binary_sensor.envcan_connected','on')
and state_attr('weather.windsor','forecast') != none %}
Environment Canada
{% elif is_state('binary_sensor.pirateweather_connected','on')
and state_attr('weather.pirateweather','forecast') != none %}
Pirate Weather
{% endif %}
I’m hoping that your PR means I can create my own forecast template which will work in a similar manner and that I’ll be able to access that data in the various places I use it. Need to dig into it to wrap my head around it.
This all might seem like overkill, but I recently had to add a fourth “layer” of data as the Weatherflow API and Environment Canada both had issues at the same time which left some big holes in my data for a couple of days. It actually works very well so I hope I am able to keep this functionality for the forecast in some fashion after the changes are implemented.
I think you meant to link FloatingBoater’s example to this post. Your current link leads to a comment you made in GitHub PR 93471.
Thanks for sharing that use case!
@gjohansson has proposed a trigger-based weather template entity which if implemented should suit your use case well to my understanding.
Im very aware that my use-case more have the character of a FR, as i need the already through the remote API requested weather-data survive a reboot (regardless of why a reboot is needed/done )
Many people live outside Main-areas, on countrysides ( which in many countries is not the first priority, after a heavy storm, massive rainfall or snow etc , so loss of internet-connection occurs several times a year just do to the weather , tree falls, "distribution-points ( basically providers switch boxes around the country gets flooded, damaged etc.) … so sometimes half a day to several days, i can not rely upon Cached(only)-cloud data, it’s as simple i can say it.
PS, if the electricity ( most often ) also many times is affected, it’s always the first that comes up again, but obviously a private backup , like solar-power, or diesel-generator is a comfy thing to have 
… And no , i live in a mobile-signal “shadow” , im lucky if i can get 3G on my phones, at home(not that i think it would help when such scenarios occurs ))
I’ve been trying to follow this discussion and I may still need some clarification…
Here is an example weather entity that returns the current hourly conditions AND in addition provides 24 hours of forecast conditions as attributes:
in the situation we are discussing am I correct that all of the “forecast:” attributes would be completely removed and we would need to create template sensors and call a service to convert those (once upon a time attributes) into individual hourly sensors in their own right whose state will contain the exact same info that was previously contained in the hourly attributes?
So each sensor would have a state of (using the first “forecast:” attribute from the above screen shot as an example)?:
I know that we were admonished to not talk about the “why?” but I’m just really having a hard time seeing the benefit. Especially if the integration API polling doesn’t change and we just need to pull the already polled and existing data from a cache somewhere to actually be able to use it?
I think it’s already covered in the thread, but the whys are basically:
For the most case the forecast data is refreshed around the clock and then just thrown way, that is, if frontend is not displaying the weather forecast and no automation, template sensor or similar accesses the forecast.
The way the caching works is that forecast data is fetched from the remote API once within a time interval defined by each weather integration when the service weather.get_forecast is called.
I thought I read that the remote API is still called at the same interval as defined within the integration. So the number of API calls wouldn’t change?
The only difference being discussed is that the API data is cached somewhere and the service call would retrieve the already cached data and supply it to the state machine at that point.
But isn’t that exactly how every other entity/integration behaves as well?
I have many entities that get regular updates of data to the entities (as often as every few seconds depending on the integration/device) that I usually do nothing with so it gets “just thrown away”. But then occasionally I need to act on that data so it’s nice to have it in the state machine to be able use them in automations if needed.
I thought that was the whole purpose of the data. to have it in the state machine so as to monitor it and use if needed.
and I actually use the attribute data to be displayed in the front end in my weather cards. I guess it means to continue to use the forecast data in the weather cards I will need to create a sensor for every daily forecast that used to be in one place in the attributes of the weather entity?
but TBH, I’m not sure that the weather cards are set up to use the data from sensors. I believe the ones that I use require the data for the forecast to be contained in attributes.
Is there some huge hit in performance that has come to light that is caused by the data being moved around that is bad enough that we need to eliminate the data availability?
But also as mentioned above it seems that in order to get back the data we already have now we will need to create additional sensors that will by definition add that same data back into the state machine. But it will then have a larger impact on the DB since most users won’t set up the recorder to exclude the new sensors so all of the data that used to be in attributes (and to my understanding wasn’t included in the db) will now be written there instead.
Or am I missing something fundamental and not seeing the issues that are being caused now? I don’t think I have seen any hit in performance of my system and I regularly collect both hourly and daily forecast data from at least 3 weather integrations.
Yeah this seems completely pointless to me and is only making things more difficult for the end user.
Every time you open the web interface or the mobile app wakes, HA has to send the entire contents of the state machine to the browser or mobile to get things back up in sync. It’s impossible to avoid this because once the connection is lost, there is no way to know the system’s state without doing a full sync on reconnect. The more that’s in the state machine, the longer you will have to wait before you can interact with HA. There are knock-on effects that affect add-ons and many other places as well. Larger attributes use more disk I/O and memory, push API calls to the limits on some systems, and have recently triggered some failures/overload scenarios due to hitting maximum payload sizes.
This problem has been getting worse over time as more integrations and entities become available and are added to the system. Only so much can be done to optimize it (we have nearly run out of things we can do). Entities with significant attributes can take up as much space as 100 entities, so reducing them is essential to keeping this problem from deepening.
While throwing hardware at the problem is one solution, HA primarily targets RPi systems, so it’s essential to ensure that the HA scales well on these systems.
This makes sense and I have definitely noticed it, especially on slow mobile data connections. Definitely seems like a worthwhile change IMO
