[PETITION] Don't delete the forecast attribute

[boheme61]

Yes, i know, well you said above, obviously i have(had) no idea(thou always new it was the largest, among few Cameras is have, thou these have been dealt with in the Tapo-integration,a few more entities but still some attributes left there thou), and as mentioned my Homeassistant installation is not a “playground”

So far all my automations/templates function, and i will notice/get noticed if i.e a switch.x is "not in the state i Assumed, same goes for lights.x ( i see the lights on), knowing when, how often, how long, i could judge by i.e power consumption etc ( so hardly relevant for me/my system currently

Tibber_Custom is a light-weight frontend (compromise), which i have modified to my view-design

And i have already forgotten what’s included in default Recorder-settings.

EDIT: IF you take an advance Camera, and all it’s entities, you will get a total of More Datas/States/Attributes-States(than weather-forecasts, well maybe not if you call all available datas in i.e met.no API), if you then have i.e 6 Cameras in an integrations, well multiply , i can’t envy the task you “struggling” with

[KTibow (Kendell R)]

@bdraco

The weather forecasts were one of the largest attributes

I have a NWS forecast with the hourly forecast for 7 days. It’s just 31.6kb raw and only 1-2kb with compression. That’s not going to slow down HA at all, is it?
(
For comparison: Loading all JS in the frontend, with compression, excluding custom cards, is 631kb (500x the size of the hourly forecast). However both of those will load pretty quickly as LAN speeds can get pretty fast - I wouldn’t worry about 1kb when you have 1gb/s speeds.
I have my own gripes with the architecture of the frontend (way too many dependencies including gulp, uses a bunch of polymer stuff, uses the shadow dom, etc), but that’s for another thread.
)

[boheme61]

@bdraco

Some of the weather forecasts had the same space cost as hundreds of switch/sensor/etc entities

But wouldn’t the usage/latency occur when one loads data from 100 different sensors/entities, rather than i.e weather-integration reads a blob and throw it in a card ? , i know user-configured-grap,h-cards demands more “resources”, but a default weather-card ?
And “Large in size” is maybe not the right way of looking at it, in a perspective where Many default, average, standard “mobile” users loggs all, and records-all , and expects performance, no matter the circumstances, im not sure what the size of an average ( most common ) DB is among HA users, but if we say 700MB-2GB ( i think it’s low, but i dont know, mine is way below ) , then the nice, already in the API-call, formatted data of few kb, should in my opinion not “tilt” a DB size, and it never hardly grows ( As stalled data should be wiped(by command from the integration, if not immediately, then regularly in chunks)

[tom_l]

Thanks Nick, I now understand the reasoning behind t/his change.

[parautenbach (Pieter Rautenbach)]

@parautenbach

I find the service call replacement very clunky.

I’d like to retract this statement, FWIW.

After all the insights in this thread and the helpful information and answers from the core devs, I have now changed my view that this is a good way forward.

Thanks for the examples and explanations in the release notes too.

[Mariusthvdb (Marius)]

cutting down resources is always on my mind
so just to be sure:

when entities on a weather integration (eg Buienradar which provides hundreds) are disabled they are counted in the integration panel, but are in fact not loaded, and dont Not weigh on the resources?

in the current (deprecated, but still working) phase, how can we stop the weather integrations forecast attribute from weighing on the system resources as is foreseen with this change, but will only be effective in 6 months?

Id love to see the efficiency change sooner than later.
we stop the polling, use the new forecast service on the forecast and update the weather integration with an automation?

if I read

While the way to access this data has changed and may impact your automations, you will not experience any loss in functionality. Even with this change, rest assured that the forecast cards in your dashboard will still work, they will now just use the service to retrieve the forecast data asynchronously instead of reading it from the state attribute.

is that already functioning as is, so without any user action writing automations using the new service etc etc.? Only asking because they are still there…

given the fact these forecast will no longer be in the states machine, how will we be able to test our templates in the dev tools?

would the release notes example {{ forecast.forecast[0].precipitation > 0 }} output correctly

sorry if this seems beyond silly, I finally got the chance to test run 2023.9 so still trying to adjust, and obviously didnt test my question above just yet, still in the preparational phase…

edit:
well I did and got this working after adding the HA start event for initial power. Update entity does not work on any of these entities btw? because that was my first try when they weren’t populated upon reload templates, and they remain unknown.
And no, I cant test this template in dev tools, so that is a bit of an issue when writing these templates

template:

  - trigger:
      - platform: state
        entity_id: sensor.date
#       - platform: homeassistant
#         event: start
    action:
      - service: weather.get_forecast
        target:
          entity_id: weather.buienradar
        data:
          type: daily
        response_variable: buienradar_forecast
    sensor:
      - unique_id: buienradar_temperature_forecast
        name: Temperature forecast
        state: >
          {{buienradar_forecast.forecast[0].temperature}}
      - unique_id: buienradar_precipitation_forecast
        name: Precipitation forecast
        state: >
          {{buienradar_forecast.forecast[0].precipitation}}
    binary_sensor:
      - unique_id: buienradar_precipitation_forecast_binary
        name: Is it going to rain?
        state: >
          {{buienradar_forecast.forecast[0].precipitation > 0}}

[TheFes]

It might be a bit tedious, but you can test your templates in devtools
Run the service call in devtools > services and copy the output
Put that in a yaml to json convertor like: Transform YAML into JSON - Online YAML Tools

Then start the devtools > templates section with:
{% set buienradar_forecast = json output from website %}

After that you can do all the testing you want.

And trigger based template sensors are only processed on the trigger, if you want them to be processed when you want, you can:

• add a HA start trigger and restart HA
• add a template reload trigger, and reload template entities
• add an event trigger with something like event_type: reload_weather_forecast and send that event from devtools > events
• add a state trigger on a state change from a test input_boolean
• etc

[Mariusthvdb (Marius)]

@TheFes

Run the service call in devtools > services and copy the output

Thank you!
that is what I was looking for… argh, I hadnt thought of that, sorry, indeed silly, all clear now, full data set is presented just nicely.

@TheFes

And trigger based template sensors are only processed on the trigger

yes, I am aware of all the triggering options, and as described used that in the starting up config. Now only use the date to update, and as usual these trigger based templates are restored (on startup), so took out that trigger again.

What I just hadnt realized is that using the homeassistant.update_entity service does Not update trigger based entities.

Would be nice if that would made be possible.

thx!

[TheFes]

using the homeassistant.update_entity service would have the same problems as using Run automation from the GUI. It won’t have a trigger, so everything which is based on trigger variables won’t work.
And I personally have a lot of trigger based template sensors which rely on trigger variables.

[Mariusthvdb (Marius)]

yep, you’re right on all accounts. printed in my mind in bold characters now…

[andrewjswan]

Regarding the Weather Forecast Service how do get forecast in the template weather now? If the forecast attribute will not be present, then it turns out that I will not be able to use it in the template weather, and the service cannot be called there. Or will I have to make crutches, get forecast in other entities and then substitute it into the template?

Template Weather Provider

[arganto]

This

@smartin

either I need to pull the Data in an Automation/Script first, and work with the Respond-Variable. In this Case, i no longer can’t directly trigger “When Tomorrows Chance of Rain is higher than 70%”, but have to set up a Time Based Trigger (or whatever), and then check for the Respond-Variable.

in the release thread brought up again this use case in my mind as well.

How to trigger security things on e.g. upcoming temperature or wind speed things when they show up in the forecast from now without periodically call the call-service and react on the response?

Because with this periodically calls and/or one to many templates entities to replace the attribute from before with entities now, I have still the same (or more) data in the state machine as before and most probably more calls/update as delivered before from the weather integration, haven’t I?

Or is the background/guiding principle/assumption, that only few users are doing it and then having the same state machine situation as before and the majority of the users are not using such scenarios and not needing them and so most of them have benefits from the change?

[petro]

That would just be a binary_sensor using a trigger, action, and your calc.

Marius above has a good example doing just that. You can add as many binary sensors as you want.

@arganto

Because with this periodically calls and/or one to many templates entities to replace the attribute from before with entities now, I have still the same (or more) data in the state machine as before and most probably more calls/update as delivered before from the weather integration, haven’t I?

You’ll have less data in the state machine, unless you put all the forecast into an attribute on an entity. And at that point, it will be the same amount of data as before.

Depending on what weather integration you use, that will be less calls/updates. ATM none of the weather providers have the service perform a call, they cache the data and return the cashed data on service call. So, it’s the exact same number of calls as before to the endpoint.

[petro]

You’d have to make a template sensor with a full forecast attribute, then use that in the template weather.

[andrewjswan]

Great solution…
How will forecast work in the Template weather if there is no forecast attribute? And why produce extra entities?
If we remove the attribute, then in the Template weather sensor for forecast we need to specify the entity_id of the parent weather, and then the template sensor, when accessing it for forecast via service call, will receive it from the specified weather, and then all new cards will work correctly.

[luka6000]

I (we?) want forecast as state. Why? To react on state changes. So sure forecast can be deleted from attributes but then we need a separate state entity for forecast. Why? Because state machine has a lot of side capabilities apart from keeping state like polling data and notifying of state changes. Why push it to template automations if this is already happening as part of the state machine? I don’t care about forecast and don’t want to call services to get forecast data as long as sun is shining. But as soon as forecast changes from “sunny” to “shitstorm”, I want my home to act accordingly. How to achieve that without forecast as state? Time triggers? But state machine is already doing just that by polling data through integrations regularly, right?

[KTibow (Kendell R)]

@luka6000

But as soon as forecast changes from “sunny” to “shitstorm”,

FYI this change doesn’t affect the state part of weather, the part that tells you what’s happening now. It only affects the forecast.

[petro]

@andrewjswan

How will forecast work in the Template weather if there is no forecast attribute?

It would work the same way as all other weather integrations. You’d use the service to get the data, but using the entity in the weather card would display the forecast like it does for any other weather entity.

@andrewjswan

And why produce extra entities?

Because you want to use it in that template, which does not have the ability to call a service. That integration probably needs an update to support the action or it needs to be rethought for extracting forecast data. I.e. a new field that doesn’t use a template, instead gets the forecast from a different weather entity.

[luka6000]

yes, I understand that. But think of a forecast as state. Simple as that. Would you still need forecast as weather attribute if you could just have weather.forecast? Or actually better yet forecast.weather (forecast.solar_wind, forecast.sun etc)?

[paddy0174 (Patrick)]

So here we are, a topic with more than 170 posts, a lot of nerves lost and some insults later…

And for what? That we still come back to the one thing that would have solved all of this upfront: communication! As easy as it could be to just say a few explaining words, no that needs a topic for the eternity…

@luka6000
You need to see the broader picture. There is no need for data to be in the state machine, when it is only updated every few hours. That is a problem, and if explained seems more than reasonable. So to get unnecessary data out of the state machine, there is now a specific possibility to configure your forecast as you want it. Daily, hourly, weekly. And it is only polled, if necessary.

So for what you want it to work, you need to do nothing. The update weather cards will take care for you of that.

You’d only need to change something, if you are already working with some template thingies for the forecast. If not, this change doesn’t concern you.

As I said above, it is all about the communication, and explaining things for users. I’m still convinced, that this discussion would have been avoidable, if only someone from the team would have asked the dev team for a few explaining words…