What happens when I open my main dashboard with a forecast card on it?
The data for this card has to be sent anyway does it not?
How would it update otherwise?
3 Likes
Could it be that the syncing of the state machineâs state is blocking (during the loading of the app, as mentioned, meaning nothing will happen until thatâs done), whereas this would be non-blocking (i.e. load async because it will be a separate service call after the fact)?
Obviously Iâm taking an educated guess, so keen to hear what the core devs say.
1 Like
This makes sense, on the other hand, my definition of a HomeAutomation system is that atleast the essential components of such system should somehow be easy accessible, stored etc. , but following this Community iâve notice that many people âoverloadâ their HomeAutomation system with integrations , that actually (in my opinion) belongs to an i.e âMedia.system/archiving system etc. etcâ
As mentioned earlier( atleast the way i use/see the weather-forecast integration) , itâs the basic info,imited in range/details, (Not as much as i , my self, can retrieve with a manual API call to i.e met.no,) which i need for i.e automations/template to control my Home , i.e Heating/Cooling system, Power.consumptions/savings, irrigation and other essential integration/functions in a Home environment.
A HomeAutomations system should prioritize these, such functions.
The limited data of the i.e 24hours/7days arrays(which is ârecycledâ) donât ever grow beyond whats visible in the âAttributesâ atleast shouldât , how ever im sure there are lotâs of use-cases with the ever growing added/supported integrations, and what some people find it cool/funny etc to Add to their homeautomations system ( these people need to understand the âCostâ of their use of and dimension of their behavior/wishes.
Sacrificing an highly essential Integration ( weather-integration ) with limited( and in most cases fixed data size ) Very useful information in i.e a Homeautomation system, for the sake of god knwos what people stuff into i.e HA, just because they can, integrations, data etc. which in my opinion doesnât belong to a HomeAtomations system.
I donât know how itâs âpenetrated/concludedâ that the weather-integration is âa burdenâ, for the state-machine( I sincerely doubt) , but that the new proposed/decided re-build of the integration, will ofcause leave âspace/resourcesâ for other non.essential" integrations etc.
Key word is prioritization, where each one have to make own decisions, and knowing the consequences, SURE mobile connection/use cases, is somehow very common, but hardly the ground/foundation a HomeAutomation core should be build upon , ⌠if some one have problems with wifi interferrence, slow phone, chocked browser etc, it has absolutely nothing to do with the HomeAutomation system, AND/OR the very litlle data from i.e stored weather-forecast data.
People have to know this, and prioritize their system cording to their needs.IF they want their whole âlibâ of movies/music or tons of âstate-detailsâ or tons of colored-dimible-lights etc. etc. Everything comes with a Costs
It seems to me that the focus, in my opinion, leaning towards a âPlay Goundâ NOT an Prioritized HomeAutomation system.
Sacrificing a âsimpleâ function/feature ( highly usable in controlling essential âStatesâ in a Home ) , for the sake of i.e Mobile browser use-cases, or other non-essential use-cases/integrations in a HomeAutomation system, shold not be the way forward for HA, in my opinion.
HA will never ever be able to fulfill a Total, all-round use.case, network-communication, hardware, common interface knowledge limitations etc etc. will always.be a factor, so âsomeâ people might considered the need for i.e an âAdditionalâ HA installation, if some have issues which HA ( or in particular, their network environment, mobile, browser) canât âcopeâ with
Already when it was announced that âAttributesâ would be removed ( unconditionally ), some 3rd part-integrations-devs , started to âmoveâ Attributes into individual sensors/entities , so iâve been wondering, how many additional entities do we have already now, do to this announced decision.
States are the essential condition in a HomeAutomation system ( even it is Forecast-data )
If i should choose between flood/Lightning-announcements VS upcoming-song-in-a-playlist ( i know, Silly example ) However, my purpose using HA is for controlling my Home
bdraco , im sure you have made an overview of the Core-Components in HA and their impact on the DB, but obviously i canât know which is HAâs prioritizes
So thaât why i constantly gets ânotificationsâ that ( Ha is âupdating sensorsâ ) ? , even thou i donât have the HA-App opened ( seems more like to me this is a âbackground functionâ regardless if i havenât open the APP for hours), ⌠if all this boils down to a âMobileâ issue, then it is what it is , NOT an HomeAutomation issues caused by âWeather-Forecast-Attributesâ
PS: Essential to mention is maybe that i only use my phone on âLocal Connectionâ, and very seldom few time a day , either just to verify certain âstatesâ i donât have a utopia-illusion, that i would ever be able to load ALL âStatesâ from ALL integrations in an instance if im abroad, or just âawayâ from home.
But i would feel kind or more âsafeâ if an weather-forecast-integration fetch the data from the API, and could keep function until all the data it just received from the API, is âstaledâ ( cording to the âTimeStampâ in each forecast-entriy , atleast it would buy me/the system some time if problems occur retrieving new data from the API ( even thou im aware that forecasts-change by the hour/day, so it might not be the âlatestâ prognoses, BUT the system will keep running, for 23-hours ahead, and 6-10 days, what ever is retrieved. NOT stop function because it âmissedâ a few API calls
1 Like
This really has nothing to do with the topic at hand
Well, Iâm going to slowly start pecking away at this total rewrite if anyone wants to help.
1 Like
I know, It just have to do, how i consider a weather-forecast-integration should function/have priority in a HomeAutomation System
( fetch the data from remote-API, store it in DB/File, Keep function until all forecast-entries is due , just in case ⌠) itâs better it keeps working, than stops working bc it couldnât retrieve new/updated info from the remote-API , i would then get a âwarningâ (canât retrieve new data) and have time to investigate, and take precautions, ⌠itâs not like weather-forecasts is so damn unreliable that the forecast (prognoses) changes rapidly and extreme, they tend to be quite âstableâ in near future.
Why would you need all this information in the state machine?
What is your use case for this forecast data?
Its going to load the forecast data async and (re-)render the card when its available
Eventually it does but not until its needed to render the specific card
If it was already in the state machine it would be loaded on the initial connection resync. Which is effectively blocking because almost everything needs the data to render
5 Likes
The answer is a bit mirky but since many cards/panels needs data from state machine (or the registries, or config, or some other place that could have changed since the connection was lost) its effectively âblockingâ (even if it technically isnât for some places) until the state is restored from the backend after reconnect.
There are ways to speed up getting back in sync with the backend, but they require the backend to keep the state of the state machine at a specific time and be able to replay only the changes between when the connection cut off and when it was restored. That might be faster, but itâs a significant workload (memory and cleanup of old data), adds considerable complexity, and has bug risks for the backend, so itâs probably a non-starter without increasing hardware requirements.
3 Likes
Am I missing something or is the weather forecast the most tiny and compressible part of the frontend?
Also crazy idea but what if states could be streamed in on the initial load instead of blocking
( 1 Simplified use-case)
IF Tomorrow the outside temperature will Dropp to -15 degree @ 00.05am
( i.e from current +10 degree) and during this time, from âNOWâ obvoius the actually(current) temperture might/will differ from âPrognosesâ, this will also be ajusted in template( by local devices) !
And temperature in âheated-water-tanks is == xâ
( or , if inside temperature is == x)
⌠or both above
start system-heating the house @ 10pm today
( there is Always some delays in i.e warming up a house(a totally individual factor), also depending upon, whether itâs water-heated(whatâs the water storage temperature currecntly etc. etc.) or direct electricity heated, how many watts do you have available, to reaching/securing to decired temperature etc. with in the actual timespan ) Even here is ofcause current temperatures(local devices) involved in the equation.
If you only use Current-data ( Local-devices ) for your i.e heating-system, the âhouse-holdâ of your resources will be ineffective, you burn alot resources, maybe even at wrong time, because "suddently, out of nowhere, the i.e temperature dropps rapidly, below the amount you have available, or the capasity to somehow instantly get the temperature to desired level ) âŚ
-
Sunny/clouded
-
Lightning/Storm
-
Heavy precipitations
-
Temperatures
-
Sunny/Clouded is even useful if one have solar systems
-
Coming Price of electricity , kind of âforecastâ that eventually becomes history in the state-machine ( Removed from Attributes soon)
etc. etc.
ALL Very " Important " factors in a HomeAutomation system(Server) , and im not Talking about what and how fast my Mobile can retrieve the DATA, because THAT is absolutely a non-essential factor for how a HomeAutomation System(Local-Server) Should work
Im not going into a debate, of all the issues people have with their wifi, or external UI access to their HA, because thats absurd, considered that 99.9% of HA users ( including me ) hardly have any clue what âLEVELâ my Router currently handles the wifi traffic, WIFIâg-Level quite often ? (do to limitation in 1 or few devices on the network ? ) , AND remote access !!!, what over metered-mobile-connection ?, on a sunken free wifi-network in outer town, or at friend/airport wifi ? ⌠forget it, and often not mentioned in many use-cases here in the Forum somehow. AND hopefully nothing which have priority, or taken into consideration when buildning the Core
PS: I know your question was to finity 
but with the new action section in trigger based templates you can create a binary sensor for that temperature drop. You donât need to have all the data available in entity states or attributes.
Based on the same trigger, you can also create (and update) a sensor which gives the precipitation of the next x hours, or a binary sensor which is on of there is precipitation forecasted etc.
So therefor I wanted to know why there is this need to actually create template sensors for each hour of the forecast.
2 Likes
Yes i know, and wait with further âtest/templatesâ until final solution is in place, but as mentioned elsware, we are just replacing attribute with a bunch of sensors/entities , but, howvever if the sensors can just survive a reboot(with the individual states intact, and the template/automatioin working regardless of loss of internet-connection, and my system not hangs during startup, for same reason( no internet ) im content, my system will be able to handle lots more entities/sensors, so just have to âajustâ for the lack of Attributes, in the reference ⌠And hope the system comes up and running, spite the lack of, and unable to preform remote API calls ( i know a few integrations that does âhangâ the start-up 
Wait, i somehow missed this point, does that mean i can create a( or 3-5) binary sensors âon the flyâ ( ofcause i then need an automation that goes through the âlatestâ forecast(from the latest remote-API call), and âpick relevantâ trigger points, to create these sensors, an then update these sensors(during each new remote API call ), so when/if my system goes down(or i manually reboot), and eventually comes up again, with no internet( I DO expect these sensors have the same State, after a reboot) , so itâs still âonâ ? ( and ofcause updated if im lucky to/or when i get a remote-API cal through)
My god we will see lots of Topics in this Forum in the future, tweaking these automations/tempates/binaries-sensors, which aims to control the automations/templates for the actual Fucntion we want to address
This the first explanation that has enabled me to begin wrapping my head around the dynamics of underlying issue here. Thank you.
I found the developer blog entry for this change but I havenât found any directly related architecture discussions about it. The issue about attributes polluting the state machine does seem to pop up in various architecture discussions though.
Iâm wondering if a laymanâs terms blog post directed at the end users about the rationale for the reduction/elimination of entity attributes might be in order to keep the pitchforks at bay. Iâm sure the weather forecast issue wonât be the last time this topic is raised.
1 Like
The weather forecasts were one of the largest attributes
Consider the case that many users donât set up custom dashboards. Without the contents of the state machine fully loaded, its impossible to auto-construct the default dashboard from the state machine data that isnât loaded.
Even for the custom dashboard case, itâs likely more expensive to try to load part of the state machine at a time as it is a lot cheaper to ask for the whole thing than try to load it piecemeal as each card is rendered as that could be thousands of round-trip requests to the backend.
2 Likes
True, but hardly a âshow stopperâ or the one causing most usage, either when running , or loading ( loading forecast yes ) but the overall loading of a view/dashboard ?, im sure there must be many Integrations and card/view configurations who have a much bigger impact on the state-event engine, or ?
PS: I have no idea, canât âtrackâ much in my state-table, as i record very little data
All entity implementations are effectively competing for their share of limited resources. Some of the weather forecasts had the same space cost as hundreds of switch/sensor/etc entities.
2 Likes
I consider these the most important:
Current conditions
forecast[0]
Alerts (if available)
The others can wait. Still, even if you give preference to the above, you wind up burning up an API request.
Yes, i know, well you said above, obviously i have(had) no idea(thou always new it was the largest, among few Cameras is have, thou these have been dealt with in the Tapo-integration,a few more entities but still some attributes left there thou), and as mentioned my Homeassistant installation is not a âplaygroundâ 
So far all my automations/templates function, and i will notice/get noticed if i.e a switch.x is "not in the state i Assumed, same goes for lights.x ( i see the lights on), knowing when, how often, how long, i could judge by i.e power consumption etc ( so hardly relevant for me/my system currently
Tibber_Custom is a light-weight frontend (compromise), which i have modified to my view-design
And i have already forgotten whatâs included in default Recorder-settings.
EDIT: IF you take an advance Camera, and all itâs entities, you will get a total of More Datas/States/Attributes-States(than weather-forecasts, well maybe not if you call all available datas in i.e met.no API), if you then have i.e 6 Cameras in an integrations, well multiply , i canât envy the task you âstrugglingâ with 
I have a NWS forecast with the hourly forecast for 7 days. Itâs just 31.6kb raw and only 1-2kb with compression. Thatâs not going to slow down HA at all, is it?
(
For comparison: Loading all JS in the frontend, with compression, excluding custom cards, is 631kb (500x the size of the hourly forecast). However both of those will load pretty quickly as LAN speeds can get pretty fast - I wouldnât worry about 1kb when you have 1gb/s speeds.
I have my own gripes with the architecture of the frontend (way too many dependencies including gulp, uses a bunch of polymer stuff, uses the shadow dom, etc), but thatâs for another thread.
)