I’m currently playing with using the state changes from the History REST API into synaptic to be able to produce suggestions on what additional automations you may want to consider. Unfortunately I can’t help with any Python questions, but figured I’d give an update as to where I’m going with this in case someone else wants to dive in as well
I have it giving training data, and using various methods I can get down to about 5% error rate and a fairly accurate result in terms of when things should be on or off (I’m limiting it to switch state based on time only for now)
In an effort to stop my head from exploding while learning about all the different types of neural networks (let alone the detail that goes into them…), I ventured into custom panels, as well, since Synaptic can be used in a browser.
I’ve kind of taken the other approach and looking at time is about to make my head explode. I look at all of my devices and realized I am only interested in some of them. For example, I’m not interested in learning about what to do when the battery in my door sensor gets low, or what to do based on what input source my TV is currently on. It’s enough to know that the TV is on. That let me reduce 160 odd devices to 63 devices to track. I assigned each device a unique bit in a binary string. This lets me look at one number and know which devices are on and which ones are off. My thinking is that this greatly simplifies the if statements that will follow (not completely sure if it will or not yet). But that aside, I can easily compare states this way to see when a combination of lights, HVAC, switches, etc are on and make decisions based off of that. The problem I am having is how to integrate time. RIght now, The Den light turns on, and I know that the next thing that happened from that state is that they master bedroom light turned on. What I don’t know is whether that was an immediate response or if it happened three hours later. How much time passing represents a causal effect? And without having a bit reserved for each minute of the day, how do I track time related events and learn from them? My database is getting HUGE just thinking about it.
@Jason_DeVelvis, have you any interesting progress to report about.
I’m surprised that not more people are involved with this since this is what I see is the real potential.
From a personal perspective I’m so tired in taking up my phone or asking alexa for simple things, like lights. Automating is good, however there are so many possibilities so I think a learning system that takes away all the small things (not turning off the TV in the middle of a football game as mentioned previously in this thread) is the future for home automation and home assistant.
@Jason_DeVelvis, do you have a repository to share your work and for people as myself to contribute in any way I can (likely very limited)
The idea of having some type of AI or simmular that can show us “hay, you keep doing things like this often, why not automat it”…
Like, often 5-10 seconds after a motionsensor is triggered, the lights turn off, then this happens… and so on…
because even though we are creatures of habit, we are not always aware of our own habits…
but as to the point earlier about tv turning off in middle of show because the sports game before it ran long.
well thats just down to having enough datapoints… if you throw in a motionsensor in the hallway it would know that your still watching the tv because the sensor has not been triggered yet.
or add a sensor to fetch whats currently on tv, the AI would see that you always watch this show, and could then also see that the show is still running…
The main limitation to an AI is not having enough data
I like this idea. A simple, focused use case would be an occupancy simulator. Tell Home Assistant about all the lights in your house, and it will watch when you tend to turn them on/off and train the model. Then, when you leave to go on vacation, Home Assistant will use this model to simulate you being home. This could produce very sophisticated, realistic occupancy simulation without a ton of manual configuration.
I have a sunroom on the back of my house. The exterior door has an august lock on it, the interior door to the house has an open/close sensor on it. I have a motion/temperature/luminance/relative humnidity sensor, an IP camera, smart TV, 6 window sensors, a ceiling fan, fan light and recessed lights all controlled individually by smart switches. About the only things that aren’t automated are the blinds and we rarely move those. My point in saying all this is that I think I have that room fully automated and almost every aspect of occupancy and activity accounted for. I am going to start thinking about this room from an AI perspective.
I have the following data sources available:
Inside temperature
Outside temperature
Inside Humidity
Outside Humidity
Weather (rain, sun, cloudy from dark-sky)
Time of day
Day of Week
Holiday
Motion
Indoor Light level
Exterior Door Status
Interior Door Status
House Status (we put the house in nighttime mode when we go to sleep)
Occupancy of house as determined using owntracks (noone home, someone home, everyone home)
state of Fan
State of Fan Light
State of recessed Lights
State of windows (6 sensors not going to repeat them all here)
That is just one room. What other information do I have available that might make sense to track? That’s around 25 on/off states to track and I have no idea how to tie time into the "intelligence without having a separate test state for every second. And then we have to know whether the state changes are important and to learn from them or not. So how would we move forward with something like this…
i think you need to have a default pattern at first.
for example lights.
you set them on on a period that you think you need them and off when you dont need them.
then you collect for a period of time the data for those lights, when they are on besides the pattern and when not.
then after you have collected enough data you can create a pattern from the data and switch them on and off according to that pattern.
every night you recalculate that data base on the latest information.
with the data you define the new actions like
if that happens, and that is in that state then do that but only when not that.
but i think the most important from this kind of stuff is collecting data in a way you can easy access it.
I don’t think we should have specific event if statements or hard coded rules. The engine we build needs to be able to take rules and process them generically. The rules fire based on events being triggered.
Given that time is an event, how do we record that without recording the state of every item every second which would be huge. Is it enough to say that nothing happened so we don’t need to record that nothing happened?
How do we deal with reviewing the data so that we can take out extraneous outliers that are exceptions to the rules and should not be used in learning/creating/updating rules.
without a basic pattern it will be very hard to get a pattern at all.
you dont need every second.
if all states from the involved entities save state like:
timestamp;state
you can calculate every state for every second.
so you can then start a calculation at night that makes an overview from all states at a given time
based on your given pattern you can the create events for the coming day based on this settings.
you dont. it would be insane to start reviewing that data manual.
you create settings. and you calculate a chance that something should happen.
for example you calculate at night how often the lights are switched on between 18:00 and 18:15
if thats over 75% of the times you switch them on automaticly on 18:00
then you realise that in the weekend they are always on when its not needed, so you add weekend days to the calculation
so your you calculate the percentage of certainty that the lights are switched on on weekdays between 18:00 and 18:15 and if its higher then 80% you switch them on.
over time you have more base rules and more data and you can do more calculation to get better results.
in the first period you could use logging to see how accurate your calculations are.
you create the eventgenerator and it logs like:
log > timestamp;i predict that the lights will be switched on, in that room inside the next 15 mins.
listen_state(light) for 15 mins
if light go on
log > timestamp;i was correct the lights did go on
if time gone by
log > timestamp;i was wrong lights didnt go on when i predicted it.
if you include states on which the prediction was based and actual states (or better send it to seperate log) then you can use that log to evaluate. calculate how often the prediction was right at the end of the day. and move that to a sensor.
now you got a visual result on how accurate your eventgenerator is.
it might be usefull to split that up at first. see how accurate it is every hour.
create a graph for that and you know in which timeframe you need to be more specific with your rules.
Ok, so now that we have lets say 26 sensors for a single room, how do we capture that data. Because the states of all the sensors needs to be identifiable at any point in time historically. Personally I think it’s easiest to handle it as a string of bits with each bit representing the on/off state of each sensor.
thats enough to calculate every state at each time later on.
you dont want to start saving every second. that would give you enormous amount of unneeded data, and it would kill your system.
with these saved statechanges, you can calculate at a quiet time (at night) what the state of everything was on which time.
you can save individual states like that and keep going back and checking to see what was the last state the other device was in (time consuming) but less storage required, or you can capture the state of all devices at the time of the event you are interested in (storage intensive) but quicker data lookups later on.
you dont get it.
you dont want to collect data every second and act upon it. thats just normal automations.
you want to collect data over a period of time and predict what will happen in the future.
predicting the future is never about what is at this moment, but what was in the past.
predicting the future is about using learnings from the past to make decisions for the future based on data in the present.
You are right we don’t want to capture data every second. I’ve already said that. As for what to capture both ways probably work. All I am saying is that there is a tradeoff between speed and storage. I’m wondering if anyone else has any insights to it.
you make actions in the present.
you dont use data from the present, because then it wouldnt be prediction anymore.
to make it workable you schedule events based on the past, for the next 24h
to make the predictions better you create a few diferent scenarios that get activated based on what is happening.
and indeed i would also love ideas from others.
at this moment i have
collected data
settings
base events
so all i actually need to do is make the calculations that predict.
i already partially do that by changing the thermostats based on an average from the last 2 months of bedtime and wakuptime.
temp, switches, motion, lux, etc.
but also sensors that are second level like times times we go to bed, get up, when someone is in a certain room, etc.
some things already for over 2 years.
ok, lets pick a specific room and tell me what sensors, switches, etc you are recording for that room only. Lets stick with one specific room in your house. Describe it to me. Don’t worry about the automations right now, just tell me about the smart devices.
