I’m building a component (in node via REST, but when I learn enough Python I’ll likely port it) that will be able to automate your lights based on time of day, various sensors, other lights, etc. It will also learn each day as you do what you do.
The barrier to that learning, however, is that right now, HA doesn’t note where the change originated - from the device, from the HA front-end, from REST, etc. In order to make the learning not learn from its own actions (which would only solidify whatever it’s doing, which might not be what you want, especially early on) I’d need to be able to know where the changes originated so I could withhold changes that were sent from the AI itself.
Any chance that’s something that HA tracks, but doesn’t expose in the History REST API JSON? Or, if not, any chance it’d be easy to add? I’m extraordinarily rusty with Python (only used it for a semester in college, 15 years ago) and while I can do simple things currently, digging through the entire HA project is going to take some time.
try setting up input booleans for each source of the change for each light. Have automations on those input booleans that toggle the actual light. Then base your learning on what happens to the input booleans. The only one you have to worry about is the physical switch. It will toggle the actual light. So for say a den light.
input boolean den_light_ha
action trigger to turn on/off switch.den_light and record event as long as the light isn’t already in that state
input boolean den_light_rest
action trigger to turn on/off switch.den_light and record event as long as the light isn’t already in that state
switch.den_light
trigger to trigger light and adjust states of the input booleans to match
The problem I’ve had thinking through the same thing here is that there are so many combinations of events that drive a change. For example, who is home out of the family, what room are they in, what time of day/night, is it raining, are there other lights on in the room, are there guests in the house that don’t show up on the presence detection? It requires a bunch of different sensors working together of course, and a quick way to compare those sensors and determine which ones are relevant and which one’s aren’t. For example, my son’s girlfriend was over the other night. She was in what we typically call his restroom because he’s the only one that uses it. He left to go pickup some pizzas for all of us. Since he left, the automation software saw that he was no longer at home and turned out all of his lights. She wasn’t very happy when the lights in the restroom turned off on her.
I was thinking along similar lines. A training mode might be requested which could last days or even weeks. Training mode would observe conditions, when lights were used in which rooms, how often, how long, which devices were present. Maybe the first part of that would be going through a list of found devices and being able to include or exclude them from the training (this phone, that room, this sensor but ignore that sensor, etc). At the end, some kind of confirmation/rejection of whatever patterns were discovered.
In my case, there are often people still in the house, but when my kids leave they have a bad habit of leaving their lights on, I already have the bathroom exhaust fans setup to turn off 5 minutes after the bathroom light is turned off, but the kids leave the bathroom light on and the fans just keep running. Yes I could go upstairs and turn it off, or I could go into the HA screen and turn it off. But I wanted the house to take care of itself. So I have certain automations setup to deal with when the kids leave. Unfortunately, that sometimes catches the girlfriends by mistake. I can tweak the automations I have some to account for some of that, but the point in talking about it was to point out some of the difficult situations we will face trying to build intelligence into the systems. But really intelligence is easy, it’s just a matter of having the data and an efficient storage and retrieval mechanism. What we need to figure out is how to make a wise house. By that I mean one that knows when and when not to implement it’s intelligence.
Loving this idea!
What would happen if you tie all sensors into a machine learning “AI” (Lucida.ai running on docker for instance) and have it output to all the controllable stuff (climate, lights, etc). And then just train it by letting it guess things to do that you can accept or reject (eventually the idea is that it would learn, thus suggesting things that you want all the time, at that point the confirmation step could be omitted).
it would be a LONG time before you could omit the confirmation step. Even through we are creatures of habit, we do stray from time to time. There are basic things it could learn, but lets say my normal bedtime is 10:00pm but a football game runs over so my favorite shows that end at 10:00pm are running about 15 minutes late. I would hate for the house to learn that I turn the tv off by 10:05 every night and turn it off on me right as we get to the climax of the show. My point is, unless you have IBM Watson running it, the level of knowledge we want is probably still out of reach from a computing perspective. That is not to say we can’t have some level of intelligence. Right now though, I think we are at the point of letting other systems point out opportunities for us to automate, I’m not quite ready to turn over the keys to the house to them yet.
Exactly. Even now - with all the rules set up and running fine in HA - they can all go down the drain with just a simple deviation from standard routines. And let’s not call some simple machine learning an AI. IT’s just an automatic templating system analyzing the data. There is no intelligence in there.
But as a system which will suggest new rules to be set up or correcting the one in place - it might be brilliant.
All great thoughts, everyone. And I agree that it’s not likely to be even close to perfect solution - especially in the beginning of its development. However, you have to start somewhere I don’t want HAL controlling my house anyhow
To describe what I’m looking to do in more detail:
I want to create a brain for HA. Something simple to install and configure, but powerful, given enough data. Ideally, it would use as much of the current HA functionality as possible, but the ability to differentiate between the source of a change would definitely be helpful in understanding what to use in understanding habits and what not to use.
I’m going to build it so you can teach it how to automate just a single light switch and time of day, or with a completely automated house and a full array of sensors. Ideally, both would be as easy as the other to set up (though, the second should be much more accurate)
It won’t be full on Watson intelligence, obviously, but I don’t know that it needs to be. I think with some solid thresholding, we can mitigate much of the difference in intelligence between something like brain.js and Watson. This won’t end up creating being like SARAH from Eureka, but it’ll be better than not having something.
What if the web sockets call_service call returned a JSON encoded state object after the service fires? My code could log that and use it to exclude that change from future learning. Not the best solution, but probably the easiest solution that would be reliable.
I can dig in and try to make this happen, I just want to make sure its the right answer first.
I know you want to look at python in the next iteration, but look at appdaemon. It gives you a python interface to do much of the work you are talking about as far as controlling devices and getting information about them.
I wonder if it the developers would add a field to the entity dictionary that tells us whether the new state came from the UI, physical switch, or automation path? If I remember the brief reading I did about HA’s guts, I think those are all coming from separate modules so they may be able to capture it.
I’m extraordinarily rusty with Python (only used it for a semester in college, 15 years ago) and while I can do simple things currently, digging through the entire HA project is going to take some time.
