Adding AI to HA

[edit] I meant to reply to turboc’s post a day ago, where he describes a light switching off at an ‘inopportune’ moment

Aren’t these exactly the type of situations (good) AI would solve though? I mean, AI would very quickly figure out, based on use, that the lights in the restroom should never be switched off if someone’s there for instance.

I would at least love to have some of these functions in home assistant, it could even be amazing when you start combining stuff like presence detection and heating. Never have a cold bathroom in the morning, but almost never have it warmed up for nothing, that kinda thing.

1 Like

A useful AI case would be to control a heater.
If you give a time and a temperature, the house should calculate when to turn on the heater so the house will have the given temperature at the given time.

Based on all the data in the database, and the current indoor and outdoor temperature it should be possible to estimate how long time it takes to heat up the house to a given temperature.

2 Likes

I agree, that entities knowing about the “executor” of their last change would be a valuable thing. That could even be useful in automations, somehow similar to trigger conditions. Could be hard though reliably identifiying this. For example, the HomeMatic switches work be sending a signal to the hub, which in turn responds back to HASS to confirm the state has changed. The same signal is being sent when you press the switch manually. So in this case it would alsways be the hub that executed the state change. And I can imagine there are some other devices that work in a similar fashion.

Anyhow, knowing there is such a problem, I still think it should be solved, given the value such information would provide.

Yes it would, you are right, but then you are back to putting a motion detector in the restroom. Kind of creepy. LOL

Agreed those types of systems are known as “expert systems” they use AI techniques aimed at a very specific situation. The challenge becomes when those expert systems have to begin talking with each other to more fully understand their environment.

yes when you are running things through a 3rd party hub like wink, Hue hub, smart things, etc, you would never be sure if the change was due to someone physically flipping a switch, or an automation managed by that other hub. However, I think there are three sources of changes to HA. One is the UI, the other is the automation engine, and the final is the interface with the outside world be it hubs or switches directly. If we could at least capture that information, we would know whether it’s an external or internal influence that caused the state to change.

To me the problem is how to setup the data structure to make an efficient storage and retrieval process. If you try to do it with bit operations (one bit per device) when you consider all the devices we have, you start running out of bits quick.

I only skimmed this thread, so if I repeat something, oops.

First off, I applaud the intention to learn from the HASS data. We tried to promote that with Jupyter notebooks but it did not get any traction.

So now about the data in Home Assistant.

First, there are 5 different types of devices and how they tell Home Assistant about their state: assume, cloud poll, cloud push, local poll, local push (read more about what they are here).

When we call a service, let’s say from the frontend, the code looks as follows (slightly simplified):

yield from device.turn_on()

if device.should_poll:
    yield from device.update_ha_state()

That means that if we are polling a device, and we refresh a device right after calling a method on it, we can assume that the state update was because of the method we called. However, it doesn’t have to be: if we update device at T, call a method at T+15 and update the state right away, any change happened in the last 15 seconds could be attributed to our service call.

It’s different for devices that push their state. In that case we will not ask for a state update after calling a method because the device will let us know when something has changed. In this case we could add some time based logic: if a new state gets pushed and we called a method within a second, attribute the state change to the method we called. However, now we are hard coding this and the time might be different per device or your algorithm sees it differently.

All of this gets even more complex when you start turning on groups or scenes, as it can impact a lot of different things.

So there is no consistent approach to attribute a state change to a device. It also would require a significant change to the core and every component.

An easier and cleaner solution would be to add a source attribute to service calls. That way, it’s up to the parser of the event stream to decide which state changes should be attributed to what. The nice thing here is also that the source parameter could be inherited. So when you call a service to activate a script and the script then turns on a light, we could still attribute that 2nd call to the original source.

For source names, we should use the entity_id of the source if possible and fall back to just the domain. Examples of sources:

  • rest (should be api because that’s how the component is called but that’s confusing…)
  • websocket
  • automation.all_gone (we can differentiate between automations using entity_id)
  • device_sun_light_trigger (this is an automation component without entities)
4 Likes

What if the web sockets call_service call returned a JSON encoded state object after the service fires?

The Rest API already does this. This is not implemented for the websocket connection because it assumes that if you are interested in these events you will subscribe to them.

It works by capturing all states that have been set while the service was running. So there is no guarantee that the changed states can be attributed to the service.

So it will be ready for 0.39.0. RIght? :slight_smile: :slight_smile: :slight_smile: Just kidding, thanks for the explanation. I knew it was more difficult than I was describing, I just wanted to get the thought process going.

1 Like

To make sure I’m understanding you - instead of simply sending a turn_off message with entity_id through web sockets, I’d also include source = “websocket.ehma” (where my project name is EHMA) in the JSON package, and that would then be delivered as part of the JSON sent when I query the history?

Well the source would be automatically set by the websocket component when calling the service.

How would that work with AppDaemon automations? Would they all show AppDaemon or would we be able to specify the module that sent the request?

I guess there could be an option for rest / websocket API to append the source. So rest.appdaemon or something.

could there be an additional descriptor to that so it would be rest.appdaemon - homeaway or rest.appdaemon - outdoorlights where homeaway and outdoorlights are specific python modules we have written to run under appdaemon?

I doubt that’s a good idea. In that case we might want to allow overwriting the source, make it appdaemon.homeaway

That’s fine. Just some way to identify which automation kicked it off.

That’s what I was thinking about in my post above, too - it would be nice to know exactly which were set by what, whenever possible. Adding an optional source to the REST & Websocket APIs could help with that quite a bit, and if it wasn’t supplied, the default could be rest.unknown or websockets.unknown or something like that to differentiate

I’ve been playing with this a little from another direction. In order to make quick retrieval of learned information, I was thinking about a bitmap of components. Whenever something is turned on or off the bit associated with that component is either turned on or off. That way we could have a simple number lookup to represent the state of the system and what to do when the system is in that state. It eliminates the
if it’s after sundown and everyone is home, and it’s a weeknight and tomorrow is not a holiday and we aren’t having a party right now then turn on the outdoor flood lights. And changes to if state=12847384 then turn on the outdoor flood lights.

my question is, how many bits do we have to play with in python as far as comparisons like that go. I just bit shifted 1 256 times, got a huge number i=256 - 115792089237316195423570985008687907853269984665640564039457584007913129639936 so what is the limit? Realize with what I’m thinking that’s 256 sensors/lights/switches/input booleans/input_state choices/etc so 256 isn’t as big as you might think.

1 Like

Lately, while deciding to give InfluxDB a try, I was thinking about something similar:

What about feeding home-assistant data from the past few months (InfluxDB) to an AI-Style algorithm that analyses those informations and gives feedback in form of propositions of new rules for my configuration-file?

In that case no need to thinker live with all the events… “Training” would be based on historical data. Could be a nice first step…

Could be. The problem for me is it gets into statistics and I hate statistics. We need to count the number of times a specific pattern of entities are in a particular state and when that value is over a certain percentage of the data points, it becomes a candidate for a rule. Something like that. That way the more the state combination occurs, the stronger the rule becomes. So for any rule combination we need to assign a weight. It’s just a LOT of data and grows exponentially every time we add a new entity whether it’s a sensor, input_boolean, or light.