Reasoning: Artificial intelligence applied to Home Assistant?

FOREWORD

My name is Daniel, i’m from northern Italy and both by passion and work, I’m a programmer and personally I’ve always been fascinated by home automation and control of my small house.
After using HomeAssistant for many years, in my opinion the best ever, I realized that even though it was very interesting, it always lacked something to make the home truly intelligent (not only smart)
I made some experiments using AppDeamon and I shared in this forum my experiences but in all sincerity I was not satisfied and did not give me that feeling of autonomy if not always disguised behind some prediction.
With this post, perhaps a bit long but certainly interesting, I want to share with you what I learned during my readings and analysis, during my failed experiments and watching the experiments of others too.

As I am Italian, I apologize if I make any mistakes with the English writing.

If you find this interesting and would like to contribute, thank you, you are welcome!
It could make for an interesting discussion!

WHAT IS (REALLY) THE ARTIFICIAL INTELLIGENCE

No, it’s not a stupid question, in fact, it’s a very complicated question.
Many might answer « it’s that thing that does other things by itself » but it is not so :slight_smile:
For what I, personally, have been able to read and understand the concept is much more evolved and extended: currently there is no system with artificial intelligence.
Smart speakers, autonomous robots, smart systems and thousands of other fantastic technologies called smart are actually only programmed to do certain things in certain situations but, if those situations should change, they fall into the so-called fallback or do not know what to do.

In simple words: they don’t know why they perform certain actions, they perform them because they are programmed to do them under certain conditions, without asking themselves any questions or any consideration of the possible effects.

In fact a real intelligent system should be able not only to evaluate the current conditions but to reason about possible actions that could lead to a certain desired solution, according to the cause->effect logic, as well as to think if there are alternative solutions and eventually choose the best one according to the current situation maybe in constant change.
Probably only the human brain is capable of making such complex reasoning, without something/someone telling it what it should do in what situation to apply

OK, BUT THERE ARE SYSTEMS THAT CAN PREDICT ACTIONS

Exactly.
Currently all smart technologies are based on mathematical calculations and algorithms that involve initial training of input data or situations and how that information should be processed to perform an output action. Often this calculation is purely mathematical based on statistics

Example with HomeAssistant entities

Temperature … … …: [18°C] [19°C] [20°C] [21°C] [22°C] [20°C] [19°C] [18°C]
Heating … … … … . : [.yes] [.yes] [.no.] [.no.] [.no.] [.no.] [.yes] [.yes]
Windows open … … : [.no.] [.no.] [.no.] [.no.] [.yes] [.yes] [.no.] [.no.]

With these data and a statistical mathematical calculation or based on a Bayesian equation, it is simple to deduce that when the temperature is below 20° we must turn on the heating, when it is above 21° we must open the windows and so on, but we do not ask the question of why below 20° we turn on the heating or why above 22° we must open the windows; we are satisfied with a statistical prediction.

If the user would open the windows when it is 18°C outside because he wants to air the house, the system would fallback or ignore the situation, this is called exception.

BUT THERE ARE MACHINE LEARNING SYSTEMS!

Yes!
But what they do is guess a possible action output based on an initial data training input, providing a solution based on the distribution of the training data.
So they can identify an APPLE from a CUP (if we talk about visual ML for example) simply because the APPLE has a different shape and color from a CUP but if you try to make them recognize a BALL, it could be half APPLE and half CUP - you have to be the one to say that it is a BALL !

OK. SO WHAT DO YOU SUGGEST?

What I’m trying to think, to experiment and to share with us, is the fact to completely overturn the thought, moving away as much as possible to a system of prediction aiming more to a system of reasoning. My goal is no longer to turn on the heating when it is cold, but to make the system reason about why and, if the heating is controlled by the computer, about the possible consequences that such action would bring (example: it makes no sense to turn on the heating with the window open).

In my (crazy) mind, always thinking of HomeAssistant, there is an idea of initial observation of user actions and sensor values (training) to then relate the elements and understand the cause->effect logic so that it can be applied to achieve the desired situation.

To achieve this I need to create a data model of actions (user) and reactions (sensors) and relate these two models in order to make the system understand what happens in case certain situations occur and apply them in case it is necessary to arrive at certain conditions.

At the moment I have a vague idea of how to do it but I think the most important thing for now is to talk with you, evaluate if my thesis is correct or not and maybe arrive TOGETHER to a possible solution.
I want to reach this goal thanks to you..

Maybe we can do it together.
Thank you!

2 Likes

hello dadaloop82
Daniel
i really like your project , do you think to use xgboost decision tree algorithm based on this home-assistant_v2 file ?

Hallo @Cao_Hoa and thank you for taking time to my reasoning!

Yes, exactly, the algorithmic tree decision is the best choice and I was just doing some experiments in these days.

But I have to be very careful not to fall in the probability “hole”, it is not my intention to turn on a switch just because there are many chances that in that condition you can turn it on.
Believe me, it’s very easy to fall into it!

Surely, as already said, the algorithmic tree is the best choice but I would also like to introduce some kind of reasoning so that it knows the possible consequences of its actions and knows how to correct them with or without the help of the user.

I went to see your suggestion that is the xgboost library that is also available in python both in standalone format (https://xgboost.readthedocs.io) and included in the Scikit library (scikit-learn: machine learning in Python — scikit-learn 1.0.2 documentation) but I didn’t understand what you mean by “based on the home-assistant_v2 file”.

In any case, I think it’s easier to start with using “DecisionTreeClassifier” also provided by SciKit - I see it as more practical and easier to understand.
What do you think?

I’ll be updating my GitHub space soon, maybe if you know Python you can help me out too!

I am very happy to talk with you.
i also want to include machine learning in homeassistant
home-assistant_v2.db which contains all the data and time and etc… of the homeassistant
i am running this to see what automation is running
and I’m writing quite a few packages

Hi Daniel, hi everybody.

I’ve been wondering the same idea for quite a while, but apart from being from North Italy too, i’m not any closer to a solution than you are. (Haven’t started yet, in fact, and work+family management won’t get any better soon, so not sure about the time i can invest in helping you but pretty sure i won’t get anywhere alone)

Here’s my problem:
Got a VMC machine in a big house. What’s bugging me is the inefficiency of the system (3 speeds on central fans, no IOT, no air quality sampling, no single room control) and what’s worst outside air quality sucks big time and even at minimum speed it even drags too much humidity from the air.

My point would be to get rid of thoose coaxial noisy fans, using single fans for room intakes based on air quality and, above all, limit the exchange with outside based on delta temperature, delta humidity, and pollution level.

Can I achive that just programming conditions. Obviously yes. Will it be any fun ? Well no, and the problem seeme to be perfecly matching a ML+AI application. Am I right ?

Cheers

1 Like

Hi Carlo!
Well, it’s a pleasure to meet you, maybe if you want we can have a chat in private (in Italian of course!).

I read your message, even if I don’t know what a VMC machine is but I can imagine.
I also understood your situation which is really very interesting and very practical.

Let’s say that, properly configured, even the simple HASS can give you satisfaction in this sense, of course you have to take into account many factors and conditions.
It could certainly be an excellent application for the dream I have in mind, that based on its own experience and reasoning would choose the best action to perform depending on the situation.

I recently published the description of my dream on my Github, I still have to finish writing it but what I will do soon.

If you want you can follow my project directly there.
I’m waiting for your private contact, if you like!

Ciao da Bolzano :slight_smile:

hi dadaloop82 i saw you uploaded to github
and I have git on your instructions
I do not receive the file Open this configuration file and modify it according to the instructions in the file

[appDeamonFolder]/apps/MyHomeSmart-HASS-AppDeamon/apps.yaml

Hello! I have not uploaded the project yet, I will do it shortly :grin: I have to start it, for now I have only written the documentation :stuck_out_tongue_closed_eyes:

thanks dadaloop82
Daniel a lot . I’ll be waiting and I’m glad to see your project go so fast

Good morning everyone,

in these days I invested a lot of time to write my whole idea on my gitHub repository, I hope I did something nice.

I also rethought my idea and redid part of the operation diagram, it’s not complete yet, but half of the work is done.

I’ve also started the programming part, I’ve only laid the groundwork for now, so there’s nothing usable yet.

If you are interested in the project I ask you to read my idea on my repository and possibly help me or support me in any way.

Thank you!

Good morning

I just updated to your new version and got this error
please help me

Hi Cao_Hoa!

I ask you to use the github repository next time for any communication about the project. Here we try to keep only the information at a general level, eventually I will write the achievements.

Anyway the project is still in development, currently I have to look for the way to identify the cause->effect condition but I think I’m close to it.

It is currently NOT working, but I’m glad you are keeping up with the developments.

I will update the project readme for any news!
Thanks!

hello gameloop 82 I’m glad to see you reply and update the project I will always follow and accompany your project I wish you a good day

Hi @dadaloop82 ,

I am actually working on a very similar idea but instead to create this as an add-on for easier use. Would be interesting to perhaps sync on this and see how we can help each other!

Regards,
Chris

Dearest Chris!

I’m really happy that you too have tried your hand at this arduous undertaking !
Is it possible to view your project in a Github ?

I started recently and for now I’m still at data collection and data normalization, that is what comes from the sensors.
The last thing I have done is the optimization of saving numerical data, which are grouped in a smart way by distinguishing them by importance and frequency.

You can see my work on my github (see signature)

I’m on my own and getting by in the (little) spare time and I would be honored to have your help, suggestion or share experiences with you.

I’ll be waiting for you, to continue our discussion, directly on the project github!
See you soon!
Daniel

I’d like to sync too :slight_smile: I am data scientist with a background in computer vision. For my next project I need to get into time-series ML. Will be happy to help

Are you aware of Motion AI - Detection and classification of entities ? I read about it and it sounds interesting, definitely contains AI concepts, may be relevant.

@dadaloop82 here’s my code GitHub - lcmchris/thesillyhome-addon-repo: add on repo for thesillyhome
Spent some time cleaning it up, feel free to try the add-on and give me comments + suggestions!

Hi Nova!

I have read and carefully examined your repository and it was really very interesting, I can see that you have a great experience in ML !

Since we have the same goal I would like to talk to you privately to see if we can collaborate together and if I can use some of your code, even if it is already explicitly stated.
Or join forces, ideas and projects.

You can contact me on Discord dadaloop82#3319 or wherever you prefer.
Hope to hear from you soon
Daniel

Have added you on discord!
Would be awesome to collaborate in some way on this. Having someone to validate potential features or new models would make this much easier :slight_smile:
Another aspect is potentially we could start sharing data → or one better setup a centralised db on cloud where all HA users to dump their events on easily. I have a theory that a centralised model on human behaviour in general could help better predict individual behaviour.