Welcome Folks to The Silly Home’s first weekly release - Model Mania
The Silly Home tries to bring true automation to your smart homes via ML. This means no more rules!
Through these regular releases, we hope to make The Silly Home a better application and allows us to broadcast insider knowledge on the developments!
Don’t feel shy to post any comments, criticism and ideas below. I want the feedback and ideas to make this amazing. We also have a discord chat if anyone needs help setting up or just wants a chat.
A huge pain-point of The Silly Home (at least for me) is that the model can trigger incorrect actions that I cannot rectify. My inability to ‘fix it’ is because any new state will re-trigger the models will generally predict the same incorrect action, which is very annoying.
Our solution is to have a small database (sqlite3) that stores all sensor states when an action that reverses a recent (within 5 secs) event occurs. These rules are then checked against before executing any actions.
- There is thought to feed this information back into the training set.
- This implementation falters if the AI predicted actions also are within 5 secs. A more comprehensive implementation is to differentiate which actuators events are created by The Silly Home vs Manual. It does not seem easy to do this within AppDaemon.
Right now, only DecisionTreeClassifier is used to train our models. Not much thought other than the fact that its easy to explain. To improve accuracy and more importantly precision, this feature checks the 4 models and returns the best one based on a metric skewed towards precision.
- Decision Tree
- Logistic Regression
- Random Forest
This is a cheap way to gain performance.
- All hyper params are defaulted. There needs to be some intuition/testing on the possible Hyperparams for each model.
- Some Data scientists may comment that this approach will lead to overfitting. I agree and its something to investigate but at the moment, having it overfitted with high precision and low recall will help cover the top 70% cases for automations rules.
The date features was previously not accounted for in the model execution step. This has now been added.
Our next focus will be to implement a Shared database for all our states data. I know many Data scientist (including myself) would love to have a larger dataset to see if there techniques we can apply to generalize the model (i.e. people switch off all the lights when no one is in the house). This global base model can then work in ensemble with the personal data model.
This will be an opt-out feature and the data will be stored on AWS (the exact service is up in the air as I’m speaking with AWS on Monday to check the best approach to do this).