HI all
the aim of this project is to take a novel approach to presence detection, by using statistical methods to make the classification home/away, rather than the traditional approach of having signals plus rigid rules. This project will combine machine learning via facial recognition with the bayesian sensor.
I have tried facial recognition and classification using the openCV component on a pi-3, but have found this to be rather slow. Therefore this project will investigate the use of this home_surveilance project, with outputs from the home_surveilance system as inputs for the bayesian sensor.
If anyone wants to get involved with this project or has suggestions for it, please post them here
Cheers
This sounds interesting… My general hope for machine learning as it relates to home assistant, is that we can leverage it so we require less complicated real-world sensors to capture the same types of events. The Bayesian Binary sensor implementation I made is a small step in that direction… This sensor could be another, but I think the real power will come when we can learn from data that resides in your instance of hass rather than pre-trained models or simple statistics based on human-generated priors.
As you have seen, machine learning (especially on image data) can be very expensive from a computational perspective, so implementations should be carefully planned. I’m curious to know how fast you would hope your ideal solution to run?
Interestingly, I’ve already implemented something very similar to the Bayesian Binary sensorfor facial recognition, which leverages AWS Lambda and AWS Rekognition. This could be an alternative if people would be interested in leveraging some AWS services. A solution like that would remove many speed concerns around facial recognition.
Not sure if any of this is helpful, but I’m interested in following along with this, and trying to help when I can. I’ve got quite a few distractions in my personal life right now, so most of my help may only be rough guidance rather than getting into some code with you.
Hi @jlmcgehee21 thanks for your post. Have you implemented your AWS approach in HA?
I will follow the model of home_surveilance with the heavy lifting processing performed on a local server (in my case a synology). A requirement of my project is that all of the data required to train the model should be capturable from HA, via cameras, sensors etc. My hope is that by combining data from multiple sources (cameras, sensors) that the image processing requirements can be reduced. For example, if there were many strong indicators that person x were home, then the rate of facial recognition could be significantly reduced or turned off entirely.
I’ve implemented a bayesian sensor but haven’t found the right combination of factors/weightings to get it to detect when I am in bed.
I hope that this project will be a demonstrator for how to use these ML/statistical approaches in practice.
@robmarkcole, I have not implemented the AWS approach for HA, but did it at my day job for a client. It was relatively simple to set up, so it should be straightforward to implement for HA.
Based on what you’re saying, it seems like it should be possible to use home_surveilance, as an input to the bayesian sensor if you can build a HA component or implement an http/rest sensor for home_surveilance.
In addition – I’ve been thinking about trying to build a utility to go along with the bayesian sensor that will spit out all permutations of your observations that would give you an “on” and all permutations that would give you an “off”. Would this be useful? Of course, the holy grail is to have the sensor learn the probabilities on its own, but that would require a frontend for users to label the event they wish to capture with the bayesian sensor.
I also thought about this but was thinking more in line of a ML cloud service that access the HA database to suggest automations. Ideally , the service will only need to know historical sensor states and the actions but need not know what each does. It can then hopefully look for statistical patterns in the sensors / actions and suggest automations. This is all way above my abilities.
Yes currently reviewing whether to use home_surveliance or create something precisely for this application.
Re Bayesian yes would be nice to have a tool/method to determine what are the optimal inputs. Using a Jupyter notebook you can explore the HA database, this could be ideal.
Will definitely keep following this, I have been trying to use machine learning to develop a model for the “time” it takes to transition between rooms by collecting motion sensor data from each room. The end goal is to do occupancy tracking per room regardless of the number of occupants and develop a automation workflow for controlling lights, etc. based on the number of occupants per room.
At the moment I am still trying to sanitize the motion sensor data so that I can develop a normal distribution of transition times.
The Bayesian sensor works very well for presence (you can check my weights here). But you are right, we absolutely need to add images to the mix. It is unfortunate that we have to rely on external home_surveillance project for image recognition. I hope that the image_processing platform in HA will get some love.
BTW, there was a bug with OpenCV and it was ignoring the classifier. It should be fixed in 0.55 and hopefully it will improve the accuracy.
