In the Netherlands recently a company involved in energy disaggregation (Ipsem Energy) went bankrupt. The company was specialized in disaggregation of energy data from a smart meter so that individual appliances became visible. This approach is starting to become available in more energy consumption meters. As the process of energy disaggregation relies on some heavy calculations (and some machine learning?) I was wondering why this isn’t available in Homeassistant? Most of us have got 1 (or multiple) consumption sensors in our setup. It would be great if someone could create a component that gives us the ability to see power consumption from multiple devices while only relying on 1 sensor.
Are there any new projects with Home Assistant?
I am currently measuring most of my devices with smart plugs, but I’m extremely interested in the approach of disaggregating data from the smart meter/meter measuring the whole house.
Seeing this many years later makes me both happy and sad. Happy since now I got a term for what I have been looking for – energy disaggregation – and a starting link with references to academic papers. But sad since there has obviously been no progress in this area for Home Assistant.
I have recently got a “smart” electricity meter for my house – as has all households in Sweden, due to a new law – that can provide energy consumption level every 10th second. This would form the perfect basis for analysing the consumption and breaking it down into appliances. It’s both annoying and frustrating to see that there seems not even be a project ongoing to try to bring this kind of functionality to Home Assistant. It would take a team of dedicated developers to get this going. I wonder if there is some way to bring attention to this need?
I have spent the last 5 hours trying to research the right terminology for this and here I am, finally seeing that someone has already requested this feature!
I vote a big +1, although a niche this feature could be a killer app, especially if HA or Nobu is willing to collect anonymised fingerprints of various appliances.
All, I have looked for this feature as well recently. I believe the academic term used for this is NILM ( Non-Intrusive-Load-Monitoring), also known as “fingerprinting”. Sense.com, a smart meter company does this, but there seems to be no integration with home assistent. Net2Grid in Netherlands provides b2b services offering the use of NILM to business users that can integrate it in their propositions. Perhaps good to contact these companies to see whether they want to make code available. Has anyone searched github for options?
I want to work on this when I have time, but that’s the problem; i don’t.
It’s really a 2 step problem: first you need to do appliance detection (figure what appliances are present on the network being metered, then do energy disagg for those detected appliances. In energy disagg, you have one model per type of appliance.
Energy disagg tends to be good with big appliances, but will be challenging for the small stuff.
To make your life easier you can name the appliances yourself, but it’s hard to account for everything.
As far as datasets, the Pecan Street dataset is the most complete and most commonly used for this type of model training:
You can build a multi-class sequence-to-vector neural net to classify appliances from summed traces. Sum up a random number of appliances from the dataset and create a sequence, then label with a vector that has 1 for the appliance present and 0 for those not present. Building sequences of a 24h period is probably a good start. sliding windows across the 24h of the day. That can produce a very large amount of data. Another approach is to use a classifier per type of appliance, with a sequence to point model, and detect presence or not. With this approach, presence detection requires many inferences, but may be more accurate.
For the dissag task, it’s similar: for each appliance type, create sequences of fixed length period for a mix of appliances containing the appliance of interest, then label with the next hour actual consumption for the given appliance the model is for. Train a sequence to point model to predict the value of consumption of the appliance from the aggregated sequence.
Note: accuracy for each model might vary greatly.
In the end, detect appliances in the network, predict consumption for each detected appliance, and then do some adjustment to correct for inaccuracies (su of predictions should equal total consumption)
Of course, this is the theory. I haven’t done this myself. Integration this with HomeAssistant is another bunch of work too
Interesting find, Jerome. Here is the link to the researchers GitLab repo with the HA integration code: Energy Disaggregation Tool · GitLab (It took a bit of digging to extract from the paper.) I have not looked any more closely at it. My quick glance at the paper indicated that the quality of the predictions were a bit so-so, and it was hard to update. So I guess this technology need some more maturing to be useful…
I could take a shot at implementing something like this, but the challenge will be to make it both lightweight and useful. Recognizing individual devices using machine learning will be very difficult, but grouping devices based on their load pattern is more feasible.
but grouping devices based on their load pattern is more feasible.
How do you mean? I have been playing with the idea of making HA rules like “if phase 2 increases 1 kW, then it is the boiler that is activated”. Is it something like that you mean?
I have been studying my consumption patterns for a number of days, and I’ve given up before I even started. There is so much noise in the data that a simplistic approach like that is likely to be of little use. On the other hand, the patterns are quite clear to a human observer. So I was hoping someone would have approached this with AI, as pattern matching is usually something that you can expect good results of.
There are signal processing methods to decompose a signal into its constituent components. But whatever method you come up with it will have to work for everyone, everywhere in the world. The consumption pattern of a boiler may differ between brands, but also between different countries. There are many reasons why this is a difficult problem. Maybe you can come up with ‘characteristic patterns’ for specific devices, and then use matched filtering / cross correlation to figure out whether the signal is present. I think machine learning on its own will fail to generalize at this task
There is so much noise in the data that a simplistic approach like that is likely to be of little use. On the other hand, the patterns are quite clear to a human observer. So I was hoping someone would have approached this with AI, as pattern matching is usually something that you can expect good results of.