Anyone using TensorFlow Lite with ESPHome?

Hi all. I have an ESP32 project where I am taking a couple of sensor inputs and would like to predict some outputs using a simple machine learning model. It looks like I should be able to do this with TensorFlow Lite so I’m wondering if anyone has been able integrate TensorFlow Lite with ESPHome? (I realise I could run TensorFlow on a server somewhere, but I’d much rather run it on the ESPHome node if possible).

I have found this GitHub repo which is described as “Esphome wrapper for TensorFlow Micro library” however there is no documentation so I’m not sure where to start. Any pointers are much appreciated.

1 Like

Well, the weekend is almost over, but I’m pleased to say I’ve done it! I’ve managed to wrap the EloquentTinyML library into a custom component and it works a treat. It’s my first attempt at using custom components too. I’ll share the details once I’ve cleaned it all up.

6 Likes

Nice one!

You should share this with the ESPHOME Discord server where a lot of the devs hang out if you haven’t already…

Any details on this plz?

Hi Saqi,

I’m actively experimenting with different machine learning frameworks at the moment. Once I’ve decided on my preferred TinyML/EdgeML framework I will tidy up and share some code.

FYI, since my original post, I’ve discovered Edge Impulse, which I find much easier to use than building native TFLite models. I’ve also looked at Neuton.AI but I prefer Edge Impulse at the moment. Both are capable of outputting a C++ library which can be imported into ESPHome.

1 Like

very kool thanks…
My thought is … based on historical data for a given light switch, HA (with the help of ML) be able to predict the state of a device at any given point in time… e.g. a light switch to be on/off Monday Morning at 9:00 am. There should also be provisions for influencing the confidence via entties e.g. motion sensor in the vicinity or sun - state / weather - cloud cover / calendar etc…

really cool! Please keep us posted. If we could empower esphome devices to be able to process data at the edge, that would be huge in my opinion!

I’m curious about ML in IoT, but I’m not sure where to start.
I have around 40 devices in HA, some of them are ESPHome, some cameras, around 10 sensors are Xiaomi motion detectors…

I don’t use a raspberry, I prefer an i5 processor laptop with HA supervised, so I think I could process ML process a bit faster, I hope… Is there a tutorial of what did you do and what can we get with ML? Face recognition? Or maybe things like to 0redict when a lightbulb should be off or when the alarm should be active.

Can you tell us a little bit about the devices and what predictions can they make now?

Hi any news of your custom components ? I will be very curious how you built everything

I am using TF-Lite with ESPHome for my Hot Tub Sensor project. Here is my device’s yaml file and TF-Lite Micro Runtime.