Anyone using TensorFlow Lite with ESPHome?

Hi all. I have an ESP32 project where I am taking a couple of sensor inputs and would like to predict some outputs using a simple machine learning model. It looks like I should be able to do this with TensorFlow Lite so I’m wondering if anyone has been able integrate TensorFlow Lite with ESPHome? (I realise I could run TensorFlow on a server somewhere, but I’d much rather run it on the ESPHome node if possible).

I have found this GitHub repo which is described as “Esphome wrapper for TensorFlow Micro library” however there is no documentation so I’m not sure where to start. Any pointers are much appreciated.

Well, the weekend is almost over, but I’m pleased to say I’ve done it! I’ve managed to wrap the EloquentTinyML library into a custom component and it works a treat. It’s my first attempt at using custom components too. I’ll share the details once I’ve cleaned it all up.

5 Likes

Nice one!

You should share this with the ESPHOME Discord server where a lot of the devs hang out if you haven’t already…

Any details on this plz?

Hi Saqi,

I’m actively experimenting with different machine learning frameworks at the moment. Once I’ve decided on my preferred TinyML/EdgeML framework I will tidy up and share some code.

FYI, since my original post, I’ve discovered Edge Impulse, which I find much easier to use than building native TFLite models. I’ve also looked at Neuton.AI but I prefer Edge Impulse at the moment. Both are capable of outputting a C++ library which can be imported into ESPHome.

very kool thanks…
My thought is … based on historical data for a given light switch, HA (with the help of ML) be able to predict the state of a device at any given point in time… e.g. a light switch to be on/off Monday Morning at 9:00 am. There should also be provisions for influencing the confidence via entties e.g. motion sensor in the vicinity or sun - state / weather - cloud cover / calendar etc…