Data science with Home-assistant

Sounds interesting. Please keep us posted. Oh I get what you mean about re-configuring the configuration and linking it to the observations now - that makes sense:). I guess ideally the bayesian probabilites could be defined and dynamaically set via a input_number or similar (like how the target temp on a themostat can be adjusted)?

@sophof I am interested to see your streamlit ui as am working on similar, is it on github?

@Mahko_Mahko for classification of balloon in image why not use tensorflow/pytorch? Also if you want to dynamically update the bayesian sensor, it would be an idea to add services to the integration to do this.

Thanks for the suggestions. I took your original suggestion on using OpenCV for the blinds actually. But I’m keen to try other platforms, so might give your suggestions a whirl for the balloons. Quite keen to dabble with tensorflow.

I don’t actually use the bayesian sensor myself anymore. Just still interested/curious about it (nice write up/tutorial by the way). Main previous application was focussed on home level presence detection and I have it working very reliably by combining multiple sensors in a much simpler way (if any of my devices are home, then I’m home - that is working fine for me). If you have any other interesting/useful use cases then I’d like to hear about them.

Not yet, it is not much more advanced than the tutorial at the moment. I’ll definitely share once I’ve achieved an mvp, right now I’m still just playing around to get an idea of what I can possibly do.

1 Like

On the data tagging theme, I had the thought that having some data pushed at you via actionable notifications and tagged up with your response could be convenient? Like in my computer vision use case - maybe my trained model sends me balloon pictures, then I confirm/negate them for future re-runs? Could maybe do the same with some bayesian sensor changes, then analyse them?

I’ve also started work on inferring indoor lighting levels based on external light sensors for better light automation control. I have one indoor sensor but it gets affected by lights when they turn on! And my external light sensors can give quite different readings (Mi Flora, bh1750).

I’m thinking I train a multiple regression model to predict indoor lighting levels given multiple outdoor readings and the condition that no internal lights are on.

Any thoughts welcome.

I am pretty sure you could use a telegram bot for tagging pictures via notification

1 Like

I do that for my front door face recognition system. It’s using appdaemon, still in beta and not that well documented but maybe you can reuse some of it.

Basic workflow is

  1. When door opens take 10 pictures
  2. Try to detect faces (using Facebox, deestack,…)
  3. Announce when a known face got detected
  4. For unknown face Send image of unknown face with telegram buttons for each known face
  5. If I press one of the buttons it puts the image in the correct directory of known faces
  6. Extra button for a new face where I can write the name of the person
3 Likes

That’s pretty cool.

Anyone doing data science with python has probably heard of streamlit by now. Anyways I just got streamlit demo running as a Hassio addon, which opens up the possibility of creating streamlit data science apps that can be installed as like any other addon. I think I will create one for doing prediction of sensor data using the prophet library.

5 Likes

Any more experiences/progress with this? I’ve just started tinkering with streamlit (pretty slick) and am keen to try something down the line.

There are issues using streamlit on RPi, follow on https://discuss.streamlit.io/t/raspberry-pi-streamlit/2900
Of course you can use streamlit anywhere else (mac, windows etc) but I havent spent any more time on it

1 Like

Hi, finally getting back to this…wondering how you have recorder and history configured?
Do you have auto purge set to never for capturing data in your PostgreSQL database?

I ask bc the docs for recorder warn that doing so will slow down HA but I’m wondering if that is really more relevant for the default SQLite db and/or possibly using an rpi…

Trying to determine if I need a to capture data via Mqtt or creating a custom component like someone did for LTSS (I’m still wanting to try PostgreSQL first before getting into time series databases)

Thanks!

LTSS and the Timescale DB used by ist are based on postgres. Timescale DB contains only the possibility to partition the data automatically by time chunks, which makes time-based queries faster.
Basically it is “only” a Postgres DB with some optimization :wink: So you can use it like any other SQL DB for analyse your data.

For example, I query my data from it directly via the jupyter integration for the MS visual studio code editor. Works wonderfully :slight_smile:

@carver I am running postgres in a docker container on a synology NAS. My config is simply:

recorder:
  db_url: !secret postgres_url
  auto_purge: False

I have relatively few entities in my prod HA, so the db is growing quite slowly.
For analytics timescaleDB would be a good choice, and as @CM000n points out it is a postgres extension, with nice features for sampling time series data. Only drawback I found is that it is not supported on rpi

1 Like

how do I add streamlit to home assistant core?

@abdalians see https://github.com/robmarkcole/hassio-addons and read https://discuss.streamlit.io/t/raspberry-pi-streamlit/2900 requires more work to make this seamless

1 Like

I just wrote a spanish post about how to extract some meaning of those bunch of data directly from sqlite to jupyter notebooks:

https://www.pacienciadigital.com/analisis-de-consumo-de-energia-con-home-assistant/

1 Like

Hi all. I’m trying to use an already trained model saved with pickle and using it in appdaemon app. I’m having problems importing pickle and scklearn to load and use model from file inside appdaemon (hassio).
Can anyone tell if it’s even possible?
Thanks.

This sounds more of an appdaemon question. Might have issues with sklearn installation

Thanks for such details. Data Science Course in Pune.