Elasticsearch component - publish home-assistant events to Elasticsearch!

Hey all,

I’ve been working on a custom component to publish hass events to Elasticsearch. It’s been working great for me, but I’d love to get the community’s feedback and input:

8 Likes

this is awesome, thanks for writing it and doubly so for sharing! :smiley:

I did run into this when first setting things up;

2018-10-01 15:41:23 INFO (MainThread) [homeassistant.setup] Setting up elastic
2018-10-01 15:41:23 ERROR (MainThread) [homeassistant.setup] Error during setup of component elastic
Traceback (most recent call last):
File "/usr/src/app/homeassistant/setup.py", line 145, in _async_setup_component
hass, processed_config)
File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 212, in coro
res = func(*args, **kw)
File "/config/custom_components/elastic.py", line 70, in async_setup
gateway = ElasticsearchGateway(hass, conf)
File "/config/custom_components/elastic.py", line 122, in __init__
self.client = self._create_es_client()
File "/config/custom_components/elastic.py", line 130, in _create_es_client
import elasticsearch
ModuleNotFoundError: No module named 'elasticsearch'

I had to pip install elasticsearch==6.3.1 to get things running correctly. now I can see events coming in so I’ll wait a bit and start with a dashboard or two to shar

Hey @warkolm,
I wish I knew how to resolve that issue, as you’re not the first to report it. As far as I can tell, I’m declaring the dependency correctly, so home-assistant should be installing the elasticsearch module automatically. Other users have had success simply restarting home-assistant after getting this error, but it’d be great if I could get this to “just work” the first time around

Did you manage to create a more robust version? I want to move away from influxdb/grafana and substitute it for elasticsearch/Kibana

@fversteegen

By “more robust”, are you asking if I’ve resolved the installation issue? I haven’t tracked the root cause down, but that certainly isn’t a blocker for using this component. I know it’s being used by several people without problems, and we’ve added a handful of new features since this was originally posted.

I encourage you to give it a try, and let me know if you have any feedback. If you run into that install error, simply restarting home-assistant should fix your setup.

Hello,

thank you very much for this component.
After the first restart I had the same error message but a second restart fixed it.

Looking forward to building my first visualizations.

Greetings from Germany!
Heiner

1 Like

Nice work!

1 Like

Thanks for your work! Would like to see what you have done with the data :slight_smile:

1 Like

This is amazing, thanks a lot. Works just great and no issue setting this up with ES version 7.6.1 today. Thanks for the great work.

1 Like

Amazing contribution, very impressive in quality/simplicity and much appreciated. Thanks again for this @legrego!!!

1 Like

Hey @legrego question for you: would it be possible to publish the data from the EVENTS table not just the STATES table? Let me know if it makes sense to throw this into GitHub for possibly a future enhancement — or if I’m missing something obvious :slight_smile: I do see mappings for it in your index_mapping.json file just nothing populating the index.
All the best!

Works a treat after a couple of HASS service restarts. My only complication was, being new to HASS, I didn’t know where to put the custom directory. Turns out it was just the same directory as the main config file. Once I’d managed to upload the code to the appliance (another small wrinkle) everything just flowed in. Excellent work sir :slight_smile:

You will need to create an “Index Pattern” called hass-events-* before you can see anything in “Discover”. This held me up a bit as well but more down to my inexperience with customizing ES than anything else.

One thing for newbies like me to note, it does take a while for ES to start actually surfacing the data, particularly if you’ve got data piling in from elsewhere.

fyi:

is it possible to publish journal logs with this? i’m trying to publish crowdsec journal logs to my elk

No, journal logs are not supported by this component