WTH - is there no Home Assistant logs to remote syslog integration?

Month of "What the heck?!"
21

Thank you very much @dproffer @CentralCommand @rct for all your inputs, suggestions and comments. Great to read all your insights about behind the scenes of syslog in general and Home Assistant in particular. As previously mentioned I run a SynologY NAS VMM based Home Assistant Supervised, so I was searching for a ready to install add-on.

But reading your posts, things are not as simple as I’ve imagined. Is it possible for me to use the Docker logging driver to pull all logs from within my VMM Home Assistant: Core / Supervisor / Host / InfluxDB / Grafana etc?

@dproffer Thanks for providing the link to the Docker Documentation.

22

Nope. Like I said, not supported and blows it up.

You’re welcome to try other ones obviously. But definitely not supported to do so and may cause supervisor to crash and not start again. Be prepared to get out of that, can’t just restore a backup in that situation.

1 Like
23

I had the same use case (wanting to send HA logs to Synology NAS), and I found this: GitHub - TheByteStuff/RemoteSyslog_Service: Home Assistant Custom Component - send Syslog message to remote server.
You need then to set up an automation, like this (you can customise the message etc.):

alias: Syslog to Syno
description: ""
trigger:
  - platform: event
    event_type: system_log_event
condition: []
action:
  - service: remote_syslog.logmessage
    data:
      loglevel: >
        {% set lvl = {'DEBUG':'debug', 'INFO':'info', 'WARNING':'warning',
        'ERROR':'err', 'CRITICAL':'crit'} -%} {{ lvl[trigger.event.data.level]
        }}
      host: [your.syno.IP]
      message: >-
        {{ trigger.event.data.level ~ " " ~ trigger.event.data.timestamp |
        timestamp_local ~ " : " ~ trigger.event.data.message ~ "< " ~
        trigger.event.data.exception ~ " >" ~ trigger.event.data.source }}
      program: "{{ trigger.event.data.name }}"
      port: 514
      protocol: UDP
      facility: USER
mode: queued
max: 30

In your config file, you must first set this to have events firing:

system_log:
  fire_event: true

And then you have to set up the receiving end on your Synology NAS.

4 Likes
24

TIL you can tell HA to fire an event for every logged message. Which I guess makes this WTH a lot easiser since you can send the details of those events wherever you want via an automation.

How big is your DB though? There’s an events table in the DB and every event is logged there. Seems like having each logged message also create a row in the DB would cause it to grow quite rapidly. Or do you exclude this event type in recorder?

1 Like
25

Good idea!
My DB is around 5 Gb. (31 days before purge).

26

Thanks for your reply, I think I may need some reading and tinkering to do tonight. Will report back asap.

Oldest Run Start Time October 10, 2022 at 00:30
Current Run Start Time October 18, 2022 at 16:49
Estimated Database Size (MiB) 1856.80 MiB
Database Engine sqlite
Database Version 3.38.5
27

Oh wait. So I just realized what this is. This isn’t actually firing an event per log, it’s the piece at the top:

Screen Shot 2022-10-18 at 12.34.36 PM
Screen Shot 2022-10-18 at 12.34.36 PM1920×1111 251 KB

It’s not actually firing an event per log entry, its firing an event per error/warning added to the bit at the top. This makes more sense. Probably still a lot to shove into your DB but not nearly as bad as actually duplicating every log entry in there.

That being said, while useful I’m not sure this answers the WTH. The real log often contains more info as well as contextual information from the surrounding logs, particular if you set default level to info. Plus you can change the level of individual components to debug things whereas no matter what you do only warnings and errors show up in the system log bit.

1 Like
28

Yes, great elaboration about the log-files and system log, the “full logs” is what I see for e.g. all my devices in my networks. The ability to collect them on one server is very convenient and helpful debugging issues for me narrowing down towards the root of cause as much as possible.

Just started with HA(Supervised) 2 weeks ago, so slowly learning the YAML coding and careful moving forward. To be continued…

29

m4v3r1ck, What Syslog server do you use on your Synology NAS?

30

The standard Log Center Package from SynologY in DSM 7.1.1

1 Like
31

Similar to @m4v3r1ck I use the built in syslog server in Synology and find it very useful.

It is very easy to create your own syslog server on any machine you have and experiment with collecting and processing log data. It is usually straight forward and easy to change the target IP address of the syslog server in the device/software that you are wanting to get logs from. Again, making it easy to test/debug things on the fly. Below are two easy to setup syslog servers, python and go. There are many others.

1 Like
32

I’ve learned a bit from this thread.

However, I don’t think I’ve learned anything that solves the case where you are running Home Assistant OS on physical hardware and you want to collect kernel/driver messages so you can catch hardware or OS errors.

Yes, you can look at the supervisor logs if the Home Assistant installation is still up/functional.

33

Thanks @rct for your reply and insight, appreciated.

For me personally, it is a more investigative WTH question to why is it missing / what is needed / is it even possible? / to create a log to remote syslog server on a Home Assistant instance.

For sure I agree with you that it needs to have more - like you mentioned to catch hardware or OS errors. - than only copy logfile from a running Home Assistant installation, although having this possibility to be able for all logs and to collect them in my SynologY syslog server would be great.

I’m in no way code savy (on any platform) so I depend on developers for this.

1 Like
34

There is an Add-On for Logspout that seems to work well on HAOS

https://community.home-assistant.io/t/logspout-add-on-for-sending-ha-logs-to-log-management-systems/423152

1 Like
35

Thank you John for your reply. I’ll have a look at your Add-On. Keep you posted.

36

Reading thru all related threads, work in progress…

37

Thanks. It was very useful for me!

38

Tying up some old threads noticed there wasn’t a link here for @mib1185’s mib1185/ha-add-on-syslog.

It is a pretty simple syslog forwarder add-on. There’s a bit of python that reads from journald and sends to syslog, so a very minimal footprint. Note that this is for system logs (the stuff from the kernel and its drivers, daemons like docker, etc.). So good for catching hardware/OS related stuff.

Original post in another thread: Export logs to a remote syslog - #17 by mib1185

39

Thanks for mentioning this :slightly_smiling_face:
But just one correction - it is not only “system” stuff, it is everything what’s logging on HAOS, so also homeassistant-core and other installed AddOns :wink: