WTH - is there no Home Assistant logs to remote syslog integration?

Month of "What the heck?!"
17

Sorry for any confusion, I didn’t mean to imply that syslog forwarding was included.

With Home Assistant Supervised you control the OS, so you can do whatever you want with syslog or the rest of the OS.

With the “supervised install”, you are running the Home Assistant Supervisor in a container. The supervisor then manages the Home Assistant container(s), and any add-on containers.

I have only every run HA in a Docker container,
[…]
On the Docker container model where I am, there is a single log file, home-assistant.log , everything dumps into this file.
[…]
You always need to take a step back and understand which of the four installation methods a person is using with Home Assistant

If you are only running the Home Assistant (core) container, then yes there is only a single log for file Home Assistant Core application. I was referring to running the Home Assistant Operating System. So there’s a kernel running and it’s “dmesg” output is what I’d like to forward to be able to see if/when hardware related log messages pop up.

I’m seeing the Home Assistant log entries forwarded to my syslog server. I was going to say I’m not sure what is doing that, but it looks like @CentralCommand just addressed that. I don’t know if docker-ce came with the logging driver that’s forwarding to syslog out of the box, or whether that’s something installing the HA supervisor did. But I’m seeing the STDERR from all containers running on that host forwarded to my syslog server.

3 Likes
18

Yea I mean it basically has to be this way. If HA was only logging to the text file then doing a docker logs homeassistant would show you nothing.

1 Like
19

Hey @rct thank you! You have just taught me something and I am going to try and apply to my setup. Thinking about your setup, I google whether I could get my docker system to forward the output of a container log to rsyslog and it appears so. And my Home Assistant seems to push its message to the docker container log the same as to it’s home-assistant.log . Looks simple, I need to restart docker so that will have to wait for a maintenance window. I am interested to see if this give me another way to look at the full and updated history. Again, thanks for opening my eyes! One of those duh moments that make having other sets of eyes so helpful!

2 Likes
20

You and @rct both opened my eyes to what appears to be good opportunity for docker based Home Assistant, letting Docker sent the containers output to rsyslog, thanks:

4 Likes
21

Thank you very much @dproffer @CentralCommand @rct for all your inputs, suggestions and comments. Great to read all your insights about behind the scenes of syslog in general and Home Assistant in particular. As previously mentioned I run a SynologY NAS VMM based Home Assistant Supervised, so I was searching for a ready to install add-on.

But reading your posts, things are not as simple as I’ve imagined. Is it possible for me to use the Docker logging driver to pull all logs from within my VMM Home Assistant: Core / Supervisor / Host / InfluxDB / Grafana etc?

@dproffer Thanks for providing the link to the Docker Documentation.

22

Nope. Like I said, not supported and blows it up.

You’re welcome to try other ones obviously. But definitely not supported to do so and may cause supervisor to crash and not start again. Be prepared to get out of that, can’t just restore a backup in that situation.

1 Like
23

I had the same use case (wanting to send HA logs to Synology NAS), and I found this: GitHub - TheByteStuff/RemoteSyslog_Service: Home Assistant Custom Component - send Syslog message to remote server.
You need then to set up an automation, like this (you can customise the message etc.):

alias: Syslog to Syno
description: ""
trigger:
  - platform: event
    event_type: system_log_event
condition: []
action:
  - service: remote_syslog.logmessage
    data:
      loglevel: >
        {% set lvl = {'DEBUG':'debug', 'INFO':'info', 'WARNING':'warning',
        'ERROR':'err', 'CRITICAL':'crit'} -%} {{ lvl[trigger.event.data.level]
        }}
      host: [your.syno.IP]
      message: >-
        {{ trigger.event.data.level ~ " " ~ trigger.event.data.timestamp |
        timestamp_local ~ " : " ~ trigger.event.data.message ~ "< " ~
        trigger.event.data.exception ~ " >" ~ trigger.event.data.source }}
      program: "{{ trigger.event.data.name }}"
      port: 514
      protocol: UDP
      facility: USER
mode: queued
max: 30

In your config file, you must first set this to have events firing:

system_log:
  fire_event: true

And then you have to set up the receiving end on your Synology NAS.

2 Likes
24

TIL you can tell HA to fire an event for every logged message. Which I guess makes this WTH a lot easiser since you can send the details of those events wherever you want via an automation.

How big is your DB though? There’s an events table in the DB and every event is logged there. Seems like having each logged message also create a row in the DB would cause it to grow quite rapidly. Or do you exclude this event type in recorder?

1 Like
25

Good idea!
My DB is around 5 Gb. (31 days before purge).

26

Thanks for your reply, I think I may need some reading and tinkering to do tonight. Will report back asap.

Oldest Run Start Time October 10, 2022 at 00:30
Current Run Start Time October 18, 2022 at 16:49
Estimated Database Size (MiB) 1856.80 MiB
Database Engine sqlite
Database Version 3.38.5
27

Oh wait. So I just realized what this is. This isn’t actually firing an event per log, it’s the piece at the top:

Screen Shot 2022-10-18 at 12.34.36 PM
Screen Shot 2022-10-18 at 12.34.36 PM1920×1111 251 KB

It’s not actually firing an event per log entry, its firing an event per error/warning added to the bit at the top. This makes more sense. Probably still a lot to shove into your DB but not nearly as bad as actually duplicating every log entry in there.

That being said, while useful I’m not sure this answers the WTH. The real log often contains more info as well as contextual information from the surrounding logs, particular if you set default level to info. Plus you can change the level of individual components to debug things whereas no matter what you do only warnings and errors show up in the system log bit.

1 Like
28

Yes, great elaboration about the log-files and system log, the “full logs” is what I see for e.g. all my devices in my networks. The ability to collect them on one server is very convenient and helpful debugging issues for me narrowing down towards the root of cause as much as possible.

Just started with HA(Supervised) 2 weeks ago, so slowly learning the YAML coding and careful moving forward. To be continued…

29

m4v3r1ck, What Syslog server do you use on your Synology NAS?

30

The standard Log Center Package from SynologY in DSM 7.1.1

1 Like
31

Similar to @m4v3r1ck I use the built in syslog server in Synology and find it very useful.

It is very easy to create your own syslog server on any machine you have and experiment with collecting and processing log data. It is usually straight forward and easy to change the target IP address of the syslog server in the device/software that you are wanting to get logs from. Again, making it easy to test/debug things on the fly. Below are two easy to setup syslog servers, python and go. There are many others.

1 Like
32

I’ve learned a bit from this thread.

However, I don’t think I’ve learned anything that solves the case where you are running Home Assistant OS on physical hardware and you want to collect kernel/driver messages so you can catch hardware or OS errors.

Yes, you can look at the supervisor logs if the Home Assistant installation is still up/functional.

33

Thanks @rct for your reply and insight, appreciated.

For me personally, it is a more investigative WTH question to why is it missing / what is needed / is it even possible? / to create a log to remote syslog server on a Home Assistant instance.

For sure I agree with you that it needs to have more - like you mentioned to catch hardware or OS errors. - than only copy logfile from a running Home Assistant installation, although having this possibility to be able for all logs and to collect them in my SynologY syslog server would be great.

I’m in no way code savy (on any platform) so I depend on developers for this.

1 Like
34

There is an Add-On for Logspout that seems to work well on HAOS

https://community.home-assistant.io/t/logspout-add-on-for-sending-ha-logs-to-log-management-systems/423152

1 Like
35

Thank you John for your reply. I’ll have a look at your Add-On. Keep you posted.

36

Reading thru all related threads, work in progress…