Logger integration: Allow multiple log files

Sometimes I need extra logging to troubleshoot some integration (like zha):

logger:
  default: info
  logs:
    homeassistant.components.zha : debug
    zigpy : debug
    zigpy_deconz : debug

This will add thousands of lines to the home-assistant.log file.
It might be nice to be able to log messages to separate files.
So log messages from a specific integration (like zha) to a separate zha.log file.

Mockup of configuration yaml:

logger:
  default: info
  logs:
    homeassistant.components.plex.media_player : error
  additional_logging:
    - filename: 'logs/mqtt.log'
      enable: true
      logs:
        homeassistant.components.mqtt : debug
    - filename: 'logs/zha.log'
      enable: true
      logs:
        homeassistant.components.zha : debug
        zigpy : debug
        zigpy_deconz : debug
        custom_components.zha_map : debug
        bellows : debug

We also need to be able to enable / disable logging without restart of HA.
Is this something more people would find usefull?

Should this feature be rejected, or in the meantime until it is implemented, you can (try and) achieve the same result by adding filters in rsyslog.

  1. create a new file in /etc/rsyslog.d/, e.g. called 01-myfilters.conf
  2. specify your filter criteria, and the logfile to put your debug messages.
  3. validate your new configuration to ensure it is valid.
    rsyslogd -f /etc/rsyslog.conf -N1
  4. restart rsyslog to read in the new configuration.
    sudo systemctl restart rsyslog
  5. check the rsyslog status if everything is still ok.
    systemctl status rsyslog

Filter examples:

# 1) Put all zigpy related messages in a separate log.
if $msg contains "zigpy" then -/tmp/logs/zha.log

# 2) Ignore all zipgy debug messages (debug = level 7).
if $msg contains "zigpy" and $syslogseverity >= '7' then {  stop }

Put your new logfile in a TMPFS directory to ensure it is stored in memory, to avoid killing your SD / SSD with huge amounts of write statements.

All that extra log data will also be in the systemd-journal. Another option is therefore to not put the debug statements in a file, but query them from the journal using journalctl filtering.

e.g.

# all messages between two dates:
journalctl --since "2021-03-08 10:00:00" --until "2021-03-08 11:00:00"

# Realtime view of all messages logged by a specific container (e.g. hassio_superivisor):
journalctl -f CONTAINER_NAME=hassio_supervisor

# Specific messages logged by a specific container (e.g. hassio_superivisor):
journalctl CONTAINER_NAME=hassio_supervisor | grep -i "zigpy"

# See all fields in journalctl that you can use to filter:
journalctl -f CONTAINER_NAME=hassio_supervisor --since yesterday -o json-pretty

can this be done with Home Assistant OS?

1 Like

Would be an awesome feature to have filename parameter for custom log files, yes :smiley:

1 Like