This is still work in progress, but I prefer continuing the discussion here rather than on a git issue .
The monit tool is generally used for monitoring services on a machine and restart them when necessary. For instance, I use it to monitor apache2, mysql, proprietary services, applications, etc. on servers, embedded systems using *nix, etc.
I discovered it can be installed on HA OS/Core using the “Terminal + SSH” add-on. I haven’t configured it yet, but in order to document it for the community, I started writing a script.
To enable monit in “Terminal + SSH”, you have have several options.
You can it to the ‘apks’ of the “Terminal + SSH” configuration as shown in the example that also includes nmap:
authorized_keys: []
apks:
- monit
- nmap
password: ''
server:
tcp_forwarding: false
You can also install packages from the command line:
# Install monit
apk add monit
# Look for packages
apk search sqlite
# Install a package - sqlite installs sqlite3 in fact
# - only specify the main part of the package.
# (sqlite could be helpful to check that the datebase is still updating).
apk add sqlite
After saving the configuration and restarting the add-on, monit should be available in the terminal.
You can verify that monit is running and accessible from CLI:
monit summary
The script below is work in progress and not tested at all at this time.
What it will be doing when tested/finished:
- Create /etc/monit.d to put the configurations into in separate files.
- Modify monitrc to include the files in that directory.
- Add some configurations for monitoring services, including home assistant.
- Add some monitoring to monitor cpu load (and restart HA).
- Reload the configuration to include the new configurations.
- Report a status (which will correspond to “Initializing”) to check that monit is ok.
# Host and port for HA,
# `curl http://homeassistant:8123` gives home page
# `curl http://homeassistant:8123/api/` gives '401: Unauthorized' (not ... Forbidden):
#
# or set to other working host, sometimes pubic host works `curl https://publichaurl`
# update protocol http to https if needed
# (Testing https, also tests the nginx proxy)
HA_HTTP_HOST=homeassistant
HA_HTTP_PORT=8123
PROTOCOL=http
# Otherwise, define HA_HTTP_HOST to empty value
EMAILLOGIN="YOUREMAILLOGIN"
EMAILPASSWORD="YOUREMAILPASSWORD"
MAILSERVER=YOURMAILSERVER
mkdir -p /etc/monit.d
# change line with uncomment include of monit.d
sed -i 's@#\(\s*include\s*/etc/monit.d\)@\1@' /etc/monitrc
sed -i 's@set log syslog@set log /var/log/monit.log@' /etc/monitrc
if [ "$HA_HTTP_HOST" != "" ] ; then
cat > /etc/monit.d/ha <<EOHA
check host ha with address $HA_HTTP_HOST
# with pidfile PATHTOPIDFILE
if failed
port $HA_HTTP_PORT protocol $PROTOCOL
# Expect unauthorized, shows that the server is up
# Could check status if API key provided.
#and request /api/ with content = "401: Unauthorized" with timeout 20 seconds
and request / with timeout 20 seconds
then
# Alert, for testing
alert
# Change to exec when confident that this works
# exec "/usr/bin/ha core restart"
group ha
EOHA
fi
# RESTART HA when CPU LOAD is high for 10 minutes
cat > /etc/monit.d/checkload <<EOCPUHIGH
check system homeassistant.local
if loadavg (1min) per core > 1.5 for 10 cycles then alert
if loadavg (1min) per core > 1.5 for 15 cycles then exec "/usr/bin/ha core restart"
EOCPUHIGH
# Its possible to set up mail alerts.
# Example for server using TLS on port 587:
cat > /etc/monit.d/mailserver <<EOMS
set mailserver
$MAILSERVER port 587 username "$EMAILLOGIN" password "$EMAILPASSWD" using tlsv1
EOMS
monit
monit reload
sleep 1
monit summary
