My Docker stack - home automation and more

nicxvan · March 9, 2019, 3:02am

@flamingm0e you were extremely helpful getting my docker setup and running last year. I promised to share mine once it gets stable.

It’s continued to grow so I thought I’d do a status update: This isn’t everything, since come of the customizations are specific to my local setup. I’ll put some explanation in the next comment but feel free to ask any questions.

version: '3.1'
services:
  nginx:
    container_name: nginx
    restart: unless-stopped
    build:
      context: ./containers/nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /srv/docker/nginx:/var/log/nginx
      - /srv/docker/letsencrypt:/etc/letsencrypt
  jenkins:
    container_name: jenkins
    restart: unless-stopped
    image: jenkins/jenkins
    ports:
      - "8080:8080"
    volumes:
      - /srv/docker/jenkins:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
  portainer:
    container_name: portainer
    restart: unless-stopped
    image: portainer/portainer
    volumes:
      - /srv/docker/portainer:/data
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - "9000:9000"
  homeassistant:
    container_name: home-assistant
    restart: unless-stopped
    image: homeassistant/home-assistant
    depends_on:
      - "mariadb"
    devices:
      - /dev/ttyACM0:/dev/ttyACM0
    volumes:
      - /srv/docker/hass-config:/config
      - /etc/localtime:/etc/localtime:ro
    network_mode: host
    privileged: true
  mariadb:
    container_name: mariadb
    restart: unless-stopped
    build:
      context: ./containers/mariadb
    volumes:
      - /mnt/mariadb:/var/lib/mysql
    ports:
      - ${MARIADB_PORT}:3306
    network_mode: bridge
    environment:
      MYSQL_ROOT_PASSWORD: ${MARIADB_PASSWORD}
  openvpn:
    build:
      context: ./containers/openvpn
    cap_add:
      - NET_ADMIN
    container_name: openvpn
    restart: unless-stopped
    volumes:
      - /srv/docker/openvpn-data/conf:/etc/openvpn
    network_mode: bridge
    ports:
      - "8999:1194/udp"
  serviio:
    build:
      context: ./containers/serviio
    container_name: serviio
    restart: unless-stopped
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/docker/serviio/config:/config
      - /mnt/video:/media/video
      - /mnt/music:/media/music
      - /srv/docker/serviio/transcode:/transcode
    network_mode: bridge
    privileged: true
    ports:
      - "8111:80"
      - "23423:23423"
      - "23424:23424"
      - "8895:8895"
      - "1900:1900/udp"
  handbrake:
    build:
      context: ./containers/handbrake
    container_name: handbrake
    restart: unless-stopped
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/docker/handbrake/config:/config:rw
      - /mnt/video:/storage
      - /srv/docker/handbrake/watch:/watch:rw
      - /mnt/video/output:/output:rw
    network_mode: bridge
    privileged: true
    ports:
      - "5800:5800"
  influxdb:
    container_name: influxdb
    restart: unless-stopped
    image: influxdb
    volumes:
      - /srv/docker/influxdb/influxdb.conf:/etc/influxdb/influxdb.conf:ro
      - /srv/docker/influxdb/db:/var/lib/influxdb
    environment:
      - INFLUX_GRAPHITE_ENABLED='true'
    ports:
      - "8086:8086"
  chronograf:
    container_name: chronograf
    restart: unless-stopped
    image: chronograf
    depends_on:
      - "influxdb"
    volumes:
      - /srv/docker/chronograf:/var/lib/chronograf
    ports:
      - "8888:8888"
  grafana:
    container_name: grafana
    restart: unless-stopped
    image: grafana/grafana:5.0.0
    depends_on:
      - "influxdb"
    volumes:
      - /srv/docker/grafana:/var/lib/grafana
    ports:
      - "3000:3000"
  nodered:
    build:
      context: ./containers/nodered
    container_name: nodered
    restart: unless-stopped
    user: root
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /srv/docker/nodered/config:/data
    ports:
      - "1880:1880"
  elasticsearch:
    container_name: elasticsearch
    restart: unless-stopped
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.4.2
    environment:
      - LOGSPOUT=ignore
      - cluster.name=docker-cluster
      - "ES_JAVA_OPTS=-Xms2048m -Xmx2048m"
    network_mode: host
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - elasticdata:/usr/share/elasticsearch/data
  filebeat:
    container_name: filebeat
    restart: unless-stopped
    build:
      context: ./containers/filebeat
    user: root
    volumes:
      - /srv/docker/nginx:/usr/share/filebeat/outsidelogs/nginx  
      - /srv/docker/hass-config/home-assistant.log:/usr/share/filebeat/outsidelogs/homeassistant/home-assistant.log
  kibana:
    container_name: kibana
    restart: unless-stopped
    image: docker.elastic.co/kibana/kibana-oss:6.4.2
    environment:
      - LOGSPOUT=ignore
      - ELASTICSEARCH_URL="http://172.16.17.20:9200"
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
  piwigo:
    container_name: piwigo
    restart: unless-stopped
    build:
      context: ./containers/piwigo
    depends_on:
      - "mariadb"
    volumes:
      - /mnt/piwigo/data/galleries:/var/www/galleries
      - /srv/docker/piwigo/data/local:/var/www/local
      - /srv/docker/piwigo/data/plugins:/var/www/plugins
      - /srv/docker/piwigo/data/themes:/var/www/themes
      - /mnt/piwigo/cache:/var/www/_data/i
      - /mnt/piwigo/upload:/var/www/upload
      - /srv/docker/piwigo/log:/var/log
      - /srv/docker/piwigo/log/apache2:/var/log/apach2
      - /etc/localtime:/etc/localtime:ro
    network_mode: bridge
    ports:
      - 2500:80
    hostname: piwigo
    domainname: piwigo.nj.lan
  nutups:
    container_name: nut_ups
    restart: always
    build:
      context: ./containers/nutupsd
#    image: instantlinux/nut-upsd:latest
    environment:
      - VENDORID=09ae
      - DESCRIPTION=Tripp Lite SMART1500LCD
      - POLLINTERVAL=12
    network_mode: host
    ports:
      - 3493:3493
    devices:
      - /dev/usb/hiddev0:/dev/usb/hiddev0
    privileged: true
    volumes:
      - /srv/docker/nutups:/etc/nut/local
    secrets:
      - nut-upsd-password
  nextcloud:
    container_name: nextcloud
    image: nextcloud
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=root
      - MYSQL_HOST=${MARIADB_HOST}
      - MYSQL_PASSWORD=${MARIADB_PASSWORD}
    ports:
      - 8500:80
    volumes:
      - /srv/docker/nextcloud/html:/var/www/html
      - /srv/docker/nextcloud/apps:/var/www/html/custom_apps
      - /srv/docker/nextcloud/config:/var/www/html/config
      - /srv/docker/nextcloud/theme:/var/www/html/theme
      - /mnt/nextcloud/data:/var/www/html/data
      - /srv/docker/nextcloud/apachelogs:/var/log/apache2
  unms:
    image: oznu/unms:latest
    restart: always
    network_mode: bridge
    ports:
      - 2080:80
      - 2443:443
      - 2055:2055/udp
    environment:
      - TZ="America/New_York"
    volumes:
      - /srv/docker/unms:/config
  autoheal:
    restart: always
    image: willfarrell/autoheal
    environment:
      - AUTOHEAL_CONTAINER_LABEL=all
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  clarity:
    restart: always
    image: spira/docker-clarity
    ports:
      - 2900:8989
    volumes:
      - /srv/docker/clarity:/data/logs
      -  /srv/docker/nginx:/data/logs/nginx
      - /srv/docker/hass-config/home-assistant.log:/data/logs/homeassistant/home-assistant.log
secrets:
  nut-upsd-password:
    file: ./nutpass
volumes:
  elasticdata:
    driver: local
    driver_opts:
      type: nfs
      o: addr=xxx.xxx.xx.xx,rw,async,noatime,nodiratime,nfsvers=4
      device: ":/volume2/elasticsearch"

#  mqtt:
#    container_name: MQTT
#    restart: unless-stopped
#    image: eclipse-mosquitto
#    volumes:
#      - /srv/docker/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf
#      - /srv/docker/mosquitto/log:/mosquitto/log
#      - /srv/docker/mosquitto/data:/mosquitto/data
#      - /etc/localtime:/etc/localtime:ro
#    ports:
#      - "1883:1883"
#      - "9001:9001"
#  kodi:
#    container_name: kodi
#    image: linuxserver/kodi-headless
#    restart: unless-stopped
#    volumes:
#      - /etc/localtime:/etc/localtime:ro
#      - /srv/docker/kodi/config:/config/.kodi
#      - /mnt/video:/media/video
#    ports:
#      - "6080:8080"
#      - "9090:9090"
#      - "9777:9777/udp"
#  snips:
#    build:
#      context: ./containers/snips
#    container_name: snips
#    restart: unless-stopped
#    user: root
#    volumes:
#      - /etc/localtime:/etc/localtime:ro
#  logstash:
#    container_name: logstash
#    image: docker.elastic.co/logstash/logstash-oss:6.4.2
#    environment:
#      - LOGSPOUT=ignore
#      - LS_HEAP_SIZE="2g"
#    network_mode: bridge
#    ports:
#      - "5000:5000/udp"
#      - "5000:5000"
#      - "9600:9600"
#      - "5044:5044"
#    volumes:
#      - /srv/docker/logstash/pipeline/:/usr/share/logstash/pipeline/
#      - /srv/docker/logstash/logstash.yml:/usr/share/logstash/logstash.yml
#    depends_on:
#      - elasticsearch
#  zoneminder:
#    build:
#      context: ./containers/zoneminder
#    container_name: zoneminder
#    volumes:
#      - /srv/docker/zoneminder/events:/var/lib/zoneminder/events
#      - /srv/docker/zoneminder/images:/var/lib/zoneminder/images
#      - /srv/docker/zoneminder/mysql:/var/lib/mysql
#      - /srv/docker/zoneminder/logs:/var/log/zm
#    environment:
#      - shm-size="512m"
#      - TZ="America/New_York"
#      - ZM_DB_USER=${ZM_DB_USER}
#      - ZM_DB_PASS=${MARIADB_PASSWORD}
#      - ZM_DB_NAME=${ZM_DB_NAME}
#      - ZM_DB_HOST=${ZM_DB_HOST}
#    network_mode: bridge
#    ports:
#      - "8775:80"
#      - "8776:443"
 #  eapcontroller:
 #   build:
 #     context: ./containers/eapcontroller
 #   container_name: eapcontroller
 #   restart: unless-stopped
 #   volumes:
 #     - /etc/localtime:/etc/localtime:ro
 #     - /srv/docker/eap/config:/config
 #   network_mode: host
 #   privileged: true
 #   ports:
 #     - "8088:8088"

nicxvan · March 9, 2019, 3:25am

First Nginx - it’s pretty standard, I build my own only so I can include conf at build time. I have a parent conf with a separate include for each domain I’m using. The only special thing here is I’m using wildcard certs to handle ssl so I don’t need a new one for each service I add.

Jenkins is also pretty standard, I hook it up to docker’s sock so I can make curl requests to shut down / start services as needed. Currently it just stops services when my ups get’s low, but I plan on running some backups and manage renewing certs at some point.

Portainer - gets used to manage the stack a LOT I am a huge fan of being able to ssh into the containers from the web. It also makes updating home assistant a breeze.

Home assistant - what we all know and love.

Mariadb - I prefer this over mysql, I used my own image here because I wanted to include my own configuration.

OpenVPN allows me to vpn back to my network so no other services need to be web accessible. I built my own image so I could audit it more closely since I was going to be exposing it.

Serviio an open source alternative to plex. I am mildly happy with it. I do not use it too much and may swap it out. I built this one because I went through a short period of tie where I wanted to build all images locally. I have since stopped that madness.

Handbrake - I needed to convert some old videos to a better format for roku. Doesn’t get much use, but worked for what I needed it to do. Same reason above for building myself.

InfluxDb - I added this because I wanted to have some nice grafana graphs

Chronograf - Same as above - management interface for influx dbs

Grafana - nice graphing interface. I use it primarily to track energy usage and temperatures.
It’s extremely powerful and nice, but doesn’t seem to do exactly what I want without a lot of finagling.

Nodered - this gets a lot of use. I do almost all automation here and use home assistant as a state manager / state viewer. I use the ha interface for switches etc. I build my own so that any custom nodes I add get added automatically at build.

Elasticsearch - I wanted a better way to visualize home assistant logs - this stores the logs in a searchable database. Not quite configured.

Filebeat - A helper to elasticsearch it moves files into elastic. I build my own to get some custom config.

Kibana - a nicer front end to elasticsearch. I haven’t finished configuring it.

Piwigo - a php based image gallery. I use it for home images. It’s not the nicest interface, but I’m happy with it. I built my own image so I could inspect it a bit closer and tweak some php settings.

Nutups - a container for running nutups a monitor for UPS I build my own to add a fair amount of customization to get it working for my setup. I also set up a custom healthcheck used below.

Nextcloud - an open source google docs / apps suite. Really slick I’m very happy with it but not fully utilizing it just yet.

UNMS - a network monitor by ubiquiti (this container isn’t official though as the official one requires about a dozen containers to get working) This is very slick and I am extremely happy with it. I can’t wait for more support for equipment outside of ubiquiti.

Autoheal - a container that watches other container’s healthchecks and restarts them if unhealthy. Used to fix nutups issues when it goes stale.

Clarity - a service for viewing logs in the browser. I’m mildly happy with it.

mqtt haven’t had the need for it yet - but kept it for future reference
kodi wasn’t able to get it working but haven’t decided 100% for serviio
snips a fairly unsuccessful attempt to dockerfy the snips.ai service.
logstash a helper for elastic I didn’t use
zoneminder an open source surveillance monitor I couldn’t get working well.
eapcontroller a controller for tplink access points. The controller was HUGE and didn’t have as many features as the web interface for the ap’s

nicxvan · March 9, 2019, 3:31am

@flamingm0e just added a rough outline, thanks again for your help!

juan11perez · March 9, 2019, 4:02am

Thank you for sharing.
Have a look at this media server
https://hub.docker.com/r/binhex/arch-jellyfin

Its working well for me.

nicxvan · March 9, 2019, 6:17pm

@juan11perez thanks for the tip! I just installed it and it’s already miles better than serviio. Thanks!

juan11perez · March 9, 2019, 6:33pm

great. glad it worked.
not tried their android app, but there’s one

finity · March 9, 2019, 6:36pm

@juan11perez

Does the jellyfish media server act as a DLNA media server for serving content to set top media boxes?

And does it handle photos, too, or just audio/video?

finity · March 9, 2019, 6:39pm

What kind of customizations did you need? I have a CyberPower 1500 I’d like to use this to monitor. Is there anything special I would need to do to get this working in Docker?

nicxvan · March 9, 2019, 8:23pm

It does have DLNA settings, but I have not yet configured it for my roku, it does look like it can handle tv tuners too. I had no trouble adding music and video. I have not tried adding photos.

nicxvan · March 9, 2019, 8:36pm

I’m not sure to be honest if it will work, my issue is that the model I have (tripplight 1500 smart lcd) just would stop sending data.

The customizations were basically increasing timeouts, adding a heathcheck and adding the autoheal container. YMMV.

Dockerfile

MAINTAINER Rich Braun "[email protected]"
ARG BUILD_DATE
ARG VCS_REF
LABEL org.label-schema.build-date=$BUILD_DATE \
    org.label-schema.license=GPL-2.0 \
    org.label-schema.name=nut-upsd \
    org.label-schema.vcs-ref=$VCS_REF \
    org.label-schema.vcs-url=https://github.com/instantlinux/docker-tools

ARG NUT_VERSION=2.7.4-r6
ENV API_USER=upsmon \
    DESCRIPTION=UPS \
    DRIVER=usbhid-ups \
    GROUP=nut \
    NAME=ups \
    PORT=auto \
    SECRET=nut-upsd-password \
    SERIAL= \
    SERVER=master \
    USER=nut

RUN echo '@edge http://dl-cdn.alpinelinux.org/alpine/edge/main' \
      >>/etc/apk/repositories && \
    echo '@testing http://dl-cdn.alpinelinux.org/alpine/edge/testing' \
      >>/etc/apk/repositories && \
    apk add --update nut@testing=$NUT_VERSION \
      libcrypto1.1@edge libssl1.1@edge net-snmp-libs@edge

RUN apk --update --no-cache add curl

EXPOSE 3493
COPY ups.conf /etc/nut/ups.conf
COPY upsmon.conf /etc/nut/upsmon.conf
COPY entrypoint.sh /usr/local/bin/
HEALTHCHECK CMD upsc ups@localhost:3493 2>&1 | grep stale -c && exit 1 || exit 0
ENTRYPOINT /usr/local/bin/entrypoint.sh

The customizations are basically the ups.conf, upsmon.conf and the HEALTHCHECK

ups.conf
[ups]
driver = usbhid-ups
port = /dev/usb/hiddev0
vendorid = 09ae
desc = “Tripp Lite SMART1500LCD”

upsmon.conf

I can’t copy this cause there is a password in it and it’s very long, but I basically just connected to the container and copied the default upsmon.conf.

I then changed two lines:
DEADTIME 30 # I don’t think this is necessary
then near the bottom
MONITOR ups@localhost 1 upsmon PASSWORD master

I think I added the MONITOR but I don’t really recall.

Then entrypoint I copied this one:

github.com

instantlinux/docker-tools/blob/main/images/nut-upsd/entrypoint.sh

#! /bin/sh -e

if [ -d /run/secrets ] && [ -s /run/secrets/$SECRET ]; then
  API_PASSWORD=$(cat /run/secrets/$SECRET)
fi

if [ ! -e /etc/nut/.setup ]; then
  if [ -e /etc/nut/local/ups.conf ]; then
    cp /etc/nut/local/ups.conf /etc/nut/ups.conf
  else
    if [ -z "$SERIAL" ] && [ $DRIVER = usbhid-ups ] ; then
      echo "** This container may not work without setting for SERIAL **"
    fi
    cat <<EOF >>/etc/nut/ups.conf
[$NAME]
        driver = $DRIVER
        port = $PORT
        desc = "$DESCRIPTION"
EOF
    if [ ! -z "$SERIAL" ]; then

This file has been truncated. show original

and I made the following modifications:
I set the API_PASSWORD=PASSWORD where the password matches the one in upsmon.conf

I commented out the lines that created ups.conf and upsmon.conf

Honestly I would test the image that instant linux provides first and see if you need modifications.

The docker-compose has the nutups password secret, but I don’t think it’s actually used.

So what happens is the health check sees that the data is stale and sets it to unhealthy. Then autoheal restarts it. I have node red monitoring it from home assistant as well and if it returns below a certain length of time I have node red kick off a jenkins job that kills the containers I need killed and I will eventually add the auto safe shutdown of my router and nas.

Sorry if that’s rambling, but it was fairly complex to get working.

flamingm0e · March 9, 2019, 9:35pm

Rather than building your own images why not just have conf files mounted to the container?

nicxvan · March 9, 2019, 9:50pm

For the nutups case specifically I had to build my own also for the healthcheck. Also the entrypoint generates the conf files so it was overwriting my conf files when I tried to just mount them.

I’m slowly converting most of the instances to use just a specific image.

juan11perez · March 10, 2019, 9:04am

@finity
dlna enabled, but I havent used it.
Not sure on photos. here’s the docs
https://jellyfin.readthedocs.io/en/latest/

SA_guy · March 26, 2019, 3:40pm

Wow, this looks great.
I’m relatively new to Docker and so far have a few containers running:

home assistant configurator (don’t use this much as edit files directly on my server)
home assistant
mosquito
hass-io supervisor
influxdb (just added this last week, but broke it already )
grafana, not quite working thanks to influxdb
portainer

I run openvpn on a raspberry pi that has nothing else running on it. Just openvpn

I’ve got a few questions if I may:

I’m feeling like my solution is very fragment, and containers don’t start up in the correct order. Is a docker compose file a better way to go about this? I assume that’s what your snippet is
How do I best backup/store my data/configuration, are you mounting directories from your local to the containers or using docker volumes?
How would I go about moving my existing containers to be a part of a docker compose file, would my current containers be overwritten if I go and just put the config into a docker compose file?

nicxvan · March 26, 2019, 4:38pm

Yes docker compose makes it easy to specify dependencies between containers “depends_on” is the flag for that.
I mount volumes to store key config: ```
volumes:
- /srv/docker/nginx:/var/log/nginx

3. You would have to probably stop them and start the docker compose ones up separately, if you use the same volumes the config should be fine.

docker-compose build will build the images
docker-compose up -d will run the containers in the background.

Jonde · January 17, 2020, 3:32pm

Nice configuration you got!

Have finished your Kibana dashboard? Would like to see what you have done with it!

nicxvan · January 17, 2020, 3:57pm

I actually stopped using Kibana, I wasn’t able to get it to parse the HA logs consistently and moved on to other projects.