Great stuff. Glad CI/CD in the HASS world is getting traction. Thanks for sharing!
Here’s my journey with HASS-related CI (and a very simplistic CD). In order to keep my code private, I use Gitea as a self-hosted Git service and Drone as my CI platform. Although syntax/etc. is different from Travis, GitLab, etc., the same concepts apply – if nothing else, the scope of what I’m testing might be of interest.
What I Test
AppDaemon: I run various tools across my AppDaemon apps to ensure that the code is linted, Black-ified, and type-checked. Additionally, although it’s not there today, I’m slowly working on incorporating the AppDaemon Test Framework, as well, so that unit tests will be a part of CI.
ESPHome: I run config checks against both beta and latest versions of ESPHome. I also run a YAML linter on my configuration files.
Home Assistant: I run config checks against dev, rc, and latest versions of HASS. I also run a YAML linter on my configuration files.
NGINX: I run both a config check and static analysis against my NGINX config (used for a reverse proxy to HASS, AppDaemon, etc.) – this ensures that the config (a) is correct and (b) adheres to good standards. I also run a config check against Fail2Ban, which I use to firewall off IPs that try to access NGINX in an unauthorized fashion.
Shell Scripts: I run a static analysis check (using shellcheck) on all of my Bash scripts to ensure that they conform to good standards.
A Drone run looks like this:
When everything completes, I get a Slack notification:
Drone Configuration File
Like all of the other modern CI tools, Drone uses a configuration file to define the pipelines, steps, etc. Here’s mine:
.drone.yml
---
kind: pipeline
name: AppDaemon
trigger:
branch:
- master
- feature/*
steps:
- name: Linting & Typing
image: python:3
commands:
- pip3 install --upgrade pip black flake8 mypy pylint yamllint
- black --check --fast ./appdaemon/settings/apps/
- flake8 ./appdaemon/settings/apps/
- mypy --ignore-missing-imports ./appdaemon/settings/apps/
- pylint --rcfile ./appdaemon/settings/pylintrc appdaemon/settings/apps/
- yamllint ./appdaemon/settings/
---
kind: pipeline
name: ESPHome
trigger:
branch:
- master
- feature/*
steps:
- name: "Config Check: Beta"
image: esphome/esphome:beta
pull: always
failure: ignore
commands:
- "for file in $(find /drone/src/esphome -type f -name \"*.yaml\" -not \
-name \"secrets.yaml\"); do esphome \"$file\" config; done"
- name: "Config Check: Latest"
image: esphome/esphome:latest
pull: always
commands:
- "for file in $(find /drone/src/esphome -type f -name \"*.yaml\" -not \
-name \"secrets.yaml\"); do esphome \"$file\" config; done"
- name: Linting
image: python:3
commands:
- pip3 install --upgrade pip yamllint
- yamllint ./esphome/
---
kind: pipeline
name: Home Assistant
trigger:
branch:
- master
- feature/*
steps:
- name: "Config Check: Dev"
image: homeassistant/home-assistant:dev
pull: always
failure: ignore
commands:
- cd /usr/src/homeassistant
- "python -m homeassistant -c /drone/src/home-assistant/settings \
--script check_config"
- name: "Config Check: Latest"
image: homeassistant/home-assistant:latest
pull: always
commands:
- cd /usr/src/app
- "python -m homeassistant -c /drone/src/home-assistant/settings \
--script check_config"
- name: "Config Check: RC"
image: homeassistant/home-assistant:rc
pull: always
commands:
- cd /usr/src/homeassistant
- "python -m homeassistant -c /drone/src/home-assistant/settings \
--script check_config"
- name: Linting
image: python:3
commands:
- pip3 install --upgrade pip yamllint
- yamllint ./home-assistant/settings/
---
kind: pipeline
name: NGINX
trigger:
branch:
- master
- feature/*
steps:
- name: "Config Check: NGINX"
image: alpine:3.10.1
commands:
- apk update && apk add --no-cache nginx
- ln -s /drone/src/nginx/settings/nginx /etc/nginx
- nginx -t -c /etc/nginx/nginx.conf -g 'pid /tmp/nginx.pid; daemon off;'
- name: "Config Check: Fail2Ban"
image: alpine:3.10.1
commands:
- apk update && apk add --no-cache fail2ban
- ln -s ./nginx/settings/fail2ban /etc/fail2ban
- touch /var/log/messages
- "mkdir -p /var/log/nginx && touch /var/log/nginx/access.log \
&& touch /var/log/nginx/error.log"
- fail2ban-client -t
- name: "Static Analysis: NGINX"
image: yandex/gixy:latest
pull: always
commands:
- ln -s /drone/src/nginx/settings/nginx /etc/nginx
- gixy /etc/nginx/nginx.conf
---
kind: pipeline
name: Shell Scripts
trigger:
branch:
- master
- feature/*
steps:
- name: "Static Analysis: bin/*"
image: koalaman/shellcheck-alpine:stable
pull: always
commands:
- shellcheck --version
- shellcheck ./bin/*
---
kind: pipeline
name: Wrap-up
trigger:
branch:
- master
- feature/*
status:
- failure
- success
depends_on:
- AppDaemon
- ESPHome
- Home Assistant
- NGINX
- Shell Scripts
steps:
- name: Send Notification
image: plugins/slack
settings:
webhook:
from_secret: slack_webhook
channel:
from_secret: slack_channel_name
template: >
{{#success build.status}}
`{{repo.name}}/{{build.branch}}`: Build #{{build.number}} successful
{{else}}
`{{repo.name}}/{{build.branch}}`: Build #{{build.number}} failed
{{/success}}
when:
status:
- failure
- success
Deployment
I don’t particularly want my Drone server – a Linode VPS – to be able to punch through my home network and modify the NUC that runs HASS, AppDaemon, etc. Therefore, I chose a simpler method: I run a regular Bash script on the NUC that checks for newly-successful Drone jobs and, upon finding one, deploys everything via docker-compose.
bin/build (the script that builds the entire stack)
#!/bin/bash
set -euxo pipefail
REPO_PATH="$( dirname "$( cd "$(dirname "$0")" ; pwd -P )" )"
function copy_config_data() {
volume_name=$1
settings_path=$2
docker volume create "$volume_name"
docker run --rm \
-v "$settings_path":/src \
-v "$volume_name":/data \
-w /src \
busybox \
cp -a . /data
}
# Update AppDaemon configs:
copy_config_data "hub_appdaemon-config" "$REPO_PATH"/appdaemon/settings
# Update Glances configs:
copy_config_data "hub_glances-config" "$REPO_PATH"/glances/settings
# Update Home Assistant configs:
copy_config_data "hub_hass-config" "$REPO_PATH"/home-assistant/settings
# Re-build the containers if necessary:
docker-compose -f "$REPO_PATH"/docker-compose.yml build
# Tear the stack down and clean up loose containers and images:
docker-compose -f "$REPO_PATH"/docker-compose.yml down --remove-orphans
docker container prune -f
docker image prune -f
# Bring up all "background" services (i.e., those services that can come up before the
# big three: HASS, AppDaemon, and NGINX):
docker-compose -f "$REPO_PATH"/docker-compose.yml up -d \
--scale appdaemon=0 \
--scale hass=0 \
--scale nginx=0
# Bring up HASS, AppDaemon, and NGINX with delays in-between:
sleep 5
docker-compose -f "$REPO_PATH"/docker-compose.yml up -d hass
sleep 5
docker-compose -f "$REPO_PATH"/docker-compose.yml up -d appdaemon
sleep 5
docker-compose -f "$REPO_PATH"/docker-compose.yml up -d nginx
# Do a final cleanup of unused networks and volumes:
docker network prune -f
docker volume prune -f
bin/deploy (the script that runs at regular intervals and, upon finding a successful Drone job, calls bin/build)
#!/bin/bash
set -euxo pipefail
# Variables to edit when copying this deployment script to other repos:
BRANCH="master"
REPO_NAME="hub"
# Variables that shouldn't need editing:
API_BASE="https://my-drone-server"
API_LAST_BUILD="api/repos/bachya/$REPO_NAME/builds"
API_TOKEN="token123"
API_USER_AGENT="User-Agent: Paw/3.1.8 (Macintosh; OS X/10.14.6) GCDHTTPRequest"
SLACK_HOOK_URL="https://hooks.slack.com/services/..."
COLOR_BLUE="#1539e0"
COLOR_GREEN="good"
COLOR_RED="danger"
COLOR_YELLOW="warning"
LAST_BUILD_FILEPATH="/tmp/${REPO_NAME}_last_drone_build"
BIN_PATH="$( dirname "$( cd "$(dirname "$0")" ; pwd -P )" )"
function fetch_latest_build_number() {
curl \
-s "$API_BASE/$API_LAST_BUILD" \
-H "$API_USER_AGENT" \
-H "Authorization: Bearer $API_TOKEN" \
| jq 'map(select(.status == "success" and .target == "'$BRANCH'"))[0].number'
}
function remote_git() {
git --git-dir "$BIN_PATH"/.git --work-tree="$BIN_PATH" "$@"
}
function send_slack_message() {
local color="$2"
local text="$1"
local message
message="\`$REPO_NAME/$BRANCH\`: $text"
curl \
-X "POST" \
-H 'Content-Type: application/json' \
-d $'{ "attachments": [ { "color": "'"$color"'", "text": "'"$message"'" } ] }' \
"$SLACK_HOOK_URL"
}
function should_update() {
local latest_build_number
local saved_build_number
# If the last build file doesn't exist, create it:
[ ! -f "$LAST_BUILD_FILEPATH" ] && touch "$LAST_BUILD_FILEPATH"
latest_build_number=$(fetch_latest_build_number)
saved_build_number=$(cat "$LAST_BUILD_FILEPATH")
# If the build number is the same as the last one retrieved, don't trigger
# a build:
if [ "$latest_build_number" == "$saved_build_number" ]; then
echo "false"
return
fi
echo "$latest_build_number" > "$LAST_BUILD_FILEPATH"
echo "true"
}
function update() {
local git_pull_result
local local_branch
local saved_build_number
# If the repo has a branch other than the target one checked out, don't trigger
# a build:
local_branch="$(remote_git rev-parse --abbrev-ref HEAD)"
if [ "$local_branch" != "$BRANCH" ]; then
send_slack_message \
"Refusing to deploy to local branch: $local_branch" \
"$COLOR_YELLOW"
echo "false"
return
fi
git_pull_result="$(remote_git pull origin master > /dev/null 2>&1)"
if [ "$git_pull_result" == "1" ]; then
send_slack_message \
"Deployment failed due to failed Git pull on target host" \
"$COLOR_RED"
return
fi
saved_build_number=$(cat "$LAST_BUILD_FILEPATH")
send_slack_message \
"Deployment for build #$saved_build_number starting..." \
"$COLOR_BLUE"
"$BIN_PATH"/bin/build
send_slack_message \
"Deployment for build #$saved_build_number completed" \
"$COLOR_GREEN"
}
dirty=$(should_update)
if [ "$dirty" == "true" ]; then
update > /dev/null 2>&1
echo -n "Deployment complete"
else
echo -n "No deployment required"
fi
The Whole Thing
For reference and interest, my entire repo can be found here: https://github.com/bachya/smart-home