How to: Use CircleCI to check your config on GitHub

Premise

Many of us like to put our configuration on GitHub. It’s a good way to back up non-sensitive configuration files and also to share your configuration with others.

Anyone wanting to put their config on GitHub who hasn’t already will probably start here…

At the time of me writing this post that page is flagged to be updated, but it’s still not a bad starting point. I’ve also dropped some other hints around the forums about more advanced ways to manage your GitHub synchronisations from your homeassistant with SSH keys and scripts, and will happily do a full guide for all of that to if people want one. But for now, let’s assume that you’re reading this because you already have some kind of GitHub sync set up.

Anyone following that guide will see that step 7 recommends using Travis CI to validate your configuration. I used this myself until the last few days when I got stuck in a loop with their service, so I started looking elsewhere.

Why use a CI?

If you don’t know what a CI is, in very simple terms it monitors your GitHub repo and then runs a script to check any changes are OK, and issues a ‘pass’ or ‘fail’ appropriately. In the case of homeassistant you would set it up to run the check_config script.

The ‘clever’ bit being that you can setup your GitHub to only allow code that passes this test to actually get in to your master branch.

Why do it on GitHub when you can do it manually on your machine? Several reasons, but here’s two good ones:

  • You can edit your configuration on GitHub whilst you’re away from home without having to have remote access to your file system, and then pull the changes down later.

  • when you test on your local machine it checks your configuration against the version you’re using now, you can test your configuration against the latest version of homeassistant with this check to make sure you’re not affected by breaking changes before you upgrade.

So anyway, I settled on CircleCI and here’s how to do it…

Get everything ready

Firstly, if you have any scripts that push your configuration to GitHub on the master branch you need to edit them to push on a different one so that you can ‘protect’ your master. For example if you were using the script from the link I posted above you would edit it to

#!/bin/bash

cd /home/homeassistant/.homeassistant

git add .
git checkout -b upload
echo -n "Enter the Description for the Change: " [Minor Update]
read CHANGE_MSG
git commit -m "${CHANGE_MSG}"
git push origin upload

exit

This pushes your configuration to a branch on GitHub called upload instead of master.

Go to your GitHub account and click on Marketplace, find the CircleCI app and authorise it, it’s free :slight_smile:

Select a Linux plan.

In your homeassistant repo (from a desktop view of the website) across the top are code, issues… and the last one is settings, click that. From the side bar select Integrations & Services and then click the Configure button next to the CircleCI app. Either choose ‘all repositories’ or select your homeassistant repository and click save.

Go to your master branch and create a new file called .circleci/config.yml and put this in it…


version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.6.4
    working_directory: ~/repo
    steps:
      - checkout
      - run:
          name: Check config
          command: |
            touch fake.pem
            mv secrets_redacted.yaml secrets.yaml
            python3.6 -m venv .
            source bin/activate
            pip3 install --pre --upgrade homeassistant
            hass -c . --script check_config --info all

Note -

  • set the python version that you want (match the one you’re using)

  • the line touch fake.pem creates a blank pem file so that there is one there when the config checker looks for it if you’re using SSL. This needs to be referenced in your fake secrets file instead of the path to the real file you have on your machine. -remove this line if you don’t need it.

  • the line mv secrets_redacted.yaml secrets.yaml means that the script will use your redacted secrets file to validate all the !secret entries in your configuration, because your real secrets.yaml wouldn’t be very secret if you upload it to GitHub! - the test will fail if you don’t have a secret to correspond to a !secret entry in your configuration.

Commit the file.

Run your first test

Now go to https://CircleCI.com and go to the projects on your dashboard. Click the cog next to your homeassistant repo and click Start Building. Watch the build, and make sure everything is OK before you continue.

Protect the master branch

Go back to the settings page on your GitHub repo and click Branches > Add Rule

Apply the rule to the master branch, and select Require status checks to pass before merging and tick the Workflow test. Do not tick any others that might be there, even though they are called circleci! Tick Include Administrators and save the changes.

Now go to https://CircleCI.com and go to the projects on your dashboard. Click the cog next to your homeassistant repo and select advanced options. Turn off GitHub status updates, and turn on Only build pull requests.

Pull the changes

This bit is pretty important. You need to pull the changes you made to your GitHub repo down to your device, either with git pull or git fetch and reset.

You should now have an identical copy of your tracked files from your repo on your device. Check your device has the .circleci/config.yml file.

Push a change

Change your configuration a bit and run your push script then head over to GitHub. You should see that you now have two branches. Master and Upload. Select the upload branch and start a pull request.

Watch the PR conversation and you will see that you cannot merge it until the check is passed. If it fails, visit your CircleCI dashboard to view the log and find out why. Fix the error and commit it and the check will be rerun.

Once the check is passed, click Squash & Merge and your changes are merged in to master. Delete the upload branch ready for a new one to be created on your next push.

Automating it all

As I said above I can do a full guide if there is demand, but for now, have a look at the bash_scripts and maintenance package on my GitHub to see how mine is automated for pushing and pulling my config.

link removed

5 Likes

@anon43302295 when i try click on Integrations & Services , i have this message:
image

any input?

Have you installed the circleci app from github market place? After that it should appear in the top bit of the page you screenshot.

@anon43302295 yes i installed it:
image

image
what the next step?

Hmmm. Good question :slightly_smiling_face:

Possibly you will have to click on that ‘configure access’ link and link it to the repository?

many problems ;(
image

Circleci file should be…

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.6.4
    working_directory: ~/repo
    steps:
      - checkout
      - run:
          name: Check config
          command: |
            touch fake.pem
            mv secrets_redacted.yaml secrets.yaml
            python3.6 -m venv .
            source bin/activate
            pip3 install --pre --upgrade homeassistant
            hass -c . --script check_config --info all

yes yes, i know… but here
image
if i select LINUX and click START BUILDING…
shows me this:

image

Bizarre. Link to your repo?

From a quick Google it looks like maybe in your circleci settings you may have accidentally ticked a box for macos builds that needs to be unchecked.

@anon43302295 I’d be interested in the guide if you can find the time, would like to stop doing it manually.

Yeah, I’ll knock something up :slight_smile:

1 Like

@anon43302295 my repo is:
https://github.com/htcheroportugal/home-assistant-config

and circleci settings i have this:

@anon43302295
Sorry for reviving an old post, I counter some issues. I want to create some folders and files, but I get constantly get permission errors.

I want to add this, but can you tell how please?

        mkdir -p /opt/scripts/
        echo "10" > /opt/scripts/cert.log
        echo "10" > /opt/scripts/cert_eu.log

Current config:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.6.4
    working_directory: ~/repo
    steps:
      - checkout
      - run:
          name: Check config
          command: |
            mv secrets_redacted.yaml secrets.yaml
            
            python3.6 -m venv .
            source bin/activate
            pip3 install --pre --upgrade homeassistant
            hass -c . --script check_config --info all

I doubt you’ll be to create an /opt folder in the CI, why do you need those files?

I have a command line sensor , I store those scripts files in that location.

What is the failure message for the CI if you don’t have those files?

mkdir: cannot create directory ‘/opt/scripts/’: Permission denied
I get a long list of error, pages long

No, I’m asking what the error is if you don’t create those folders?