Development Tips and Tricks

[Limych (Andrey "Limych" Khrolenok)]

I have a small hobby: in my free time I create my own smart home system based on Home Assistant.
And, like any program, it gradually becomes more and more. And it is becoming more and more difficult to keep in mind the internal interrelations of its various components. Therefore, testing comes to the rescue through a system of continuous integration. I use Travis CI.

I like this system, but it has one feature — it uses the data publicly posted on GitHub. And to test the Home Assistant configuration, you need a secrets.yaml file, which contains various parameters that are not allowed for open publishing.

How to be?…

For myself, I solved this problem in the style of a “lazy programmer” — I wrote a script that can create a fake secrets.yaml file based on the original one. It saves all the same fields, but fills them with random data.

The idea turned out to be very useful…

You can always find the code for this and other tricks in the repository of my config for HA:

Limych/HomeAssistantConfiguration

[nickrout]

Can’t you just exclude secrets.yaml from git?

Also, I saw an article recently on setting up your own git server at home. Would a local instance work better for this?

[Limych (Andrey "Limych" Khrolenok)]

@nickrout

Can’t you just exclude secrets.yaml from git?

Yes, you can exclude secrets.yaml. I say more: you MUST exclude secrets.yaml from git.
But above I write about continuous testing configs for errors. And for that purpose you need to have some secrets.yaml file. You have to be an idiot to share the original file in open access. That’s why I created a fake secrets.yaml generator.

@nickrout

Also, I saw an article recently on setting up your own git server at home. Would a local instance work better for this?

Local instance may be convenient for some needs, but Travis CI provides a free service only for projects publicly hosted on GitHub.

[Limych (Andrey "Limych" Khrolenok)]

Here is example of fake secrets.yaml from my repository:

Limych/HomeAssistantConfiguration/blob/master/tests/fake_secrets.yaml

[danielperna84]

@nickrout

Can’t you just exclude secrets.yaml from git?

I think the point is, that when you exclude your real secrets.yaml, but then put a fake secrets.yaml in your repo, you have a conflict. Hance creating a fake one on the fly during CI can be useful.

[nickrout]

@Limych

But above I write about continuous testing configs for errors. And for that purpose you need to have some secrets.yaml file. You have to be an idiot to share the original file in open access. That’s why I created a fake secrets.yaml generator.

nickrout:

I understand your use case better now and completely see your point. Good stuff.

[Limych (Andrey "Limych" Khrolenok)]

@danielperna84

I think the point is, that when you exclude your real secrets.yaml, but then put a fake secrets.yaml in your repo, you have a conflict. Hance creating a fake one on the fly during CI can be useful.

No, there is no conflict there. Fake secrets.yaml are copied to the original location only in Travis CI environment just before testing begins (my Travis CI config:
https://github.com/Limych/HomeAssistantConfiguration/blob/master/.travis.yml

But to create a fake secrets.yaml at the time of testing is impossible, because we can no longer find out at this moment what the field names were. It is possible to create fake secrets.yaml only having the original secrets.yaml before your eyes.

[Limych (Andrey "Limych" Khrolenok)]

@danielperna84

I think the point is, that when you exclude your real secrets.yaml, but then put a fake secrets.yaml in your repo, you have a conflict. Hance creating a fake one on the fly during CI can be useful.

The conflict will be if you put fake secrets.yaml exactly in place of the original in the repository. Then when updating configs from the git, you destroy the original secrets.yaml.

That is why fake secrets.yaml are stored in a separate file and are copied to the place of original secrets.yaml only in a temporary environment at the very last moment before testing.

[paddy0174 (Patrick)]

Administering a git server is a lot of work. I’d simply use Gitlab instead of Github. There you can setup as many repos as you like, and you can have every repo either hidden, for members only or public. Saves the hassle in administering.

But since this is also an online service, this changes nothing about the secrets.yaml.

@Limych Thanks, that comes in handy, I’m just setting up my gitlab repo. Including testing

[ChaoticUnreal]

So I’m not sure where to post this but I’ve run into an issue with my Travis integration recently.

For the last 5 days I’ve been getting the following error for all my builds

Failed config
General Errors:
- Unable to install all requirements: homeassistant-pyozw==0.1.4, pydispatcher==2.0.5

This had been working fine till 0.93 was released. My actual install is working fine.

This is my .travis.yml

language: python
python:
  - "3.5"
before_install:
  - mv not_secrets.yaml config/secrets.yaml
install:
  - pip3 install homeassistant
script:
  - hass -c ./config/ --script check_config

[Limych (Andrey "Limych" Khrolenok)]

Try to use python 3.6

[Codec303]

That’s a nice way around the problem.

I use BitBucket for my HA configs, is it possible to use Travis CI with BitBucket instead?

[ChaoticUnreal]

I get the same error when I try that. (I had tried it prior then reverted it to 3.5 when it didn’t fix it.)

not sure if its helpful but more error logs

INFO:homeassistant.util.package:Attempting install of homeassistant-pyozw==0.1.4
ERROR:homeassistant.util.package:Unable to install package homeassistant-pyozw==0.1.4: 
Command "/home/travis/virtualenv/python3.6.3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-2j9cm3te/homeassistant-pyozw/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-0qaphv_2-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/travis/virtualenv/python3.6.3/include/site/python3.6/homeassistant-pyozw" failed with error code 1 in /tmp/pip-build-2j9cm3te/homeassistant-pyozw/
ERROR:homeassistant.requirements:Not initializing zwave because could not install requirement homeassistant-pyozw==0.1.4

[Limych (Andrey "Limych" Khrolenok)]

You have a problem not with Home Assistant, but specifically with ZWave. Here I can not help you, because I do not use this system and have no experience of using it. Sorry.

[Limych (Andrey "Limych" Khrolenok)]

Let’s continue …

I like it when the background music in the house. There is a media server that can distribute music to the speakers. But I like listening to the radio more.

In Home Assistant there is no regular convenient way to maintain a list of radio stations. Enthusiasts have come up with a way to get around this boundary in a couple of automation scripts (Combined music player). But editing them in the usual way is difficult - you have to dig into long lists of URLs and text strings. High probability of error.

Going along the path of a “lazy programmer”, I wrote a script that makes changes to the automation scripts for me:

Limych/HomeAssistantConfiguration/blob/master/bin/update_radio_stations.sh

For its work, I just need to make a list of radio stations, similar to the usual YAML:

Limych/HomeAssistantConfiguration/blob/master/radio_stations.yaml

[Limych (Andrey "Limych" Khrolenok)]

Perhaps, a solution to your problem will be written here soon: Travis CI build issues

[ChaoticUnreal]

Thanks I just read that thread and it looks like it should solve the issue I was having.

[BrianHanifin (Brian)]

Thanks to you I have added a button to my UI which regenerates my fake_secrets.yaml file and rebuilds my README.md. This is amazing!

I’d love to automate it with local CI, but I’m not quite ready to figure that out yet. Thank you.