Since my home assistant config is growing on a daily basis, I wonder how you guys have setup your testing environment. Currently, my travis check is the only health check done before I deploy my config to my production system. Is there any way of having additional tests to also cover the functional behavior of home assistant? I wish I could have some unit tests to describe my desired behavior.
Furthermore, the travis check sometimes succeeds, even though the config is broken (e.g. see this build: https://travis-ci.org/TribuneX/home_assistant/builds/307488137). Is there a way to prevent this?
But I am also interested in your general testing environment.
I run ha as a virtual machine under VMware ESXi. In a few minutes, I can quickly clone my virtual machine, shut down the original, and power up the clone.I can do major changes to the clone to test and then power off and delete when I am done. Another thing I do regularly is to take a snapshot just before an update. If the update is successful, I delete the snapshot to consolidate the disk. If it is not successful, I can revert to the snapshot.which removes the changes.
Thanks for your input. I am looking into Test Driven Development recently and thought about having comprehensive tests covering my automations. Sort of like writing would Inexpect an automation to do and then test for the action given the input of a trigger. This would be especially helpful to see if a certain input also triggers other automationa, which I did not intend.
The config check only allows for testing if my syntax is correct, but does not allow for functional testing. Any ideas how to achieve something like this?
If you are wanting to do functional tests without actually connecting to the devices I think you would need to develop a testing environment that mimicked the devices. Since there are hundreds of components and potentially hundreds of devices that are controllable by each those components, it would be a big job to create. You might want to post in the developer chat area to get some feedback. https://discord.gg/c5DvZ4e