Hi,
did you found a some good way? I’m just starting to look at it.
normally there are 2 ways to test.
- if you run AD on the commandline you can use timetravel
- just let an app run and use enough logging to make sure it does what you want.
The kind of tests I (and I believe also @jedi7) was talking about are automated ones that can be executed every time we need to (ex: when there are breaking changes or a new version is released).
If the automated tests need to look at the logs or do a time travel that would be part of their code.
In the end I believe we should be looking for a python testing framework, probably mock real components (this can be hard to do), instantiate the AppDaemon app with those and then use the tests to assert the things we want.
Thoughts / Experiences ?
Thanks,
Claudio.
I work with AD for a long time now, and i cant think of a use for automated testing.
if an app is written, debugged and running it just keeps working.
i cant see why i would stop the working environment to start a test if it is still working.
if i update HA then there could be breaking changes, you are right. but all that can result in is that some entities are not available for AD.
ill notice that as soon as i start AD, because i get warnings. so i dont need a test for that.
AD rarely has breaking changes at all, but if there are any they are well documented.
but if you really want it, i guess it would be possible to start HA with a clean setup, then use an app in AD to setup a test environment with sensors and switches.
but you need to update your test environment every time you change something in HA or AD.
and you still need to run the tests in time travel, or you wouldnt be able to see what is happening.
as i see it, it would be an extreme amount of work to create such a thing and an extreme amount of work to keep it up to date. and the advantage from it i still dont see 
Yes, I’m talking about unit tests. For example if I have a complex automation, then I need to test it by this way. For example simulate all occasions what can happen. It also helps with developing of the automation.
I will try unit tests in few days, so I will notice you how far I’m 
Excellent @jedi7 ! As I am learning all the HA environment and installing z-wave components at my own home, didn’t have more time to dig into unit test automation, but I am very interested in it.
My case is that if everything goes as expected at my own home, I would apply these (and other) automations in the future at customers homes so I need to have a way to verify the scripts I deploy there in an automated way. Also I agree unit testing helps with the development of the script itself (TDD).
@ReneTode Not sure why is that you don’t see advantage of automated testing ? I believe unit tests in any kind of software are always helpful, they do take time but pay for themselves as time goes by. Indeed HA itself has unit testing in their device components.
@jedi7 Please let me know your findings
Thanks,
Claudio.
Ok, I have a progress here.
I’m testing my object Presence.
And it seems like the mock is working
dir structure:
.homeassistant/apps/presence.py
.homeassistant/tests/test_presence.py
from apps.presence import Presence
def mock_hass(func):
def func_wrapper(self):
patcher = mock.patch.object(Presence, '__bases__', (mock.Mock,))
with patcher:
patcher.is_local = True
presence = Presence()
return func(self, presence)
return func_wrapper
....
@mock_hass
def test_start(self, presence):
"""Test the start."""
presence.args = {'ename':'TestName',
'google_maps_tracker':'gmaps_tracker',
}
presence.initialize()
print("name: %s" % presence.ename)
2 Likes
Great!
Thing is, I am new to python (I do have experience in testing but in other languages), so sorry to ask but:
Which Unit Testing framework are you using and how are you executing the tests ?
Can you post the complete code of test_presence.py ?
thanks!
Claudio
I’m using unittest. The file is now quite big for full posting.
Most of these things you can google
import unittest
class TestPresence(unittest.TestCase):
run:
python -m unittest discover
it might be the way i work?
i really have no idea what i should do with an automated test?
that might be helpfull if i write big complex apps and implement them only when i am done writing.
but i do write small automations and implement them immediatly.
how would you like to automaticly test something like:
turn on the lights at sunset
i wait for sunset or use time travel and see if it works.
same with turn on light when movement
turn of light when to bed
or put heating on when temp is below 20 degrees.
i just dont see how to automate test such stuff.
As @jedi7 is doing, in your case by mocking lights heating and temperature components, passing those to the app daemon scripts and then execute the corresponding asserts in the tests.
That way we can verify that the scripts logic work correctly and have the tests executed automatically whenever the need arise
The cases you mentioned are simple, but as @jedi7 said there could be much complex logic and that is where tests automation adds value.
But I respect your view and I agree not every script needs to be automated.
1 Like
I am interested in automated tests as well… my motion light implementation is getting quite complex and I’m tired of running around the house to test all the different configs before pushing new releases to Github hahaha
break things down to the smallest pieces instead of creating 1 large part.
if you got working pieces then you dont even have to test if it is working.
the only thing that break my parts are typos, but that can happen in automated tests as well
i did put it in my favourites.
in most of my cases it would be to much trouble, but it might be helpfull at some point.
Can you give an example?
i break everything down to little rules. or little events just how you call it.
as small as possible.
like:
if < it is some time> then but not when
and thats an app actually. which can be reused.
i give the events a name that explains everything.
so my app yaml is looking like:
heating_livingroom_evening:
if: {"time": "18:00"}
then: ("input_number.heating_livingroom": 22}
not_when: {"sensor.rene": "away"}
heating_livingroom_kitchen:
if: {"time": "18:00"}
then: ("input_number.heating_kitchen": 20}
not_when: {"sensor.rene": "away"}
i dont try to get everything in 1 big automation.
for example 1 motion detector can trigger 7 or 8 different apps.
motion_kitchen_day:
motion_kitchen_evening:
motion_kitchen_night:
motion_kitchen_everyone_away:
motion_kitchen_rene_alone:
motion_kitchen_olinde_alone:
motion_kitchen_after_motion_hallway:
etc.
off course i started out with motion_kitchen at first.
and then i realised: hmm, i want to do different for evening and night
so i split it up, the i add event after event, after event.
I’m way beyond “if then not when…”. Its not one big automation its one automation for a specific purpose that is reused for 8 different motion sensors.
2 Likes
thats what i tried at first too.
untill i found out that the bigger and more complicate the app was getting, the worse it was to add new situations.