Out of frustration in figuring out how to write tests for custom components, I have created a, hopefully, easy to use library. It aims to enable all the fixtures and helpers that exist in homeassistant/core. It is semi-automatically updated from the latest dev branch in homeassistant/core, with the goal of being fully automatic in the future.
I hope others may find it useful, and that it leads to less frustrating dev experience for custom component developers.
For all those HACS components, you will need to add an __init__.py file in your custom_components folder.
Very nice, I had created something similar (https://github.com/boralyl/pytest-homeassistant) by pulling out fixtures, but it’s not automatically updated at all and has broken before w/ new releases. I’ll have to dig into how you went about this I’m not terribly happy with my solution.
To be very clear, I did see your package, and considered contributing there, but I couldn’t tell how you were deciding to choose which pieces to incorporate. I’d be very willing to accept contributions if you see any improvements that could be made.
Everything ‘happens’ in the generate_package.py file and the GitHub actions.
fahr’s blog post gives a decent overview. You pip install pytest-homeassistant-custom-component or pip install pytest-homeassistant, if you want to use fahr’s version. I would suggest doing this in a virtual environment.
Then you write your tests just like in the home-assistant/core repository in a tests. Then run the tests with pytest. There is an example custom component and tests in my repository as well.
I took a quick look through the repository you linked. I would probably start by mocking the update_method of the EmailData class and using MockConfigEntry to set up your integration to bypass the config flow logic. Then you would have a test for only the workings of the sensor entity. Then you could mock out the actual IO and test the logic of the EmailData class only.
You could also start from the other end by mocking out any IO in the config flow and test that your config flow works the way it should form by form.
Honestly, I just pulled out the fixtures that I ended up using in my custom components, and ignored the rest. I really like your implementation since it’s automated and probably less likely to run into issues with upgrades.
I suggest using the patch function in your test. You’ll need to put an __init__.py in the custom_components folder too. This a rabbit hole to fall in, but you can then make changes to the custom component more confidently.
Something like this:
async def test_some_stuff(hass):
with patch("custom_components.myintegration.sensor.EmailData.update") as mock_update:
# do some stuff with a mocked method
ah ok that makes sense, I’m actually migrating all the functions and such out of the sensor.py into __init__.py and keeping only the sensor bits in there.
Sorry to be a bother, I have this most of the way done, but I’m getting stuck and would like to pick your brain if you’re game.
I am getting the following error (during pytest):
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/homeassistant/helpers/entity_platform.py", line 193, in _async_setup_platform
await asyncio.shield(task)
File "/home/runner/work/Home-Assistant-Mail-And-Packages/Home-Assistant-Mail-And-Packages/custom_components/mail_and_packages/sensor.py", line 70, in async_setup_entry
data = EmailData(hass, config)
File "/home/runner/work/Home-Assistant-Mail-And-Packages/Home-Assistant-Mail-And-Packages/custom_components/mail_and_packages/sensor.py", line 101, in __init__
self.update = Throttle(self._scan_interval)(self.update)
File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/site-packages/homeassistant/util/__init__.py", line 206, in __call__
and "." not in method.__qualname__.split(".<locals>.")[-1]
File "/opt/hostedtoolcache/Python/3.7.9/x64/lib/python3.7/unittest/mock.py", line 601, in __getattr__
raise AttributeError(name)
AttributeError: __qualname__
Seems it doesn’t like the self.update's Throttle?
(I am switching this out to use the update coordinator in a yet to be released version.)
It looks like you are trying to dynamically update the fetch interval?
If you are using DataUpdateCoordinator, there is no need to use Throttle explicitly. The coordinator class has a built in debouncer. If the default debouncer time doesn’t work for you, you can adjust it. The coordinator also makes it easy to dynamically update the fetch interval.
A note to any developers using this package. I fixed a bug in my GitHub actions that prevented the package to be uploaded to PyPI. The package was 1-2 versions behind homeassisant on PyPI because of this, but it is now fully up to date.
I have updated this library to handle the transition to fully drop Python 3.7. This has occurred in version 0.1.0. Additionally, the dependencies that were not directly related to pytest testing (the obvious ones anyway) were moved to a separate requirements_dev.txt. This enables developers to pin their own versions of those libraries, such as pylint.
With recent changes in homeassistant causing some issues with the use of this package, I’d like to solicit feedback on path(s) forward. Right now, one of the main pain points is linking a version of homeassistant in testing to the version of pytest-homeassistant-custom-component (phcc for short). phcc currently does not pin a homeassistant version in its dependencies, which will cause users of version 0.2.0 of phcc to get errors unless they pin homeassistant to versions prior to 2021.3.0.
Pin the homeassistant requirement to currently dev or stable version of homeassistant at the time I generate the package. The downside to this is that it will break sometimes like recently when major changes were made to the testing that required the coexisting infrastructure in homeassistant package.
Generate phcc package only for stable or dev releases. The upside is that components could test against specific home assistant versions. The downside is increased maintainability requirements to make sure they are in sync (this is a big downside to me), and not being able to proactively test against upcoming changes in dev.
Pin the homeassistant requirement to a GitHub commit ref. This one is basically drop in to the current workflow and would install a version of homeassistant according to the exact commit used to extract phcc. This seems the best path forward, but I’m unaware of any potential pitfalls of such an approach. Hence my solicitation of feedback before implementing.
I went with option 2, so that it is now generated based on the latest release by homeassistant/core, including beta releases. The latest version was generated based off of 2021.4.0b4.
The downside is that we won’t be able to test based on bleeding edge changes, if needed (this didn’t really work in the previous state either). My plan is to introduce a cli to allow developers to reference a local homeassistant/core repo and generate off an arbitrary commit ref. This allows for the ability to get cutting edge testing if needed.
