I’m trying to run the build in benchmark script on HassIO as detailed here:
Has anyone managed to do it? If so how?
yep
(homeassistant) homeassistant@nuccy:/home/nick$ hass --script benchmark async_million_events
Using event loop: asyncio.unix_events
Benchmark async_million_events done in 0.7540765689918771s
Benchmark async_million_events done in 0.7610460779978894s
Although I am not using hassio.
Try this:
#Open HA container shell
docker container exec -it homeassistant bash
#Run Benchmark
hass --script benchmark async_million_events
Using event loop: uvloop
Benchmark async_million_events done in 6.156998317019315s
Benchmark async_million_events done in 6.122647846990731s
I’m using HassIO on a RPi3 so that would be why I’m almost ten times slower than @nickrout’s nuc 
However I’m not sure if it’s working 100% because the event loop is different.
2 Likes
That worked after disabling protection mode. Hassio on a Chinese mini PC:
Thanks @HypnoToad those commands seemed to have worked for me.
Right now I’m testing 2 x RPi 3’s, one running HassIO and the other running Hass with VENV. Both Pi’s are using the same SD card type, and official PSU, the only difference here is software.
Believe it or not, the HassIO benchmarks are quite a bit better despite Docker being used.
This is what I get with VENV:
Using event loop: asyncio.unix_events
Benchmark async_million_events done in 9.03224296999997s
And this is what I get with HassIO:
Using event loop: uvloop
Benchmark async_million_events done in 6.146387876000954s
It seems that using uvloop may give a 50% boost in network performance in this benchmark at least. It’s a shame there aren’t more benchmarks really, I’d quite like to try a file IO comparison.
1 Like
Does anyone know how to run the benchmark test when Home Assistant runs in a Docker container (not hass.io). It doesn’t appear to contain the “hass” command.
I’m using Portainer, I opened a console window in the Home Assistant container, ran the following command:
find / -name "hass"
and it came back empty-handed. Without it, I don’t see how it’s possible to run the benchmark test …
I guess you could download the script and run it locally?
I thought hass was the home assistant run file.
I think the script is present in the Docker container:
Me too which is why it’s perplexing that hass can’t be found in the container.
I peeked into /etc/systemd/system, hoping to find some clue of how Home Assistant is started, but found nothing related to it. 
What comes up if you do
find / | grep "/hass"
In the docker container?
It returns more results than fit in the console’s buffer … because there are several directories that begin with /hass, each containing many files.
Added a | more to the command but there are still too many results to step through …
Hmm, Ok actually if nothing came up when you did
find / -name "hass"
then there probably isn’t a file called ‘hass’ anywhere, very odd.
Just to be sure, when HA us running, try
ps -A | grep -i hass
and see if you get any output
This isn’t the result I expected:
root@plutonium:/usr/src/app# ps -A
PID TTY TIME CMD
1 ? 00:00:11 python
58 pts/0 00:00:00 bash
73 pts/0 00:00:00 ps
root@plutonium:/usr/src/app#
Perhaps my docker-fu is weak but I did expect to see a few more processes running within the container …
I’m not that experienced either I’m afraid, give ps -aux a try, it may show more info. pstree is nice also, if it’s installed that is.
pstree isn’t installed in the container but ps -aux reveals a little bit more information:
root@plutonium:/usr/src/app# ps -aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 5.1 2.4 1324768 96268 ? Ssl 08:11 0:11 python -m homeassistant --config /config
root 57 0.2 0.0 5752 3604 pts/0 Ss 08:11 0:00 bash
root 64 0.0 0.0 9392 2988 pts/0 R+ 08:14 0:00 ps -aux
It would appear that Home Assistant is initiated without the use of hass.
python -m homeassistant --config /config
Maybe
python -m homeassistant --script benchmark async_million_eventsPS my hass script, on my venv system is pretty short
#!/srv/homeassistant/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from homeassistant.__main__ import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
The ‘-m’ is telling python to run the homeassistant library as a script.
you may just be able to run the benchmark script directly with python, something like
python /path/to/benchmark/__init__.py async_million_events
may work
Some progress but not the desired result just yet:
root@plutonium:/usr/src/app# python /usr/src/app/homeassistant/scripts/benchmark/__init__.py async_million_events
Traceback (most recent call last):
File "/usr/src/app/homeassistant/scripts/benchmark/__init__.py", line 10, in <module>
from homeassistant import core
ModuleNotFoundError: No module named 'homeassistant'
The tenth line in the benchmark script attempts to get a module from somewhere it can’t find.
I’m guessing the hass script sets up the correct environment so that when the benchmark script is called it finds all required modules.
My production system is running Home Assistant in a python virtual environment. I’ll take a peak at its copy of hass and see what it does.
HA 2021.1.2
Supervised on Ubuntu 18.04.5
bash-5.0# hass --script benchmark async_million_events
I’v got:
usage: hass [-h] [--script {benchmark}]
{fire_events,time_changed_helper,state_changed_helper,state_changed_event_helper,logbook_filtering_state,logbook_filtering_attributes,_logbook_filtering,filtering_entity_id,valid_entity_id,json_serialize_states}
hass: error: argument name: invalid choice: 'async_million_events' (choose from 'fire_events', 'time_changed_helper', 'state_changed_helper', 'state_changed_event_helper', 'logbook_filtering_state', 'logbook_filtering_attributes', '_logbook_filtering', 'filtering_entity_id', 'valid_entity_id', 'json_serialize_states')