I want to do something very simple: run a simple python script when HA starts. The script is in its own folder in HA’s /config directory. The script collects modbus data and logs it to a file.
Despite extensive reading, I haven’t found a way to do this. Things like writing addons seems huge overkill when all I want to do is:
python /myfolder/myscript.py
Any advice you can give me on how to do this?
Thanks
Have you looked at pyscript?
I have, but I can’t see how it can do this at all, let alone simply. Python_scripts is no good, as I need to import a couple of packages, and it prohibits imports.
I had a similar need but was never able to get anything working natively within HA. I have a series of web scrapers that use Selenium that publish data to MQTT. I ended up just hosting these on my network (off of a secondary PI that I’m using for other purposes)
It does seem remarkable that something so basic should need such complex solutions. All the script does is poll modbus data over a wired connection (so web sc won’t get it) and write it to a csv file.
I just moved my system from Core to HAOS and needed to convert about a dozen scripts. Maybe this small example will help
Here’s how I did it:
@pyscript_executor
def mainline():
import datetime
import time
import paho.mqtt.publish as publish
# Determine next GoodMorning
suffix = [ ' ', 'st', 'nd', 'rd', 'th', 'th', 'th', 'th', 'th', 'th', 'th',
'th', 'th', 'th', 'th', 'th', 'th', 'th', 'th', 'th', 'th',
'st', 'nd', 'rd', 'th', 'th', 'th', 'th', 'th', 'th', 'th', 'st' ]
date = datetime.datetime.now()
day = int(date.strftime('%-d'))
msg = '<b> ' + date.strftime('%m/%d/%Y') + ' - '
desc1 = 'Good Morning, Today is ' + date.strftime('%A') + ', ' + date.strftime('%B') + ' ' + date.strftime('%-d') + suffix[day] + ' </b>'
desc2 = 'Good Morning, Today is ' + date.strftime('%A') + ', ' + date.strftime('%B') + ' ' + date.strftime('%-d')
msg = '{ "state": "' + msg + desc1 + '", "clocktalk": "' + desc2 + '. " }'
publish.single('rPi-HA/nextGoodMorning', msg, qos=0, retain=True, hostname="localhost", auth = {'username':"xxxxxx", 'password':"xxxxxx'})
@service
def run_create_next_goodmorning():
import sys
if "/config/pyscript/modules" not in sys.path:
sys.path.append("/config/pyscript/modules")
import create_next_goodmorning
create_next_goodmorning.mainline()
Thanks, that looks quite promising. I’ll have a go tomorrow (getting a bit late now as I’ve still got other things to do).
Don’t forget to add pyscript to your configuration, good luck
It’s important to know that the shell command will be executed within the context of Home Assistant’s docker container (if you are using Home Assistant OS, Home Assistant Supervised, or Home Assistant Container).
To confirm your python script is called with the correct path, open a shell within Home Assistant’s docker container using the following command and then try calling your python script. If it’s not found then you’ll need to explicitly indicate the correct path.
docker exec -it homeassistant bash
Once you know you have the correct path, create a shell_command (see link above). Example:
shell_command:
my_modbus: /path/to/python3 /path/to/my_python_script.py
Then make a simple automation that calls the shell_command when Home Assistant starts.
alias: example
trigger:
- platform: homeassistant
event: start
condition: []
action:
- service: shell_command.my_modbus
Cyn and Taras, thank you both very much. Using Shell command looks like it is the simplest option. Unfortunately I got bamboozled by the documentation, eg the first example “restart_pow: touch ~/.pow/restart.txt” doesn’t really suggest it can also be used to run a python script, more that it can be used to run linux cli commands. In fact, I’m not even sure what “restart_pow: touch ~/.pow/restart.txt” really means - I thought “touch filename.txt” created an empty file called filename.txt but the shell_command line starts restart_pow, all rather confusing.
I already know my scripts will run after using “python myscript.py” after the “docker exec -it homeassistant bash” command, because that is how I currently run them. I have high hopes using shell_command will work, will report back once I have confirmed it does (or, hopefully not, doesn’t).
If it is not running in HA container but it is running in a ssh terminal, you can use a shell command like this… see examples below (the procedure will be executed like being in a SSH terminal). I am using the add-on “SSH & Web Terminal” sponsored/supported by Frenck …
The first example, you can pass arguments from HA to the shell script (where you can launch your python program for example). In the second example, the output of the shell script is stored in a log file…
send_sms: ssh Your_SSH_Username@HA_IP_Adrress -o 'StrictHostKeyChecking=no' -i /config/.ssh/id_rsa '/bin/bash /config/private_shell_scripts/send_sms_ini.sh "{{ arguments }}"'
list_entities: ssh Your_SSH_Username@HA_IP_Adrress -o 'StrictHostKeyChecking=no' -i /config/.ssh/id_rsa '/bin/bash /config/private_shell_scripts/list_entities_ini.sh >&/config/private_logfiles/list_entities_ini.log 2>&1'
I am using this mainly for accessing modules/packages not available in HA container… but available or added as additional “Alpine packages” in SSH & Web Terminal (see documentation for more details on this feature)…
Trying the shell_command route, and I’m now currently in linux path hell. I can still run the scripts in the terminal using python ./mypath/myscript.py but the HA shell_commands fail with and without the . prefix and/or the paths to the files in the scripts are screwed up. Difficult to know, the logs are the usual mess. Why is everything so impossibly difficult in HA?
In the documentation, it says “The command is executed within the configuration directory.” Surely that is /config so if my script is in config/mypath/myscript.py then the path is just /mypath/myscript (no . prefix).
Likewise, if the file the data gets written to is also in /mypath, then the path to it from inside the script is still /mypath/mydata.csv because the script is running in /config not /mypath.
Because all this is running behind closed doors there is no way to test what the paths actually are.
Some feedback on various trials of shell_command:
without the dot prefix to the path, I get error code 2 - file not found? Not clear…
with the dot prefix to the path I get this in HA’s log:
2023-03-28 11:35:26.920 ERROR (MainThread) [homeassistant.components.shell_command] Timed out running command: python ./modbus/modbus.py, after: 60s
Traceback (most recent call last):
File “/usr/local/lib/python3.10/asyncio/streams.py”, line 502, in _wait_for_data
await self._waiter
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.10/asyncio/subprocess.py”, line 195, in communicate
stdin, stdout, stderr = await tasks.gather(stdin, stdout, stderr)
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.10/asyncio/tasks.py”, line 456, in wait_for
return fut.result()
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/usr/src/homeassistant/homeassistant/components/shell_command/init.py”, line 83, in async_service_handler
stdout_data, stderr_data = await asyncio.wait_for(
File “/usr/local/lib/python3.10/asyncio/tasks.py”, line 458, in wait_for
raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError
Using ps in the terminal doesn’t show modbus.py is running.
Conclusion: neither /path (no dot prefix) or ./path (with dot prefix) work…
Have you ever looked at AppDaemon?
I initially installed it to run the “schedy” heating scheduler but I ended up doing a lot of things with it
That’s quite literally the killer. Apparently shell_command terminates any process after 60s. Full stop. No questions. No prisoners. Another dead duck floating in the HA pool, another wasted morning I shall never get back.
One of the main if not the main problem with HA is the failure of basic documentation. Why doesn’t the Shell_command documentation say the command WILL be terminated after 60s? All it mentions is a passing throwaway “In case a command results in a non 0 exit code or is terminated after a timeout of 60 seconds, the result is logged to Home Assistant log.” Nothing about the termination being by design.
I may give the pyscripts a go once I have got over my frustration.
@jfparis - I did look at AppDaemon but found the documentation was written in gobbledygook and gave up. I may have to have another look if pyscripts fails.
Yes. Shared that feeling. I worked my way through the examples.
I believe the operative word is timeout in “timeout of 60 seconds”.
Perhaps it’s not as clear as you want it to be but it also wasn’t crystal clear that your python script apparently runs forever:
The script collects modbus data and logs it to a file.
Immediately, within seconds, or runs forever? It’s unclear from that description. 
The good news is that you can improve it because it’s composed of user contributions. Scroll to the bottom of the documentation page and click the Edit button. You can submit a Pull Request, containing your suggested changes, and a member of the development team will vet it. If accepted, it will be merged into the official documentation.
Something that times out is terminated. I used the more active verb because HA actively terminates (or times out if you prefer) the script without telling you it will do that. You only find out in the logs, and even there it is buried in a lot of gobbledygook.
Fair enough, but I thought it was implied, insofar as there would be little point in reading modbus data once. It actually reads it once a minute, and is intended to run forever. Another script aggregates the data every hour into hourly data and also logs that. It is also intended to run forever.
Done, adding a new second paragraph:
“Note that the shell command process will be terminated after 60s, full stop, there are no options to alter this behaviour. This is by design, because Home Assistant in not intended to manage long-running external processes.”
We’ll see what happens…
More trouble…this time trying pyscripts. Why am I not surprised it doesn’t work? In fact, I can’t even install it.
I followed the manual installation instructions to the letter:
downloaded latest and unzipped hass-custom-pyscript.zip into config/custom_components/pyscript
added this to config/configuration.yaml:
pyscript:
allow_all_imports: true
hass_is_global: true
check configuration prior to restarting HA only to get:
Configuration invalid!
Integration error: pyscript - Integration ‘pyscript’ not found.
Other custom components I have work.
I’m not going to start a rant because if I do, I’ll probably never stop…
PS I see my suggested addition to the Shell Command documentation has been added, I appreciate the prompt and positive response to my suggestion. Good thing I didn’t start that rant…
I think I might have cracked this, based on posts here:
and some key documentation here:
GitHub - aio-libs/async-timeout: asyncio-compatible timeout class.
I am running HA OS on a mini PC, seems to work for this setup, not sure about others because I can’t test on setups I don’t have.
Objective: to run a python script (or scripts) every time HA starts and have the script carry on running in the background (I’m using two to collect process and log modbus data) without being terminated by Home Assassin.
Overview: create a new custom_component x_shell_command and modify it to remove the timeout, which uses async-timeout, which can be disabled by setting the timeout to None. From the above
github link: “timeout parameter could be None for skipping timeout functionality”. Then use x_shell_command with the time out removed instead of shell_command.
No doubt this will be seriously frowned upon, but if it works… I will of course keep an eye on things to see if it goes rogue.
Steps:
(1) create a folder in custom_components called ‘x_shell_command’
(2) download the three files __init__.py manifest.json and services.yaml from here (this is the HA core github) to your newly created ‘x_shell_command’ folder
(3) edit the manifest.json file so it looks like this:
{
"domain": "x_shell_command",
"name": "Shell Command (Extended)",
"version": "1.0",
"documentation": "https://www.home-assistant.io/integrations/shell_command",
"codeowners": ["@home-assistant/core","@local"],
"iot_class": "local_push"
}
(4) edit the __init__.py file lines 17 and 19 as follows:
DOMAIN = "x_shell_command"
COMMAND_TIMEOUT = None
(5) Add the following to the top of your python script. This (apparently) has the same effect as running ‘nohup … &’ on the command line (eg ‘nohup python myscript.py &’) ie no hangups allowed and run in background:
import os
import sys
os.fork() and sys.exit()
(6) Add the following to /config/configuration.yaml (you can add more than one entry as long as the have different names, and note that dot at the start of the path):
x_shell_command:
run_myscript1: python ./pathto/myscript1.py
run_myscript2: python ./pathto/myscript2.py
(7) Add the following to /config/automations.yaml:
alias: run_myscript
trigger:
- platform: homeassistant
event: start
condition: []
action:
- service: x_shell_command.run_myscript1
- service: x_shell_command.run_myscript2
(8) Stop the scripts if they are currently running (via terminal) and then check configuration (Developer Tools > YAML tab, correct if necessary but it should be OK) and if OK restart HA twice, the first time to load custom component x_shell_command, the second to trigger the actions.
I think that is it. The logic is simple enough for even me to follow it. Restart trigger > run custom x shell command action > start python script(s). Any comments welcome.