Pyscript - new integration for easy and powerful Python scripting

Share your Projects!
,
#1

Pyscript is a new integration that allows you to write Python functions and scripts that can implement a wide range of automation, logic and triggers in a very simple manner. State variables are bound to Python variables, and services are callable as Python functions, so it’s easy and concise to implement logic.

Functions you write can be configured to be called as a service or run upon time, state-change or event triggers. Functions can also call any HASS service, fire events and set state variables. Functions can sleep or wait for additional changes in state variables or events, without slowing or affecting other operations. You can think of these functions as small apps that run in parallel, independently of each other, and they could be active for extended periods of time.

Pyscript has some similar features to AppDaemon, which I just found out about. It complements the existing yaml automations and templates, and provides a much easier interface than Python Scripts

Pyscript is available as a HACS custom component. To install, go to HACS -> Integrations, click on “+” to search for Pyscript, and install. See the repository for the documentation.

I’d appreciate any feedback or suggestions to improve it.

34 Likes
Why the heck do automations require Jinja templates instead of real Python?
Complex Automations - State Machine
Outdoor illuminance template sensor
'any' is not defined in Python scripts
Set Value to "0" if entity is not Available
Can't find python
Folder Watcher | file sensor?
Run (Domoticz) Python scripts in HA
Human occupancy emulation - a noob question
Best way to implement "away"
Australian BOM Rain Radar Card
Best practices to keep automations readable
HomeAssistant OS or Raspberry OS
#2

Sounda interesting, what are the advantages compared to AppDaemon?

1 Like
#3

Thank you for your contribution to Home Assistant. So far, my experience with pyscript is limited to just reading its documentation. Its ability to permit importing (some) modules immediately makes it more capable than the python_script integration. In addition, its syntax, such as for calling a service or accessing an entity’s state, is simpler and easier than python_script.

It also offers the ability to use cron as a trigger which means one can easily create sophisticated time-based triggers.

The next step for me is to experiment with pyscript and see if it overcomes some of the limitations I’ve encountered in python_script: For example:

  • Can we import the json module? I want to pass a dictionary as an argument and then, within the script, easily access the dictionary’s items. To achieve that, I need to do a json.loads() and python_script doesn’t support that natively and disallows importing the json module.
  • Maybe this is a naive question but how do you stop a running pyscript function? For example, after starting a pyscript function that employs a time-based trigger, I assume it survives a restart so, unlike an automation that can be turned on/off, how does one stop a running pyscript function?

EDIT
I believe I’ve answered my first question; one cannot import the json module.

Exception in /config/pyscript/json_test.py line 1 column 0: import of json not allowed

Should I submit a feature request in your pyscript repo to permit the import of the json module? Or is there some reason that it cannot be imported?

#4

Thanks for checking it out and the excellent questions.

The two issues with allowing more imports are:

  • Functions that do I/O (ie, can block while reading/writing files) should not run inside async tasks (which is how pyscript runs everything). So importing a module that includes functions that do I/O (eg, json.dump()) is not currently allowed to avoid those functions. That said, I could consider supporting that by running such functions as executor jobs, but there would need to be some way of denoting which functions in each module need to be called in that manner - perhaps just a list of “I/O” functions in useful modules could be created. If you only want the json module to just convert to/from strings, then importing it should be fine. You can test it by just adding it to the list of ALLOWED_IMPORTS in custom_components/pyscript/eval.py (you’ll need to re-start HASS after adding it).
  • Potential security issues if imported functions can make system calls or access HASS internals (eg, through manipulating objects etc). Perhaps this isn’t a real issue, since if you are writing pyscripts you could just as easily directly modify the HASS code. I’d appreciate feedback here. I saw that Python Scripts was careful about sandboxing, so I adopted a similar approach.

On the stopping/restarting question, none of the functions that are waiting for triggers (using the decorators) are considered running. So when you reload, all those pending triggers are canceled and replaced by whatever is specified in the newly loaded scripts. I should make that clearer in the documentation. The only case of a truly “running” function is a function that is actually executing after a trigger occurred, and hasn’t yet returned. Reload doesn’t cancel those functions. Usually those functions run for just a short time. However, there could be cases where a function uses task.sleep() or task.wait_until() to wait a while before doing something else. I debated whether it was the right default to keep or cancel those truly running functions on reload. In the current version I chose to keep them running on reload, but I’m definitely open to changing that, or providing an option that allows you to cancel those on reload too.

2 Likes
#5

As user of Appdaemon, I will try to play with your project to see if I can migrate easily some of my automation.

#6

i just looked at it, but i dont see any.
its a complete different approach which is way more object orientated then AD.

i dont think there is anything you can do, that you cant do with AD, only different.
the other way around is certainly true.
so its a matter of taste.

2 Likes
#7

I thought the same, but I don’t see any readon why I should do that yet.

#8

Great to have choices. Well done. :+1:

#9

@Burningstone Good question about AppDaemon. First, a caveat - I’ve only read its documentation and not actually used it (I actually only heard about it a few days ago, since I’m quite new to HASS), and I spent some time re-reading the docs before responding.

Overall the goals of pyscript and AppDaemon are similar - allow users to implement powerful Python-based automation, triggers, logic and actions with much less scaffolding, overhead and expertise than HASS internals or Python Scripts require, and with a lot more flexibility and richness than the builtin yaml automations and triggers.

Broadly, pyscript has a higher-level of abstraction than AppDaemon, meaning it has less details the user has to code or worry about. There will be many cases where pyscript and AppDaemon are relatively similar in terms of user implementation complexity. There will be cases where pyscript is simpler and easier. There will be cases where AppDaemon can do things that pyscript cannot (at least not yet). Please tell me about those latter cases since I’m happy to implement new features.

Here are a few more specific differences:

  • Pyscript allows state triggers to be full Python expressions involving multiple state variables; AppDaemon allows callbacks to be attached to each state change event, so it can implement a trigger expression involving multiple variables, but it requires a couple more steps - attach a callback to each state variable and put the trigger logic inside the callback.
  • An automation with multiple steps (eg, do something, wait a while for something to happen, then do something else) can usually be implemented as a single function in pyscript, which can sleep or wait for new triggers in the middle of any Python code without needing to split things up using callbacks. In AppDaemon, each step typically will need a callback to wait for some time or another event like a state change. (AppDaemon also allows yaml sequences to execute several automation steps, but generally they don’t include conditional logic.) It’s usually more difficult to split up logic across multiple callbacks, especially when some callbacks can involve two outcomes (eg: handle either outcome of waiting for a state change within a timeout) versus just an if statement based on the return value of task.wait_until().
  • Pyscript allows HASS state variables to be directly used as Python variables, versus the helper functions (get_state() and set_state()) in AppDaemon. That’s not much difference. Plus pyscript provides helper functions too in case you need to set attributes or dynamically compute the state variable name.
  • Pyscript is fully async based and runs inside HASS, without the user having to use await or even know what async is. AppDaemon uses a thread per app, although it supports async tasks too. AppDaemon uses threads and it supports async tasks too, although the AppDaemon
    documentation notes that the user needs more manual control and expertise to use async correctly.
  • Since async tasks are much more lightweight than threads, pyscript should easily be able to support hundreds or thousands of functions/apps. AppDaemon uses more resources with one thread per app, but it can also support async tasks without one thread per app. However, as its docs say, the user needs more manual control and expertise to use async correctly.
  • Pyscript has some richer time triggers, like cron.
  • Pyscript interprets the Python code, which is how it hides almost all the scaffolding and complexity, but long pieces of user code will run slower because it’s an interpreter (all imports are native, so they run at the same speed as real Python). AppDaemon uses native Python code, so it will run faster. Offsetting that is that pyscript runs inside HASS, but AppDaemon is a separate process, so all events and state changes have some IPC and context switch overheads.
  • Pyscript’s interpreter doesn’t implement all of Python - it doesn’t (yet) support new classes, try/except, eval, with, yield and generators. AppDaemon is native Python, so the whole language is available.
  • Any pyscript functions you designate can be called as services from HASS. @apop mentioned that AppDaemon doesn’t support apps to be exposed inside HASS as a service.
  • AppDaemon has some clever testing features like accelerated time to debug your time trigger logic. I need to think about how to make pyscript script testing easier.
  • AppDaemon has some other interesting features like log message triggers. AppDaemon has a lot of other interesting features, including an admin dashboard showing detailed status, direct access to MQTT, plugins and more.

Please tell me about any corrections or omissions, or if you think this isn’t balanced or fair, and I’ll edit the list.

Edited Aug 3, 2020 with feedback from @ReneTode. I’ve created a Wiki entry with the new text.

5 Likes
#10

I’ve been running into some issues with my automations (mostly the thermostat ones) and was going to look into rewriting them this weekend anyways. Now I’ll take a look at this and see if I can redo them with this.

1 Like
#11

To compare the required amount of code or the readability.

1 Like
#12

i think thats why it is that several of your statements are incorrect.

  • AD can create services that can be called from HA (allthough i really never understand why people would like that if you use AD for automation.) or at least with an extra step.

  • automations with multiple steps can be created in AD easy wit sequences

  • state variables in AD are also directly available

  • you can run AD fully async if you want to, but threading has big advantages.

  • you dont need to run 1 thread per app, the threads and which app runs in which thread are fully configurable

  • AD can certainly run hundreds or thousands of functions/apps as well. at this point i am around 125.000 callbacks a day

and you also miss a lot of things that are recent and not really good described in the docs.

  • an admin interface showing you all your apps, threads, calllbacks, when they run, your entities and more
  • a fully configurable dashboard that can directly interact with the apps, which can even be created by apps.
  • direct access to MQTT
  • plugins that can connect to any other program you like (DB plugin coming up AD-AD plugin coming up, but also the possibility to connect to any program like HA)
  • a lot of other things that i dont have the time to write all down.

like i said before pyscript does things differently, which gives people options to chose.
but AD has way more possibilities.

1 Like
#13

readability really is based on the user.
whay 1 finds more readable the other finds less readable.

thats why its great to have options.

#14

Thanks for the detailed explanation.

Ultimately, I’ve discovered that, at least for my immediate needs, I don’t actually need the json module. :man_facepalming:

I was attempting to pass this sample data (in both pyscript and python_script):

birthday: 
  - name: Bob
    date: '1955-10-15'
    reminder: '0 1 3 5'
  - name: Carol
    date: '1953-11-05'
    reminder: '0 1 3 7'
  - name: Ted
    date: '1956-07-21'
    reminder: '0 1 3 7'
anniversary: 
  - name: Bob and Carol
    date: '1975-01-01'
    reminder: '0 1 3 5'
  - name: Ted and Alice
    date: '1977-05-08'
    reminder: '0 1 3 7'

In python_script, I used dot notation to access name in the birthday list and it returned None. I got the same result when I repeated the experiment in pyscript. I assumed the data was still a JSON string and I needed json.loads() to convert it

I then tried bracket notation and discovered it works perfectly, meaning it’s already a python list/dict (in both python_script and pyscript).

So given the data shown above, this:

@service
def json_test(birthday=None):
    log.warning(f"json_test: {birthday[1]['name']}")
    log.warning(f"json_test: {birthday[1].name}")

produces this:
Screenshot from 2020-08-03 17-42-05

So chalk it up to a novice error but ultimately I don’t need the json module. Nevertheless, thank you for explaining how it’s possible to import it.

OK, that’s clear.

#15

@ReneTode - thanks for the corrections and additional information about AppDaemon. I’ve updated my earlier post and I’ve also created a Wiki page with the new text.

2 Likes
#16

Good detective work.

If you are unsure of the data type, you could initially just log the whole thing:

@service
def json_test(birthday=None):
    log.warning(f"json_test: {birthday}")
#17

Are you still in the AppDaemon dev team?

#18

Any chance you could put some examples of the different trigger events on the github readme (or wiki)

I’m fairly certain that I know how they should work but examples to follow are always welcome. (I’m willing to share mine once I write them)

#19

I created a new wiki page with some discussion and examples of event triggers. I haven’t tested the examples yet, and I need to add some text for user-defined events.

The wiki is open for any git user to edit, so you are welcome to create new pages or edit existing ones.

#20

Is this in the default repositories for HACS? It doesn’t seem to show in the search for me unless I add it as a custom repository.