Pyscript - yet another Python scripting component

Development
,
1

I’ve developed an initial version of pyscript, a new component for user-created python scripting.

The git repository is home-assistant-pyscript, which contains three directories (one for the component itself, a testing directory, and the markdown documentation, as rendered by github).

This integration allows you to write Python functions and scripts that can implement a wide range of automation, logic and triggers. State variables are bound to Python variables, and services are callable as Python functions, so it’s easy and concise to implement logic.

Functions you write can be configured to be called as a service or run upon time, state-change or event triggers. Functions can also call any service, fire events and set state variables. Functions can sleep or wait for additional changes in state variables or events, without slowing or affecting other operations. You can think of these functions as small programs that run in parallel, independently of each other, and they could be active for extended periods of time.

State, event and time triggers are specified by Python function decorators (the “@” lines immediately before each function definition). A state trigger can be any Python expression using state variables - the trigger is evaluated only when a state variable it references changes, and the trigger occurs when the expression is true or non-zero. A time trigger could be a single event (eg: date and time), a repetitive event (eg: at a particular time each day or weekday, or daily relative to sunrise or sunset, or any regular time period within an optional range), or using cron syntax (where events occur periodically based on a concise specification of ranges of minutes, hours, days of week, days of month and months). An event trigger specifies the event type, and an optional Python trigger test based on the event data that runs the Python function if true.

Pyscript implements a Python interpreter using the ast parser output, in a fully async manner. That allows several of the “magic” features to be implemented in a seamless Pythonesque manner, such as binding of variables to states, and functions to services. Pyscript supports imports, although the valid import list is restricted for security reasons. Pyscript does not (yet) support some language features like declaring new objects, try/except, eval, and some syntax like “with”. Pyscript provides a handful of additional built-in functions that connect to Hass features, like logging, accessing state variables as strings (if you need to compute their names dynamically), sleeping, and waiting for triggers.

Pyscript provides functionality that complements the existing automations, templates, and triggers. It presents a simplified and more integrated binding for Python scripting than the Python Scripts component, which provides direct access to Hass internals.

Anyhow, I’m very new to Hass (first post), and only a few months into writing Python code, so feedback would be appreciated. It’s possible pyscript uses the internals in incorrect ways, or there are better ways to do things, or some features are missing. Also, the tests are pretty incomplete - for starters I’ve only run them on Python 3.7, but I’ll work to improve coverage soon.

4 Likes
2

Pyscript has been released as a custom component in HACS. You can install it using HACS -> Integration, select “+” and search for “pyscript”.

3

I didn’t catch this when it was released, but this is an interesting looking project! I will say that, if you are fairly new to Home Assistant, and you haven’t heard of AppDaemon, you may want to check it out. There is definitely some overlap here in terms of goals and functionality. That being said, even as someone who’s already pretty invested in using AppDaemon, I may have to check this out. I like the use of decorators and the ease of exposing things as services that can be called from Home Assistant, as that’s one thing that AppDaemon does not offer.

4

Thanks for the pointer about AppDaemon! I wasn’t aware of it. Yes, it has very similar goals and has some similar features - definitely an impressive project. It’s fun to see an independent development result in some similar design choices and features.

In the spectrum of different automations, we have yaml at one end (simplest but least flexible, and can become very verbose), followed by templates (more flexible, but a bit arcane), and Python Scripts at the other (like bare metal - you need a good understanding of internals, and running async tasks and high-level actions takes a lot of scaffolding). Pyscripts and AppDaemon are somewhere in the middle - trying to be flexible and powerful with much of the complexity hidden from the user. I’ll add a mention of AppDaemon to pyscript’s README.md.

For pyscripts I tried to abstract away almost all the usual scaffolding to make it as simple as possible, while trying to maintain a high level of power and expressiveness. AppDaemon does that too - while it does have some scaffolding (but much less than Python Scripts), it offers significant power too.

There are some features in AppDaemon like randomized time intervals and log-based triggers that I could implement in pyscript if there is interest. The accelerated time testing mode is really clever - not sure how I can do that since pyscript runs inside HASS. Maybe I should have a standalone interactive test mode where you can advance the time, set state variables and call services to make sure your code does the right thing?

5

I never really got into AppDaemon. So, now I have to ask: why would I want to pick one or the other?
I like Pyscript’s approach, being a component for HASS. With AppDaemon, there seems to be quite a bit of bloat, including the dashboard. And I have to run a separate process.
Is Pyscript at a disadvantage when it comes to running background tasks?

6

Pyscript can also have long-running functions. They run as async tasks in the main event loop. They can sleep or wait for triggers (time-based, state changes or events), while implementing any application logic. But typically most functions are short-lived and run only when trigger conditions occur, since pyscript allows a lot of flexibility in specifying triggers, meaning the functions usually just implement the action following a trigger.

I wrote a wiki entry comparing pyscript to AppDaemon.

The next major feature I’m implementing is supporting Jupyter frontends. That allows fully interactive development and testing of pyscript code. You can write and immediately run any snippets of code, including calling services, firing events, viewing or setting state variables, writing and testing expressions as you develop your automation logic, and creating triggers and functions. It also supports autocompletion, so you can readily see available state variables or functions. Hopefully I’ll commit those changes to github in the next few days, and release a new version of pyscript within a week or two.

4 Likes
7

I just wrote my first “real” bit of pyscript recently (see A new approach for Xiaomi BLE Temperature sensors with ESPHome, MQTT and pyscript - #5 by swiftlyfalling) and the Jupyter notebook interface is wonderful! It’s such a great way to interactively tweak and test and do inspection of state. Very powerful, and a great learning tool as well.

I’ve done a little bit of work with AppDaemon as well, and there’s often comparisons between the two. This Jupyter interface is a real differentiator between the two, beyond the stylistic differences and choices made in each implementation.

8

Thanks for posting this example - it’s fun to see new applications. And thanks for the feedback too.

9

Pyscript is exactly what I needed to populate an input_select dynamically with a list of movies obtained from my Kodi media player. So thanks! I simply couldn’t get this to work using any other means.

But when HA restarts, the input_select reverts back to its static option list, which is required in order to define it. So I got the idea to persist my dynamic option list using pickle, and create an automation to run another pyscript at HA startup to restore the option list from the saved file.

Of course, I can’t use file I/O in pyscript, so I’ve written a standard Python module to do the I/O. The problem I’m having is how to include my I/O module in pyscript. I haven’t been able to figure out where to place the Python module, how to include it in the pyscript module, and actually have the task.executor() call work.

The doc talks about modifying sys.path and putting modules in config/pyscript_modules. I tried that but get this error when calling pyscript.reload:

2021-01-30 13:25:53 ERROR (MainThread) [custom_components.pyscript.file.restore_all_movies] Exception in </config/pyscript/restore_all_movies.py> line 11:
    import file_ops
    ^
ModuleNotFoundError: No module named 'file_ops'

I’ve also tried putting the Python module in config/pyscripts/modules (which does not give an error concerning the import), but the response from task.executor() is:

<coroutine object EvalFuncVarAstCtx.__call__ at 0x70a81ae8>

instead of the expected movie list. I’m wondering if this is because modules included under the pyscript directory are expected to be pyscript and not regular Python (the doc contains this statement: " Pyscript code can be put into modules or packages and stored in the config>/pyscript/modules folder.")?

Any help is greatly appreciated!

10

Sounds like a cool application. There are several alternatives.

First, pyscript supports persistence of pyscript entities (state variables) by calling state.persist. So you could stash your settings in some pyscript state variable (which has to be a string, so perhaps json would be a good format) and it will be persisted. Then you could use a startup time trigger to initialize the input_select on startup. But this method doesn’t help if the data comes from some external source and isn’t easily available or generated in pyscript.

Your native python module should go in a directory like config/pyscript_modules. For pyscript to find modules in that directory, you have to add it to sys.path. That is accomplished by putting this code in one of your pyscript files (not in the native python module):

import sys

if "config/pyscript_modules" not in sys.path:
    sys.path.append("config/pyscript_modules")

If you have that already, perhaps it gives an error (have you checked the log?), since you can’t import sys without setting the allow_all_imports yaml config parameter to true.

If you are using Jupyter, you can easily check sys.path interactively to make sure it contains that directory. Just type this in a Jupyter cell and look at the list it displays:

import sys
sys.path

If your I/O function is short, an alternative is to use the new @pyscript_compile decorator (which is available in the master version and will be in the soon-to-be-released v1.2.0), which allows you to define native compiled python functions in the middle of a pyscript file. They can then be called via task.executor, or used anywhere python needs a regular function (eg, callbacks). See the docs.

Finally, as you discovered, task.executor can only call native python functions. I just updated the code to give a clearer error message if you call it with a pyscript function.

Anyhow, that’s a long answer, but hopefully you have a few different ideas about how to solve your problem.

11

Thanks for the detailed explanation, very helpful! I used the append to sys.path method, which I did try before, but had to add a slash at the start of the appended path, like so:

if "/config/pyscript_modules" not in sys.path:
    sys.path.append("/config/pyscript_modules")

Thanks again!

12

I guess the relative path doesn’t work on some installations. I’ve updated the docs.