Got a minute to test a python_script?

[123 (Taras)]

Something that used to work now generates an error (after upgrading). I distilled the problem down to this simple, easy to test, python_script.

test1.py

def check_date(date):
  if not len(date) >= 10:
    return None
  else:
    date = date[:10]
  try:
    dt = datetime.datetime.strptime(date, "%Y-%m-%d").date()
    return dt
  except ValueError:
    return None

dt = check_date('2021-05-20')

Executing python_script.test1 produces the following error message:

Logger: homeassistant.components.python_script.test1.py
Source: components/python_script/__init__.py:221
Integration: Python Scripts (documentation, issues)
First occurred: 11:53:20 (1 occurrences)
Last logged: 11:53:20

Error executing script: '__import__'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/python_script/__init__.py", line 221, in execute
    exec(compiled.code, restricted_globals)
  File "test1.py", line 12, in <module>
  File "test1.py", line 7, in check_date
KeyError: '__import__'

For some reason, this line is the culprit (but not in the past):

    dt = datetime.datetime.strptime(date, "%Y-%m-%d").date()

I may be wrong but the python interpreter seems to have trouble importing the datetime module (I think).

Can someone else try this and report if it generates the same error or not?

[CentralCommand (Mike)]

I don’t actually, I just tried it. I set the log level for python script up to debug but the only log I see is this:

2021-05-20 13:44:59 INFO (SyncWorker_26) [homeassistant.components.python_script] Executing test1.py: {}

I don’t see any errors like you’re showing.

I know python scripts execute in a sandboxed environment which prevent the use of any import statements. It looks like that’s what your script is hitting? But its confusing because there’s no import statement at that spot, its like its trying to import datetime when you go to use it and then failing.

[123 (Taras)]

@CentralCommand

there’s no import statement at that spot, its like its trying to import datetime when you go to use it and then failing.

Exactly and that’s what has me confused.

The weird thing is it worked when I upgraded from 2021.4.6 to 2021.5.1. Today I upgraded to 2021.5.5 and then that happened.

So I downgraded back to 2021.5.1 and it still generated the same error?!? I downgraded yet again to 2021.4.6 but the error remained.

None of that makes sense to me because I am using Home Assistant Supervised so its operating environment, including its python interpreter, is self-contained (i.e. docker container). Reverting to 2021.5.1 should have put everything back to the way when it worked but it didn’t. I can’t think of any external configuration that can influence the python_script integration across versions.

I am now back to 2021.5.5 and at a loss to understand what is causing this fresh hell. The datetime module ought to be baked into the python_script integration so references to it should not cause errors about __import__ and “restricted_globals”.

FWIW, I opened an Issue in the Core repo but I think it will get a very low priority. Until this is resolved, a python_script I have used for months remains dead in the water.

[123 (Taras)]

Here’s line 12 of the python_script integration’s init file:

It imports utility_builtins from RestrictedPython (which, according to its PyPi page, hasn’t changed since October 2020).

The python_script integration hadn’t had any major changes with the latest minor one occuring at the end of March.

A showstopper (for me) with no obvious cause.

[123 (Taras)]

I upgraded a second instance of Home Assistant, also running Home Assistant Supervised on Debian 10 (but its an RPI instead of a generic PC) and executing the python_script doesn’t produce the error.

So it’s not reproducible on the second instance, nor by CentralCommand, thereby making it all the more challenging to resolve on the first instance!

[VDRainer]

Tested your script on core-2021.5.5 in 3.8.10 venv and HA OS with 2021.5.5.
Both without errors.
Good luck!

[123 (Taras)]

I’ve gathered a lot of information to characterize the problem but, unfortunately, nothing to solve it yet.

The problem persists regardless of the version I have tried. It even persists if I uninstall/re-install the python_script integration.

(I uninstalled the python_script integration, and deleted its related directory, upgraded to 2021.5.5, recreated the directory and re-installed the python_script integration and … the error message persists.)

---

This is sufficient python code to produce the error message:

test1.py

dt1 = datetime.datetime.strptime("2021-05-20", "%Y-%m-%d").date()
dt2 = datetime.datetime.now().strftime("%Y")

Its the first line’s use of strptime that causes the error KeyError: '__import__'.

Error executing script: '__import__'

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/python_script/__init__.py", line 221, in execute
    exec(compiled.code, restricted_globals)
  File "test1.py", line 1, in <module>
KeyError: '__import__'

In contrast, the second line, the one using strftime, causes the same error message on my second instance of Home Assistant. However, there is precedent for strftime causing an error:

Python_script for countdown to birthdays/anniversaries/significant dates

… still some left. I think we need to quote the ‘birthday’ because it errors out on not defined otherwise. friendly_name = "{}'s {}".format(name, eventtype) if eventtype=='birthday' else "{} {}".format(name, eventtype) more importantly: Fri Nov 22 2019 16:22:05 GMT+0100 (CET) Error executing script: '__import__' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/python_script/__init__.py", line 195, in execute exec(compiled.code, restricted_globals…

How to: python time notation to strip (leading) zero

yes it does, and i must confess it was what I started to look for in the beginning (the dash ‘%-h’) but didn’t realize it could be done directly in python scripts… I use it in yaml all the time duh. no it doesnt… sorry: Fri Oct 11 2019 13:16:44 GMT+0200 (CEST) Error executing script: '__import__' Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/python_script/__init__.py", line 195, in execute exec(compiled.code, restricted_globals, loc…

And as far back as 2017 where it even surprised balloob:

Does the datetime module work in python_script?

@balloob that does not work. Here’s the error: 2017-10-12 15:37:32 ERROR (Thread-4) [homeassistant.components.python_script.time_test.py] Error executing script: '__import__' Traceback (most recent call last): File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/python_script.py", line 143, in execute exec(compiled.code, restricted_globals, local) File "time_test.py", line 1, in <module> KeyError: '__import__'

The odd thing is that my first instance of Home Assistant accepts neither strptime or strftime whereas my second instance accepts strptime but not strftime. How is this even possible when both use the same version of Home Assistant?

Home Assistant’s implementation of RestrictedPython appears to allow for both:

[Mariusthvdb (Marius)]

afraid this is not going to help, but I just ran your script, and exactly nothing is in the logs.
so, I added

logger.error("Test1 for Taras: " + str(dt) )

and the result is:

[123 (Taras)]

Thanks for trying it.

If you have a moment, try this one. It will probably fail on the second line (strftime has definitely failed for you in the past).

dt1 = datetime.datetime.strptime("2021-05-20", "%Y-%m-%d").date()
dt2 = datetime.datetime.now().strftime("%Y")

Something is wrong with the python_script integration because strftime shouldn’t be rejected (it’s commonly used and is definitely “safe”). Plus one of my two Home Assistant instances rejects both strftime and strptime. This kind of variability shouldn’t be possible.

[Mariusthvdb (Marius)]

dt2
dt1
dt

[Troon]

On my Synology NAS:

2021-05-21 13:51:44 INFO (SyncWorker_4) [homeassistant.components.python_script] Executing test1.py: {}

EDIT: your second one returned this:

2021-05-21 13:54:24 INFO (SyncWorker_6) [homeassistant.components.python_script] Executing taras2.py: {}
2021-05-21 13:54:24 ERROR (SyncWorker_6) [homeassistant.components.python_script.taras2.py] Error executing script: '__import__'
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/python_script/__init__.py", line 221, in execute
exec(compiled.code, restricted_globals)
File "taras2.py", line 2, in <module>
KeyError: '__import__'

[123 (Taras)]

@Mariusthvdb

dt2 <--- Error executing script:'__import__'

As expected, strftime appears to be universally unacceptable to the python_script integration.

I have the bad luck of having a python_script integration that also dislikes strptime.

The ability of python_script integration to behave differently on two instances is incomprehensible to me. I don’t know what external factors could influence its behavior.

[Troon]

@123

I don’t know what external factors could influence its behavior

Architecture?

[Mariusthvdb (Marius)]

yes, and re-reading those links you posted, if you desperately need the strftime, you could use AppDaemon as Burningstone suggests.
Besides that, it really feels like a bug, this should not be happening. Did you file an issue yet?
btw mere dt = datetime.datetime.now() returns the same

[123 (Taras)]

@Troon

Architecture?

The one that doesn’t accept strptime is running on a generic PC (Intel CPU). Both are using Home Assistant Supervised on Debian 10.

[123 (Taras)]

In this case, I don’t need strftime, I need strptime.

If you are still using the code discussed in the “Python_script for countdown to birthdays/etc” thread, then you have this line somewhere in your system:

origin = datetime.datetime.strptime(dateStr, fmat).date()

If you were experiencing the problem I am currently experiencing, that line would fail. Without strptime, converting date strings to datetime objects becomes messy.

---

BTW, this happened to me ‘out of the blue’. After upgrading to 2021.5.5, I noticed a sensor, normally generated by a python_script, was missing. I tracked the problem down to strptime now no longer acceptable within the sandboxed environment. If this can happen to me without any apparent reason, it can happen to someone else too.

I opened an Issue and posted lots of details but there has been no response yet. My impression is that the python_script integration has a low priority.

[Mariusthvdb (Marius)]

@123

If you are still using the code discussed in the “Python_script for countdown to birthdays/etc” thread,

yes, but may script is somewhat modified and using:

date = datetime.date(dateYear,dateMonth,dateDay)

and

today = datetime.datetime.now().date()

I recall rebuilding that, and mixing it with some of Petro’s code. got to be able to find that in the community

[Troon]

Just to double-check it’s not something gone wrong with your installation, can you fire up a shell within the system? Don’t know how Supervised works, but on my DSM Docker setup, I can run a bash terminal in the HA container and fire off this:

bash-5.0# python3 -c 'import datetime; print(datetime.datetime.strptime("2021-05-21", "%Y-%m-%d").date())'
2021-05-21

I’d imagine you’d have bigger problems if strptime was AWOL across the system, though.

[123 (Taras)]

That’s not doing a date string to datetime object conversion. Your dateYear,dateMonth,dateDay variables already have the desired integer values. Anyway, I don’t want to go off-topic with this; the issue here is that strptime is normally available and I just found a case where it isn’t.

[123 (Taras)]

@Troon

run a bash terminal in the HA container and fire off this

I get the same result but I don’t believe it sheds any light on the problem because the python_script integration employs RestrictedPython. Basically python3 cut down to only “safe” things (where the development team can decide what’s considered safe or not for use within Home Assistant).