Can't get response_variable when stop is in nested block of script

Configuration
,
1

Wonderful idea returning values from scripts. Here’s what I want to do:

I have an automation that performs my early morning wakeup routing. One of the many things required is to ramp up the light intensity. To accomplish that I wrote a script just for the rampup and I call the script (service script. blocking call) from my automation. Got that part working just fine.

If someone “controls” the lamp (turn off or change intensity) partway through the rampup, the script can easily determine that. I want to terminate the script early and let the automation know that there was manual intervention so it can alter the wakeup routine. Perfect application for response variable in the script. Couldn’t ask for anything better.

So I tried returning a response variable (of success) at the end of the script and it was easily seen (and used) in the application. A code snippet from the script would be:

ramp_up_light:
  sequence:
  - repeat:
      count: 10
      sequence:
.
<code for the ramping>.
.
  - variables:
      return_var:
        success: true
        reason: "Successful completion"
  - stop: "Complete"
    response_variable: return_var

Great start. Now to put the “manual intervention” stop code in as follows:

ramp_up_light:
  sequence:
  - repeat:
      count: 10
      sequence:
.
<code for the ramping>.
      - if: <test for manual intervention>
        then:
        - variables:
            return_var:
              success: false
              reason: "Manual intervention"
        - stop: "Incomplete"
          response_variable: return_var
.
  - variables:
      return_var:
        success: true
        reason: "Successful completion"
  - stop: "Complete"
    response_variable: return_var

OK, so this continues to work perfectly in the absence of manual intervention. If there is manual intervention, though, it returns {} (empty dict? None?) as shown in the automation trace “changed variables”.

So I am guessing it is a scope issue that the intervention “stop” is nested a block or two down from the other “stop” which is at the top level of the script. Is stop only looking for variables in the top level?

So I tried moving the four manual intervention lines starting with “-variables:” up to the top before “repeat” (i.e. the default return_var is the failure one). No change. I’m really hoping I’ve missed something because right now it looks like you can’t use response_variable from anything other than the top scope.

An alternative is to only use stop at the top level but how would I pass the failure reason from a nested scope up to the top level scope?

Any help would be very much appreciated. I would accept an alternative means of doing this (preferably without using “input” entities as I find that poor practice) but I would really like to understand what “stop” is doing that prevents me from using a response_variable while nested.

TIA
Keith

2

Correct.

The variable return_var defined inside the if block is undefined outside of the if statement.

  • The scope of variables defined at the main level is the entire script.
  • The scope of variables defined inside of choose, if, etc are undefined elsewhere.

If you define a variable named x at the main level and then define another variable with the same name x inside an if block, you’re actually creating a second x variable (with limited scope) and not redefining the first x variable.

3

@123 Thank you for your reply. I totally understand scope as you mentioned it but that does not explain this behaviour.

It would be my understanding that the main level is the first sequence: block. In my first try at reporting failure (the second code snippet), I declared the variable with final values inside an if: block inside a repeat: block but I “used” the variable at the same level (with the response_variable:) and it failed. So I assumed that the “use” of the variable by the system was more global (ultimately sending the value back to the calling automation) so as I mentioned, I moved the declaration with final values (NOT copied, moved) to the main block (right after sequence:). It failed.

Note that from a scope point of view, the variable is the same as in the test case (the first code snippet). The only difference I can see is that the “use” of the variable (i.e. the response_variable:) is nested inside two other blocks. According to my understanding of scope (and yours as stated), the variable should be the entire script yet it fails.

I think there may be some issue with response_variable. Perhaps I must only do that at the top level. Gonna try a few things…

You can see why I had to go to the community for help here!

4

That so-called “only difference” is all about scope.

I believe the behavior of the examples posted above conforms to Home Assistant’s scoping rules. If you have questions about other examples, post their code.

5

I totally understand that. That is why I moved the definition of return_var to the top level and REMOVED it from the double nested level. Though I described that, I should have provided the code snippet:

ramp_up_light:
  sequence:
  - variables:
      return_var:
        success: false
        reason: "Manual intervention"
  - repeat:
      count: 10
      sequence:
.
<code for the ramping>.
      - if: <test for manual intervention>
        then:
        - stop: "Incomplete"
          response_variable: return_var
.
  - variables:
      return_var:
        success: true
        reason: "Successful completion"
  - stop: "Complete"
    response_variable: return_var

Now the return_var is defined (then redefined if it makes it that far) with top level scope only and therefore should be accessible to the entire code. Still I get no value in the calling automation for the nested stop:.

I’m sure I’m just misunderstanding what you are explaining. All I want to do is to to send back a return_var that has (conditionally) one of two values.

6

I agree with you. The outcome suggests that even if a variable with global scope is assigned to response_variable inside the if block it doesn’t persist outside of it. That’s not what I would expect.

7

Crap. Many, many thanks for your help @123, though. The one shining light in my sad situation.

Right now that leaves me thinking there is only one very, very poor solution: a helper. Very poor because it is not parallel-use capable, helpers weren’t meant for this etc. etc.

I think this is all related to the big problem that variable declaration is the only way to set variables. If the writing of variables was separated from the declaration, we could pass data up in the block nesting heirarchy by defining higher up and writing lower down without redeclaration.

Note that the use of response_variable: with stop: is the exact opposite of using response_variable: for a service call. In the case of stop:, the script IS the “service” and response_variable: is specifying the SOURCE of the data. In the case of using response_variable: for a service call (e.g. in the many calendar examples) it is specifying the DESTINATION. Documentation is actually quite weak here. Don’t know if, in a script, response_variable: is tied to stop “action” (service?) or is simply a top level key pair. I could check by removing the final stop: line and see if I get the return values in my automation. I could try many other things…

8

You would need to use namespaces if you want to use the variable outside the scope.

Scoping Behavior

Please keep in mind that it is not possible to set variables inside a block and have them show up outside of it. This also applies to loops. The only exception to that rule are if statements > which do not introduce a scope. As a result the following template is not going to do what you might expect:>

{% set iterated = false %}
{% for item in seq %}
{{ item }}
{% set iterated = true %}
{% endfor %}
{% if not iterated %} did not iterate {% endif %}

It is not possible with Jinja syntax to do this. Instead use alternative constructs like the loop else block or the special loop variable:

{% for item in seq %}
{{ item }}
{% else %}
did not iterate
{% endfor %}

As of version 2.10 more complex use cases can be handled using namespace objects which allow propagating of changes across scopes:

{% set ns = namespace(found=false) %}
{% for item in items %}
{% if item.check_something() %}
{% set ns.found = true %}
{% endif %}
* {{ item.title }}
{% endfor %}
Found item having something: {{ ns.found }}

Note that the obj.attr notation in the set tag is only allowed for namespace objects; attempting to assign an attribute on any other object will raise an exception.

Via: Template Designer Documentation — Jinja Documentation (3.2.x)

9

Let us know the results of your experiments because the situation you’re facing is likely to come up more often in the community forum.

1 Like
10

Will do. I really think a developer would have the best insight. It seems that more is going on than we know like stop:/response_variable: defining a variable at its current scope that must be top level scope perhaps.

I have a perfect workaround in my case: I only have two possible outcomes and I have control over one (success) and the system always returns the same value for the other (None?). So I can actually test in the automation for success and assume failure otherwise. Works for now but if I have to identify what failure or where it failed I’m screwed!

Thank you so much for your time.

11

I’m getting a similar issue here, and I’m pretty sure it’s not a scope issue.

I have a script that sends a Telegram message and expects a Action or No Action response. I have a global variable defined at the top level of the script to store the script’s outcome. Here’s a truncated version of the script to demonstrate

alias: Ask home to close windows
# ...
variables:
  result:
    no_action: false
# ...
sequence:
  - service: telegram_bot.send_message
    alias: Ask for user input on Telegram
    data:
# ...
  - wait_for_trigger:
# ...
  - choose:
# ...
    default:
      - stop: User wants manual_mode to stay on
        response_variable: result
  - variables:
      result:
        no_action: true
  - stop: Done
    response_variable: result

If I let the script play out such that the final stop: Done is reached, I get a response with:

no_action: true

If I force it to enter the default block where the early stop is, I get a response with

Nothing at all… even if I add a similar variables definition right before the stop at the same indent level, I get nothing. Also tried using a different variable altogether so that it doesn’t collide with the global one, nothing.

It’s not a scope issue, the stop command just literally doesn’t seem to pass anything as a response no matter what if it’s in a nested block.

4 Likes
12

Yup. That’s the conclusion some of us have already come up with. Gotta say it’s a bug. Time to get @balloob or some other developer to help out…

See this as another thread of the same problem

1 Like
13

I suggested a workaround in my post you refered to. I welcome all opinions and feedbacks.

14

Yes. Very creative workaround indeed.

It is such a shame that the result variable doesn’t work as we expect because it means it is useful only in a script that doesn’t conditionally return information which many, if not most, do.

@baloob I stress that this should be fixed and if that is not to be soon, at least document the inability to return data where a stop is in nested scope.

6 Likes