Using Response variable in a script

I’m trying to make some AI camera checking automations to automate some of those difficult tasks but I’m completely stumped trying to use response variables in templates within my automation sequence.

I have first created a script to evaluate an image using Google generative AI

alias: Evaluate Camera Image
sequence:
  - variables:
      filename: /config/www/tmp/snapshot-{{ camera.lstrip('camera.') }}.jpg
  - service: camera.snapshot
    data:
      filename: "{{ filename }}"
    target:
      entity_id: "{{ camera }}"
  - service: google_generative_ai_conversation.generate_content
    metadata: {}
    data:
      prompt: "{{ prompt }}"
      image_filename: "{{ filename }}"
    response_variable: response
  - stop: Stop running the rest of the sequence
    response_variable: response
fields:
  camera:
    selector:
      entity:
        domain: camera
    name: Camera
    required: true
  prompt:
    selector:
      text:
        multiline: true
    name: Prompt
    required: true

This sequence works reliably. However, I can’t seem to figure out how to use this script within other automations.

An example script that isn’t working

alias: Check for Truck
description: ""
trigger:
  - platform: time
    at: "07:00:00"
condition: []
action:
  - service: script.evaluate_camera_image
    metadata: {}
    data:
      camera: camera.garage
      prompt: >-
        This image is of a garage. Is there a white truck parked in the garage? 
        If yes, say "true"
        Otherwise, say "false"
    response_variable: ai_response

  - variables:
      result: |
        {{ {'value': ai_response} }}    
  - if:
      - condition: template
        value_template: "{{ result }}"
    then:
      - service: input_boolean.turn_on
        target:
          entity_id: input_boolean.truck_in_garage
    else:
      - service: input_boolean.turn_off
        target:
          entity_id:
            - input_boolean.truck_in_garage
mode: single

I think I defined the variable correctly, and when I use a notification to test the result I appear to be getting the correct result from both {{ result }} and {{ result.value }} Is anyone able to help me with the value_template portion of the If block?

the issue you are hitting is that when you define a variable, it is evaluated and defined at the start of the block where you defined it. not at the place in a sequence where you defined it.

so in your case, the variable is being defined at the start of your action block… before your script.evaluate_camera_image.

does this work for you? i think sequence will define a new block such that your variable can be defined and evaluated at the top of that new block.

alias: Check for Truck
description: ""
trigger:
  - platform: time
    at: "07:00:00"
condition: []
action:
  - service: script.evaluate_camera_image
    metadata: {}
    data:
      camera: camera.garage
      prompt: >-
        This image is of a garage. Is there a white truck parked in the garage? 
        If yes, say "true" Otherwise, say "false"
    response_variable: ai_response
  - sequence:
      - variables:
          result: |
            {{ {'value': ai_response} }}    
      - if:
          - condition: template
            value_template: "{{ result }}"
        then:
          - service: input_boolean.turn_on
            target:
              entity_id: input_boolean.truck_in_garage
            data: {}
        else:
          - service: input_boolean.turn_off
            target:
              entity_id:
                - input_boolean.truck_in_garage
            data: {}
mode: single

i think that should get the variable defined in the scope you want. i’m not sure it will end up doing what you want it to do (maybe it will)… i think you’re defining a string that looks like a dictionary but is still actually a string… i think… but maybe that’s what you intend.

regardless, the problem i think you’re hitting at this moment is the scope and sequence of the variable definition.

Ok that makes a lot of sense, and I didn’t realize that was how variables were defined. However, I have added that sequence step and I still seem to be getting the same result. I added the following step at the end of my sequence so I can easily see what is happening to the variable:

service: persistent_notification.create
data:
  message: "{{ result }}"

I am consistently getting the right answer reported here, but the if statement value template is failing every time. I have been known to mix up booleans, dictionaries, strings, etc in the past so maybe that’s where I’m screwing up. I have tried the following and nothing seems to work:

value_template: "{{ result }}"

value_template: "{{ result.value }}"

value_template: "{{ result == 'true' }}"

value_template: "{{ result.value == 'true' }}"

value_template: "{{ ai_response }}"

value_template: "{{ ai_response == 'true' }}"

could you

  1. post your latest code.
  2. post what you get on the new notification message. ie, what’s the {{ result }} you see?
  3. what are you intending with that condition statement? what is it you want to check?

btw… as i said above, i think you’re getting a “string” of something like

"{{ {'value': 'true'} }}"

so your template might need to be something like

value_template: > 
  {{ result == "{{ {'value': 'true'} }}" }}

or something like that. but the quotes are making my head hurt so i’m sure i got that partially wrong… but hopefully you get the idea.

still i don’t think this is actually what you want. so give me those 3 answers above and let’s see if we can get this solved correctly.

So as I worked to answer your questions, I think I discovered what was happening

Ok taking a step back… I removed the AI portion of the code until I figure out how to use service responses in my automation. I replaced it with a “test script” who’s only job is to return whatever text what provided. This is speeding up my testing as I don’t have to wait for the AI to review the image.

The code for the test script is here.

alias: Test Responding Script
sequence:
  - variables:
      output: |
        {{ {'prompt': prompt} }}    
  - service: script.send_notification
    data:
      destination:
        - Persistant
      message: "{{ output.prompt }}"
  - stop: returning value 
    response_variable: output
description: ""
fields:
  prompt:
    selector:
      text: {}
    name: Prompt
    required: true

So now back to your question:

  1. post your latest code.
alias: Check for Truck
description: ""
trigger:
  - platform: time
    at: "07:00:00"
condition: []
action:
  - service: script.test_responding_script
    metadata: {}
    data:
      prompt: "true"
    response_variable: ai_response
  - sequence:
      - variables:
          result: |
            {{ {'value': ai_response} }}    
      - if:
          - condition: template
            value_template: "{{ result }}"
        then:
          - service: input_boolean.turn_on
            target:
              entity_id: input_boolean.truck_in_garage
        else:
          - service: input_boolean.turn_off
            target:
              entity_id: input_boolean.truck_in_garage
            data: {}
        enabled: true
      - service: script.send_notification
        metadata: {}
        data:
          destination:
            - Persistant
          message: "{{ result }}"
mode: single
  1. post what you get on the new notification message. ie, what’s the {{ result }} you see?

If the message field is “{{ result }}”, I get true

image

If the message field contains anything else, such as message: “result: {{ result }}”, I get result: {‘value’: {‘prompt’: ‘true’}}

image

!!This above is where I got a clue of where I was going wrong. I seem to have nested dictionary keys (or something) and I’m not sure the cleanest way to fix this. I’m clearly overcomplicating things somewhere.

Now, if change my conditional value template to {{ result.value.prompt }}, it all works as expected.

  1. what are you intending with that condition statement? what is it you want to check?

All I want is to use the service response (true or false) to turn a boolean on or off. Once I understand how this works I can use this logic to create a template sensor instead of an automation/boolean. I don’t frankly care how it works, just trying to understand an efficient way to do it

oh… then this?
(make sure “true”) matches the capitalization you get out.
screw the extra variable…

alias: Check for Truck
description: ""
trigger:
  - platform: time
    at: "07:00:00"
condition: []
action:
  - service: script.evaluate_camera_image
    metadata: {}
    data:
      camera: camera.garage
      prompt: >-
        This image is of a garage. Is there a white truck parked in the garage? 
        If yes, say "true"
        Otherwise, say "false"
    response_variable: ai_response
  - if:
      - condition: template
        value_template: "{{ ai_response == 'true' }}"
    then:
      - service: input_boolean.turn_on
        target:
          entity_id: input_boolean.truck_in_garage
    else:
      - service: input_boolean.turn_off
        target:
          entity_id:
            - input_boolean.truck_in_garage
mode: single
1 Like

Well, that was where I originally started, and it didn’t work. Reading up on response variables is what set me down the whole variable definition rabbit hole to begin with. I have now figured out I didn’t need to add a dictionary key/value, I just needed to correctly reference the one that was created.

You’re right though, the extra variable is completely pointless.

I changed my value template to the below and it worked perfectly.
value_template: "{{ ai_response.prompt}}"

When I plugged my AI script back in, I had to do a little debugging before landing on the below solution that works with my AI script. Not clear to me how/where the “text” key is coming from but that seems to be the one containing the true/false.
value_template: "{{ ai_response.text }}"

My new working script is:

alias: Check for Truck
description: ""
trigger:
  - platform: time
    at: "07:00:00"
condition: []
action:
  - service: script.evaluate_camera_image
    metadata: {}
    data:
      camera: camera.garage
      prompt: >-
        This image is of a garage. Is there a white truck parked in the garage? 
        If yes, say "true"
        Otherwise, say "false"
    response_variable: ai_response
  - if:
      - condition: template
        value_template: "{{ ai_response.text }}"
    then:
      - service: input_boolean.turn_on
        target:
          entity_id: input_boolean.truck_in_garage
    else:
      - service: input_boolean.turn_off
        target:
          entity_id: input_boolean.truck_in_garage
    enabled: true
mode: single

Thank you for your help! I definitely needed some extra eyes on this one

ah, duh… need to get the .prompt member.
regardless, glad that it’s working for you now!