AI camera analysis - extended

Hi there,

I used Paulus Schoutsens initial blueprint and extended it with an output and the possibility to have a complete custom promt to ask to the AI.

For me it worked fine with the Google Generative AI Integration (not yet with the open router integration): Google Generative AI - Home Assistant

This blueprint will gonna be extended more and more - so input is welcome :slight_smile:

Open your Home Assistant instance and show the blueprint import dialog with a specific blueprint pre-filled.

blueprint:
  name: AI camera analysis
  description: >-
    📷 ⚙️ v0.15 <br>
    Analyze camera footage with AI when motion is detected.
    This is an extended version, of Paulus Schoutsens intitial blueprint to analyze camera footage with AI.
  domain: automation
  author: Marcels Custom Shop & Paulus Schoutsen
  input:
    result_target:
      name: "Result Helper (Note: 255 characters max!)"
      selector:
        entity:
          domain: input_text
    motion_entity:
      name: Motion Sensor
      selector:
        entity:
          filter:
            device_class: motion
            domain: binary_sensor
    camera_target:
      name: Camera
      selector:
        entity:
          domain: camera
    extra_instructions:
      name: Your promt here
      description: >
        Ask the AI for what you want to know about the image.
      selector:
        text: null
      default: ""
    analysis_delay:
      name: Delay before analysis
      description: Time to wait before analyzing the camera after motion is detected.
      default: 0
      selector:
        number:
          min: 0
          max: 3600
          unit_of_measurement: seconds
    cooldown_time:
      name: Cooldown
      description: Time to wait between analyses.
      default: 60
      selector:
        number:
          min: 0
          max: 3600
          unit_of_measurement: seconds


mode: single
max_exceeded: silent

triggers:
  - trigger: state
    entity_id:
      - !input motion_entity
    from: "off"
    to: "on"

actions:
  - variables:
      camera_entity: !input camera_target
      extra_instructions: !input extra_instructions

  - delay:
      seconds: !input analysis_delay
  - alias: Analyse camera image
    action: ai_task.generate_data
    data:
      task_name: "{{ this.entity_id }}"
      instructions: >
        {{ extra_instructions }}
      structure:
        analysis:
          selector:
            text: null
      attachments:
        media_content_id: "media-source://camera/{{ camera_entity }}"
        media_content_type: ""
    response_variable: result
  - alias: Write analysis to input_text
    service: input_text.set_value
    target:
      entity_id: !input result_target
    data:
      value: "{{ result.data.analysis }}"

  - delay:
      seconds: !input cooldown_time
4 Likes

I also did a YouTube Video on my Channel (GERMAN) about and how to use this :slight_smile:

2 Likes

Hallo Marcel ich hab mir dein Video angeschaut und es genauso nachgemacht. Leider füllt sich der Helper nicht und wenn ich auf das System → Protokoll gehe sehe den output als error info was eigentlich im helper sein sollte. Hast du noch einen Tip?
Zweitens wie würde denn der Output auf einem Homepod gehen wie du in deinem Video es beschreibst?
Drittens :slight_smile: Wie würde den das auch als IOS notification aussehen oder wie würde das gehen?
Und viertens :slight_smile: :slight_smile: es gibt ja noch die LLM Vision Timeline card, die man eventuell auch füttern könnte…
Ideen über Ideen aber der Ansatz ist schon hervorragend :slight_smile:

@efkay How to help us help you - or How to ask a good question Please use English only.

3 Likes

Thanks! Are you sure, your output is not longer than 255 characters? Maybe put “limit to 10 words maximum please” at the end of your promt.
About voice-output and notifications, I’ll do another video :slight_smile:

1 Like

Yes Thank you! That was the trick :slight_smile: Now it works :slight_smile: I’m definitely curious about your next YT video to enlarge the possibilities. Already subcribed to your YT Channel :slight_smile: