Nest image snapshot on Doorbell ring --> Google AI --> Alexa TTS

Configuration
,
1

Example of sending a Nest doorbell-triggered webcam snapshot to Google AI for summarization. The image description response is announced over Alexa TTS.

alias: Doorbell Ring Google AI will describe
description: When doorbell rings, Google AI will summarize what it sees.
trigger:
  - platform: device
    device_id: abc123
    domain: nest
    type: doorbell_chime
action:
  - service: camera.snapshot
    data:
      filename: /media/nest/screenshot/front_door.jpg
    target:
      entity_id: camera.front_door
  - service: google_generative_ai_conversation.generate_content
    data:
      prompt: Very briefly describe the people in this image. Only mention people.
      image_filename: /media/nest/screenshot/front_door.jpg
    response_variable: generated_content
  - service: notify.alexa_media_tube
    data:
      message: "{{ generated_content.text }}"
  - service: notify.all_mobile_devices
    data:
      title: Doorbell ring
      message: "{{ generated_content.text }}"
mode: single

ha notif

3 Likes
2

Hi, I’m a noob and just started with HA. I’ve two questions:

  • Should this code be put in configuration.yaml?
  • Is it possible to attach the picture to the notification message?
3

You can write the automation in the Automations section. The raw YAML of my automation is below. You will need to tinker for your specific setup.

Not sure about including an image in a device notification. Could be possible.

alias: Doorbell ring describe via Google AI
description: When doorbell rings, Google AI will summarize what it sees.
triggers:
  - device_id: YOUR_DOORBELL
    domain: nest
    type: doorbell_chime
    trigger: device
conditions: []
actions:
  - data:
      filename: /media/nest/screenshot/front_door.jpg
    target:
      entity_id: camera.front_door
    action: camera.snapshot
  - data:
      filenames: /media/nest/screenshot/front_door.jpg
      prompt: briefly describe any people you see
    response_variable: generated_content
    action: google_generative_ai_conversation.generate_content
  - data:
      message: "{{ generated_content.text }}"
    action: notify.alexa_media_tube
  - data:
      message: "{{ generated_content.text }}"
    action: notify.alexa_media_kip_s_echo
  - data:
      title: Doorbell ring
      message: "{{ generated_content.text }}"
    action: notify.all_mobile_devices
mode: single