Cannot receive Discord notifications with images from /api/ or Nest

Configuration
,
1

I’m attempting to update my notification to use the Discord integration, but the service is giving me issues when pulling images from the local /api/ folder (used by integrations such as Nest). I’ve had no issue with using images from the camera snapshot storing an image in the /config/www/ folder, but I can’t see to locate if or where in the system (Home Assistant Container) the /api/ folder is pulling files.

My current notification yaml

alias: 0Test Nest
description: ""
trigger:
  - platform: device
    device_id: [REDACTED]
    domain: nest
    type: camera_motion
condition: []
action:
  - service: notify.discord
    metadata: {}
    data:
      target: "[REDACTED]"
      message: test camera
      data:
        images: 
          - >-
            /api/nest/event_media/{{ trigger.event.data.device_id }}/{{
            trigger.event.data.nest_event_id }}/thumbnail

The current error message when attempting to pull the image:

`2024-01-18 23:39:16.857 WARNING (SyncWorker_11) [homeassistant.components.discord.notify] Path not allowed: /api/nest/event_media/[REDACTED]/WyJBVlBId0V1TE5mS3cwZ05qdW91X0FEZW5OOUVSOEZUWUJ0OW9hZmZic2x4RHExT2YzZGVlZlU0RXc4YzVYMDI4UUV4SDlZeHpoYkduVUNUcG9ndk1idkJqMldaWHpnIiwgIkNpVUEydnV4cjlUWk9Rb016NHhHVlBtWXN1OHEyYXYwVG9UTk5nbjRFUkZpODZSOFRmNTNFbzBCQUczakw0VTIxY25XQjItUi1uZkF6ekt2d1VzR0I5b3ZZM01FUUN6QjZ3RXVzZ3ZYQ0JualhpSnU3LXZ5ZENhR1l5REgzRTJUSjRDM3pVVmRBM0VSREtuSUV3N1BoRnRvY0FUVWZJUFV1akNvalFlUU9IRVJ5V3Fnc1dmQjJsZ1haTEJLR1dmSy1PNHdTRDFZNzhMYUk2TXU1WG1FYVB6YUpuMmJCLTVOMHpWVWh4ZDhZanp5SWNYMUlCVU0iXQ==/thumbnail`

Per the documentation, any remote hosts will need to be in your allowlist_external_urls list. However I am pulling a file locally within the /api/ folder, unless I am mistaking how this directory works. My instance is public facing using a FQDN (i.e. https://hass.fqdn.com).

Relevant sections of my current configuration.yaml

homeassistant:
  allowlist_external_dirs:
    - /tmp

default_config:
api:

lovelace:

http:
  ssl_certificate: !secret ssl_cert_loc
  ssl_key: !secret ssl_key_loc
  use_x_forwarded_for: true
  trusted_proxies:
    - 172.0.0.0/8
    - 127.0.0.1
    - ::1

Any help on what I missed?

2

Attempting to add my public domain name to allowlist_external_urls in configuration.yaml results in a 401 error.

Updated automation:

alias: 0Test Nest
description: ""
trigger:
  - platform: device
    device_id: [REDACTED]
    domain: nest
    type: camera_motion
condition: []
action:
  - service: notify.discord
    metadata: {}
    data:
      target: "[REDACTED]"
      message: test camera
      data:
        images: 
          - >-
            https://fqdn.com/api/nest/event_media/{{ trigger.event.data.device_id }}/{{
            trigger.event.data.nest_event_id }}/thumbnail

Updated configuration.yaml

homeassistant:
  allowlist_external_dirs:
    - /tmp
  allowlist_external_urls:
    - https://fqdn.com

default_config:
api:

lovelace:

http:
  ssl_certificate: !secret ssl_cert_loc
  ssl_key: !secret ssl_key_loc
  use_x_forwarded_for: true
  trusted_proxies:
    - 172.0.0.0/8
    - 127.0.0.1
    - ::1

Error message:

2024-01-20 16:13:42.755 WARNING (MainThread) [homeassistant.components.http.ban] Login attempt or request with invalid authentication from unifi.local (192.168.1.1). Requested URL: '/api/nest/event_media/16015c01b7a49cafbfcdb5d184a8b3a1/WyJBVlBId0Vzb1hUUTY1S05RX21WZmExdmJLdmxiZlZ5SEhVYjNiQUlKR2NoMjdFOVBKRTcxblM3MzR3cWljelQyVE5fQ3JKdUJDMW5qRk5IaW1XbWUzdjMtWU5CeFpRIiwgIkNpVUEydnV4cjRvUkZaUzBDX1gwU0tUZkVEZ0ROZWsyTDhkaGZtamtyUndKQUlIc0dFWWdFb3NCQUczakw0WHVIMXc3TGFHZFd1bFpQRk5GQldSUTVYM3FBNHQ5UlVjaDVTTndKODBHTFVWMjBaeEN2dkNIT3lRQUh6RG5QaUdXUEItcy1BYkZSdlZIWExGS3pvdGxpTXdrOHpuOEkxQnlwWjNtamlzdnhCeVphbktuVEFtM01lZ0tvb2twZXJNNTMxNkdFWDZPT3dyRGw0X0UzMHlqSTlveGdpcTM1ZS1wb1BRY0JCQ2o3aVFKRmJ2alpnIl0=/thumbnail'. (HomeAssistant/2024.1.3 aiohttp/3.9.1 Python/3.11)
2024-01-20 16:13:42.772 ERROR (MainThread) [homeassistant.components.automation.0test_nest] 0Test Nest: Error executing script. Unexpected error for call_service at pos 1: 401, message='Unauthorized', url=URL('https://fqdn.com/api/nest/event_media/16015c01b7a49cafbfcdb5d184a8b3a1/WyJBVlBId0Vzb1hUUTY1S05RX21WZmExdmJLdmxiZlZ5SEhVYjNiQUlKR2NoMjdFOVBKRTcxblM3MzR3cWljelQyVE5fQ3JKdUJDMW5qRk5IaW1XbWUzdjMtWU5CeFpRIiwgIkNpVUEydnV4cjRvUkZaUzBDX1gwU0tUZkVEZ0ROZWsyTDhkaGZtamtyUndKQUlIc0dFWWdFb3NCQUczakw0WHVIMXc3TGFHZFd1bFpQRk5GQldSUTVYM3FBNHQ5UlVjaDVTTndKODBHTFVWMjBaeEN2dkNIT3lRQUh6RG5QaUdXUEItcy1BYkZSdlZIWExGS3pvdGxpTXdrOHpuOEkxQnlwWjNtamlzdnhCeVphbktuVEFtM01lZ0tvb2twZXJNNTMxNkdFWDZPT3dyRGw0X0UzMHlqSTlveGdpcTM1ZS1wb1BRY0JCQ2o3aVFKRmJ2alpnIl0=/thumbnail')
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 476, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 713, in _async_call_service_step
    response_data = await self._async_run_long_action(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 675, in _async_run_long_action
    return long_task.result()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2149, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2186, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/notify/legacy.py", line 265, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/discord/notify.py", line 167, in async_send_message
    file = await self.async_get_file_from_url(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/discord/notify.py", line 80, in async_get_file_from_url
    async with session.get(
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 689, in _request
    resp.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1059, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url=URL('https://fqdn.com/api/nest/event_media/16015c01b7a49cafbfcdb5d184a8b3a1/WyJBVlBId0Vzb1hUUTY1S05RX21WZmExdmJLdmxiZlZ5SEhVYjNiQUlKR2NoMjdFOVBKRTcxblM3MzR3cWljelQyVE5fQ3JKdUJDMW5qRk5IaW1XbWUzdjMtWU5CeFpRIiwgIkNpVUEydnV4cjRvUkZaUzBDX1gwU0tUZkVEZ0ROZWsyTDhkaGZtamtyUndKQUlIc0dFWWdFb3NCQUczakw0WHVIMXc3TGFHZFd1bFpQRk5GQldSUTVYM3FBNHQ5UlVjaDVTTndKODBHTFVWMjBaeEN2dkNIT3lRQUh6RG5QaUdXUEItcy1BYkZSdlZIWExGS3pvdGxpTXdrOHpuOEkxQnlwWjNtamlzdnhCeVphbktuVEFtM01lZ0tvb2twZXJNNTMxNkdFWDZPT3dyRGw0X0UzMHlqSTlveGdpcTM1ZS1wb1BRY0JCQ2o3aVFKRmJ2alpnIl0=/thumbnail')
2024-01-20 16:13:42.794 ERROR (MainThread) [homeassistant.components.automation.0test_nest] While executing automation automation.0test_nest
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/automation/__init__.py", line 669, in async_trigger
    await self.action_script.async_run(
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1587, in async_run
    return await asyncio.shield(run.async_run())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 426, in async_run
    await self._async_step(log_exceptions=False)
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 479, in _async_step
    self._handle_exception(
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 502, in _handle_exception
    raise exception
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 476, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 713, in _async_call_service_step
    response_data = await self._async_run_long_action(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 675, in _async_run_long_action
    return long_task.result()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2149, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2186, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/notify/legacy.py", line 265, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/usr/src/homeassistant/homeassistant/components/discord/notify.py", line 167, in async_send_message
    file = await self.async_get_file_from_url(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/discord/notify.py", line 80, in async_get_file_from_url
    async with session.get(
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1187, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 689, in _request
    resp.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1059, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url=URL('https://fqdn.com/api/nest/event_media/16015c01b7a49cafbfcdb5d184a8b3a1/WyJBVlBId0Vzb1hUUTY1S05RX21WZmExdmJLdmxiZlZ5SEhVYjNiQUlKR2NoMjdFOVBKRTcxblM3MzR3cWljelQyVE5fQ3JKdUJDMW5qRk5IaW1XbWUzdjMtWU5CeFpRIiwgIkNpVUEydnV4cjRvUkZaUzBDX1gwU0tUZkVEZ0ROZWsyTDhkaGZtamtyUndKQUlIc0dFWWdFb3NCQUczakw0WHVIMXc3TGFHZFd1bFpQRk5GQldSUTVYM3FBNHQ5UlVjaDVTTndKODBHTFVWMjBaeEN2dkNIT3lRQUh6RG5QaUdXUEItcy1BYkZSdlZIWExGS3pvdGxpTXdrOHpuOEkxQnlwWjNtamlzdnhCeVphbktuVEFtM01lZ0tvb2twZXJNNTMxNkdFWDZPT3dyRGw0X0UzMHlqSTlveGdpcTM1ZS1wb1BRY0JCQ2o3aVFKRmJ2alpnIl0=/thumbnail')
3

For comparison, here is the same automation using the Home Assistant app as the notifier instead with the action working successfully (an image is attached and sent):

alias: 0Test Nest
description: ""
trigger:
  - platform: device
    device_id: [REDACTED]
    domain: nest
    type: camera_motion
condition: []
action:
  - service: notify.mobile_app_peter
    data:
      data:
        image: >-
          /api/nest/event_media/{{ trigger.event.data.device_id }}/{{
          trigger.event.data.nest_event_id }}/thumbnail
      message: test camera
4

I finally got this working with a “likely” hacky workaround. I couldn’t rely on “camera.snapshot” as by the time the event was noticed, the snapshot would show no one at the door if they were quick. I wanted to use the person detection image directly to upload to google_generative_ai_conversation so it would stop saying “I see a tree and some made up animals that arent really there”

Step 1 - Create a new MQTT template sensor to receive the generated /api/event/blahblah url from the nest event.

- name: "Nest MQTT Person Url"
  state_topic: "home/nest/last_person_url"
  value_template: "{{ value_json.timestamp }}"
  json_attributes_topic: "home/nest/last_person_url"

The annoying thing is that sensor states are limited to 255 characters…so I had to use an attribute (which seem to have no limit). I set the sensor value to show the timestamp instead.

Step 2 - Have the person detection event instead just publish to MQTT

alias: Nest Person MQTT Publish
description: publishes the internal url of the nest snapshot
trigger:
  - platform: device
    device_id: xxxxxxxxxxxxxxxx
    domain: nest
    type: camera_person
condition: []
action:
  - service: mqtt.publish
    metadata: {}
    data:
      qos: 0
      retain: true
      topic: home/nest/last_person_url
      payload: |-
        { 
          "url": "api/nest/event_media/{{ trigger.event.data.device_id }}/{{ trigger.event.data.nest_event_id }}",
          "timestamp": "{{trigger.event.data.timestamp}}"
        }
mode: single

Step 3 - Create a ‘generic camera’ object with a template URL. The camera doesn’t allow for a relative URL ‘i.e. /api/event/blah’, so I put my local HA ip address there (http://127.0.0.1 likely would have worked too)

For the ‘still image URL’, I use the ‘url’ attribute of my MQTT sensor
http://192.168.0.65:8123/{{ state_attr('sensor.nest_mqtt_person_url', 'url') }}
Set RTSP as ‘HTTP’ and basic auth and no SSL cert verification.

Annoyingly, this has to resolve to a valid image before it lets you commit this (via the UI), so I had to pre-publish a valid event. Alternatively, pure YAML probably doesn’t care if it’s valid or not and would just work.

Step 4 - Have your person detection events trigger off this new MQTT sensor instead of the nest device. In step 1 - I have it create a snapshot from my ‘generic camera’…which had its url updated from the mqtt publish, so the snapshot should always be the latest image.

alias: Doorbell Google AI
description: ""
trigger:
  - platform: state
    entity_id:
      - sensor.nest_mqtt_person_url
    attribute: url
condition: []
action:
  - service: camera.snapshot
    metadata: {}
    data:
      filename: /media/images/nest_person.jpg
    target:
      device_id: xxxxxxxxxxxxxxxxx
  - service: google_generative_ai_conversation.generate_content
    data:
      prompt: >-
        Very briefly describe what you see in this image from my doorbell
        camera. Your message needs to be short to fit in a phone notification.
        Don't describe stationary objects or buildings or trees.
      image_filename: /media/images/nest_person.jpg
    response_variable: generated_content
  - service: notify.discord_doorbell
    metadata: {}
    data:
      message: Front Door AI {{ generated_content.text }}
      data:
        image: /media/images/nest_person.jpg
        images:
          - /media/images/nest_person.jpg
mode: single

Hope this helps someone!

EDIT: Well crap…the /api url generated from the event doesn’t seem to work. Just get 401 unauthorized. When I was testing, I’d go into my media folder and find the nest image that was recent and just copied the url from there. No idea why it never seems to match the event trigger urls