Displaying Birdnet-go detections

Birdnet-go has just added mqtt reporting of the last detection. It publishes this data to the birdnet topic:

{
  "ID": 0,
  "SourceNode": "BirdNET-Go",
  "Date": "2024-04-06",
  "Time": "08:23:51",
  "InputFile": "",
  "BeginTime": "2024-04-06T08:23:48.736132138+11:00",
  "EndTime": "0001-01-01T00:00:00Z",
  "SpeciesCode": "railor5",
  "ScientificName": "Trichoglossus moluccanus",
  "CommonName": "Rainbow Lorikeet",
  "Confidence": 0.8614452481269836,
  "Latitude": <redacted>,
  "Longitude": <redacted>,
  "Threshold": 0.8,
  "Sensitivity": 1,
  "ClipName": "clips/2024/04/trichoglossus_moluccanus_86p_20240406T082353Z.wav",
  "Comment": "",
  "ProcessingTime": 598766979,
  "Results": null
}

Each new detection overwrites the last.

I would like to display the detections for the day in a table by time like this:

Time, CommonName, Confidence

@petro suggested auto entities, but I’m not sure how to do this. Even with his configuration as it seems to rely on a sensor with an event attribute:

type: custom:auto-entities
    card:
      type: vertical-stack
    card_param: cards
    filter:
      template: >
        {% set category = states('input_select.event_category').lower() %}
        {% if category != 'all' %}
          {{ state_attr('sensor.xx', 'events') | map(attribute='card') | selectattr('category', 'eq', category) | list or [] }}
        {% else %}
          {{ state_attr('sensor.xx', 'events') | map(attribute='card') | list or [] }}
        {% endif %}

The new webpage dashboard works while I am at home (both HA and Birdnet are HTTP) but not when I am away as then I access HA via HTTPS.

Any other ideas?

First, you need to decide how to store the data for the day. Typically I use MQTT to store a list inside an attribute. @TheFes typically uses template sensors with attributes.

This is what I use for MQTT discovery and state updates:

mqtt_automated_states:
  alias: Publish State and Attributes
  mode: parallel
  fields: 
    <<: &mqtt-fields
      domain:
        description: The entities domain
        selector:
          text:
            type: text
      unique_id:
        description: The entities unique_id
        selector:
          text:
            type: text
      object_id:
        description: The entities object_id
        selector:
          text:
            type: text
    state:
      description: The entities state
      selector:
        text:
          type: text
    attributes:
      description: The entities attributes
      example: A dictionary {} in yaml
      selector:
        object:
  variables: 
    <<: &mqtt-variables
      root: "homeassistant"
      topic_root: >
        {%- if domain is not defined or unique_id is not defined %}
          {{- [ root, 'error'] | join('/') }}
        {%- else %}
          {{- [ root, domain, unique_id ] | join('/') }}
        {%- endif %}
    service_data: >
      {{ {
        'topic': topic_root ~ '/state',
        'payload': '' ~ { 'state': state, 'attributes': attributes | default({}) } | tojson,
        'retain': retain | default(True)
      } }}

  sequence:
  - service: mqtt.publish
    data: "{{ service_data }}"

mqtt_automated_config:
  alias: Publish Discovery
  mode: parallel
  fields: 
    <<: *mqtt-fields
    device_class: 
      description: The entities device class
      selector:
        text:
          type: text
  variables:
    name: >
      {% if object_id is defined %}
        {{ object_id | default('') | replace('_', ' ') | title }}
      {% elif unique_id is defined %}
        {{ unique_id | default('') | replace('_', ' ') | title }}
      {% else %}
        Unknown Entity
      {% endif %}
    <<: *mqtt-variables
    service_data: >
      {%- set items = [
        ("name", name),
        ("unique_id", unique_id | default(none)),
        ("object_id", object_id | default(none)),
        ("state_topic", topic_root ~ "/state"),
        ("value_template", "{{ value_json.state }}"),
        ("json_attributes_topic", topic_root ~ "/state"),
        ("json_attributes_template", "{{ value_json.attributes | tojson }}"),
        ("device_class", device_class | default(none) ),
        ("unit_of_measurement", unit_of_measurement | default(none) ),
        ("state_class", state_class | default(none)),
        ("device", device | default(none))
      ] %}
      {% set payload = dict.from_keys(items | rejectattr('1', 'none') | list) %}
      {{ {
        'topic': topic_root ~ '/config',
        'payload': '' ~ payload | tojson,
      } }}
  sequence:
  - service: mqtt.publish
    data: "{{ service_data }}"

Then with an automation, make the entity taht will store the info

- alias: Birdnet List
  trigger:
  - id: reset
    platform: time
    at: "00:00:00"
  - id: discovery
    platform: homeassistant
    event: start
  - id: update
    platform: mqtt
    topic: topic/of/source/birdnet/data
  variables:
    <<: &common
      domain: sensor
      object_id: birdnet_list
    
  action:
  - choose:
    - conditions:
      - platform: template
        value_template: "{{ trigger.id == 'reset' }}"
      sequence:
      - service: script.mqtt_automated_states
        data:
          <<: *common
          state: "{{ today_at() }}"
          attributes: >
            {{ {'items': []} }}
    - conditions:
      - platform: template
        value_template: "{{ trigger.id == 'discovery' }}"
      sequence:
      - service: script.mqtt_automated_config
        data:
          <<: *common
          name: List
          device_class: timestamp
          device:
            name: Birdnet
            identifiers:
            - 0fff6
            manufacturer: Tom
            model: Birdnet List
            sw_version: "1.0"
    - conditions:
      - platform: template
        value_template: "{{ trigger.id == 'update' }}"
      sequence:
      - service: script.mqtt_automated_states
        data:
          <<: *common
          state: "{{ now() }}"
          attributes: >
            {% set current = states(domain ~ '.' ~ object_id, 'items') or [] %}
            {{ (current + [{
              'datetime': trigger.payload_json.BeginTime,
              'name': trigger.payload_json.CommonName,
              'confidence': trigger.payload_json.Confidence,
            }]) | sort(attribute='datetime', reverse=True) }}

This automation will create discovery on startup for a sensor, it will reset the sensors data at midnight, and it will update the sensors state when your mqtt source topic updates.

Next, if you just want a table of this in markdown:

type: markdown
content: >
  Time|CommonName|Confidence
  :---:|:---:|:---:
  {%- set t = now() %}
  {%- for bird in states('sensor.birdnet_list','items') or [] %}
  {{ relative_time(bird.datetime | as_datetime) }} | {{bird.name}} | {{ bird.confidence }}
  {%- endfor %}

If you want fancy cards, you need to decide what card to use. Above I used mushroom I think?

    type: custom:auto-entities
    card:
      type: vertical-stack
    card_param: cards
    filter:
      template: >
       [{%- set t = now() %}
       {%- for bird in states('sensor.birdnet_list','items') or [] %}
       {
         "type": "custom:mushroom-template-card",
         "primary": "{{ bird.name }}",
         "secondary": "Confidence: {{ bird.confidence }}\n{{ relative_time(bird.datetime | as_datetime) }}",
         "multiline_secondary": true
       },
       {%- endfor %}]

Yes, I typically use a trigger based template sensor which stores the data sent in an event.
In this case you could use an MQTT trigger and store the data you need in an attribute, on which you add data on each trigger.
You could add an additional trigger to reset periodically or store the last X events

Thanks Petro. That is way beyond my ken. I’m reading about fields now. Never used them before, only variables.

Using concatenate?

Or is there a better way to add new data to a list in an attribute?

Something like this

trigger: 
  - platform: mqtt
    ...
  - platform: time
    at: "00:00:00"
    id: reset
sensor: 
  - unique_id: something unique
    name: Bird Events
    attributes: 
      bird_events: >
        {% if trigger.id == 'reset' %}
          {{ [] }}
        {% else %}
          {% set time = trigger.mqtt.... %}
          {% set name = trigger.mqtt.... %}
          {% set confidence = trigger.mqtt.... %}
          {% set current = this.attributes.get('bird_events', []) %}
          {% set new = dict(time=time, name=name, confidence=confidence) %}
          {{ current + [new] %}
        {% endif %}

I’ve implemented this triggered template sensor:

- trigger: 
    - platform: mqtt
      topic: "birdnet"
    - platform: time
      at: "00:00:00"
      id: reset
  sensor: 
    - unique_id: c893533c-3c06-4ebe-a5bb-da833da0a947
      name: BirdNET-Go Events
      state: "{{ today_at(trigger.payload_json.Time) }}"
      attributes: 
        bird_events: >
          {% if trigger.id == 'reset' %}
            {{ [] }}
          {% else %}
            {% set time = trigger.payload_json.Time %}
            {% set name = trigger.payload_json.CommonName %}
            {% set confidence = trigger.payload_json.Confidence|round(2) * 100 ~ '%' %}
            {% set current = this.attributes.get('bird_events', []) %}
            {% set new = dict(time=time, name=name, confidence=confidence) %}
            {{ current + [new] }}
          {% endif %}

Unfortunately I have very noisy earthworks going on in my garden at the moment (replacing my PITA lawn with a native garden) so I will be waiting a while for the next detection.

Once I get some data I’ll start on the display.

EDIT: Landscapers were quiet for a bit. Baby steps…

type: markdown
title: "BirdNET-Go"
content: |-
  Time|CommonName|Confidence
  :---|:---|:---
  {%- set t = now() %}
  {%- for bird in state_attr('sensor.birdnet_go_events','bird_events') or [] %}
  {{ bird.time}} | {{bird.name}} | {{ bird.confidence }}
  {%- endfor %}

Screenshot 2024-04-08 at 11-38-59 Administration – Home Assistant
.

Now to make it fancier.

2 Likes

Oops. Need to change the state template.

        state: >
          {% if trigger.id == 'reset' %}
            {{ now() }}
          {% else %}
            {{ today_at(trigger.payload_json.Time) }}
          {% endif %}

Iquiring minds want to know - have you made it fancier?

No fighting with other important issues at the moment (shitty iotawatt integration), but something has to be done. I replaced the microphone and now the birdnet list of detections is 100s of lines long each day.

Yes, me too - for now, I reverse sort them by time, and show the top 10…

1 Like

That’s a good idea. Care to share your config for this?

Just a small addition to your markdown card:

  - type: markdown
    title: "BirdNET Events"
    content: |-
      Time|CommonName|Confidence
      :---|:---|:---
      {%- set t = now() %}
      {%- set bird_list = [] %}
      {%- for bird in (state_attr('sensor.birdnet_intake','bird_events')| sort(attribute='time', reverse=true))[0:10] or []  %}
      {{bird.time}} | {{bird.name}} | {{bird.confidence }}
      {%- endfor %}

My next attempt will be to show just the most recent ‘sighting’ of each bird…

2 Likes

I’ve also used the new webpage dashboard to put the full BirdNET dashboard in the side menu but it only works when I’m at home and using http access.

This code presents the most recent sighting of each unique bird in your events:

  - type: markdown
    title: "BirdNET Events"
    content: |-
      Time|Common Name|Day Count|Max Confidence
      :---|:---|:---:|:---:
      {%- set t = now() %}
      {%- set bird_list = state_attr('sensor.birdnet_intake','bird_events') | sort(attribute='time', reverse=true) | map(attribute='name') | unique | list %}
      {%- set bird_objects = state_attr('sensor.birdnet_intake','bird_events') | sort(attribute='time', reverse=true) %}
      {%- for thisbird in bird_list or [] %}
      {%- set ubird = ((bird_objects | selectattr("name", "equalto", thisbird)) | list)[0] %}
      {%- set ubird_count = ((bird_objects | selectattr("name", "equalto", thisbird)) | list) | length %}
      {%- set ubird_max_confidence = ((bird_objects | selectattr("name", "equalto", thisbird)) | map(attribute='confidence') | max) %}
      {{ubird.time}} | {{ubird.name}} | {{ubird_count}} | {{ubird_max_confidence }}
      {%- endfor %}

(edited to show most recent version)

3 Likes

That’s awesome. Thank you for sharing. The only addition I made was a couple of &nbsp; spaces in front of the name and confidence for legibility:

Screenshot 2024-04-17 at 11-22-44 Mobile View – Home Assistant

Time|&nbsp; Common Name|Day Count|&nbsp; Max Confidence
:---|:---|:---:|:---:
{%- set t = now() %}
{%- set bird_list = state_attr('sensor.birdnet_go_events','bird_events') | sort(attribute='time', reverse=true) | map(attribute='name') | unique | list %}
{%- set bird_objects = state_attr('sensor.birdnet_go_events','bird_events') | sort(attribute='time', reverse=true) %}
{%- for thisbird in bird_list or [] %}
{%- set ubird = ((bird_objects | selectattr("name", "equalto", thisbird)) | list)[0] %}
{%- set ubird_count = ((bird_objects | selectattr("name", "equalto", thisbird)) | list) | length %}
{%- set ubird_max_confidence = ((bird_objects | selectattr("name", "equalto", thisbird)) | map(attribute='confidence') | max) %}
{{ubird.time}} |&nbsp; {{ubird.name}} | {{ubird_count}} |&nbsp; {{ubird_max_confidence }}
{%- endfor %}

So I got a new microphone. In the nighttime recordings (after traffic noise has died down) I can hear the waves crashing on the beach 1.3Km away and 100m below.
Lots more bird detections. Too many in-fact:

Logger: homeassistant.components.recorder.db_schema
Source: components/recorder/db_schema.py:598
integration: Recorder (documentation, issues)
First occurred: 15:26:57 (14 occurrences)
Last logged: 15:53:00

State attributes for sensor.birdnet_go_events exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored

1 Like

Create multiple sensors and filter based on time?
So 1 for the hours 00:00 - 06:00
the 2nd none for 06:00 - 12:00 etc

Hello, can you share configuration of mqtt sensor for birdnet-go? I’m trying to get the data from birdnet

mqtt:
  sensor:
    - name: "BirdNet"
      state_topic: "birdnet"
      unique_id: bird

I don’t use an mqtt sensor. I use a triggered template sensor as shown above Displaying Birdnet-go detections - #6 by tom_l

And this post: Displaying Birdnet-go detections - #7 by tom_l

1 Like

thanks Tom, it’s works for me, maybe you can tell me how to add a link to the wiki about each bird like this https://community.home-assistant.io/t/birdnet-pi/444654/10