Facial recognition & room presence using Double Take & Frigate

The box is a Intel NUC10I5FNH4 10th Gen Core i5 NUC with proxmox with Portainer on a debian 10 turnkey lxc container.

@Sumit, my installation is a bit old on a Mac mini. Maybe my docker-compose.yaml might help you a step further. As I am also not an experienced docker user I assume that I have just used the standard version of the compose file. And I also cannot help to translate it into a portainer stack. But let’s have a try: (and as you can see I am also new to Markdown, my apologies)

version: '3.4'

volumes:
  postgres-data:

services:
  compreface-postgres-db:
    image: postgres:11.5
    container_name: "compreface-postgres-db"
    environment:
      - POSTGRES_USER=${postgres_username}
      - POSTGRES_PASSWORD=${postgres_password}
      - POSTGRES_DB=${postgres_db}
    volumes:
      - postgres-data:/var/lib/postgresql/data

  compreface-admin:
    image: ${registry}compreface-admin:${ADMIN_VERSION}
    container_name: "compreface-admin"
    environment:
      - POSTGRES_USER=${postgres_username}
      - POSTGRES_PASSWORD=${postgres_password}
      - POSTGRES_URL=jdbc:postgresql://${postgres_domain}:${postgres_port}/${postgres_db}
      - SPRING_PROFILES_ACTIVE=dev
      - ENABLE_EMAIL_SERVER=${enable_email_server}
      - EMAIL_HOST=${email_host}
      - EMAIL_USERNAME=${email_username}
      - EMAIL_FROM=${email_from}
      - EMAIL_PASSWORD=${email_password}
      - ADMIN_JAVA_OPTS=${compreface_admin_java_options}
    depends_on:
      - compreface-postgres-db
      - compreface-api

  compreface-api:
    image: ${registry}compreface-api:${API_VERSION}
    container_name: "compreface-api"
    depends_on:
      - compreface-postgres-db
    environment:
      - POSTGRES_USER=${postgres_username}
      - POSTGRES_PASSWORD=${postgres_password}
      - POSTGRES_URL=jdbc:postgresql://${postgres_domain}:${postgres_port}/${postgres_db}
      - SPRING_PROFILES_ACTIVE=dev
      - API_JAVA_OPTS=${compreface_api_java_options}
      - SAVE_IMAGES_TO_DB=${save_images_to_db}

  compreface-fe:
    image: ${registry}compreface-fe:${FE_VERSION}
    container_name: "compreface-ui"
    ports:
      - "8022:80"
    depends_on:
      - compreface-api
      - compreface-admin

  compreface-core:
    image: ${registry}compreface-core:${CORE_VERSION}
    container_name: "compreface-core"
    environment:
      - ML_PORT=3000

Pojay-hcz, thanks for you reply. I reviewed some of the changes you had in your docker compose to the standard docker compose file. I tried what you used in docker compose and unfortunately had no luck and the same outcome as before.

I have similar issue here, wanted to check if you were able to solve this ?

Friends, please tell me how to limit the download of data? I have 1.9 gigabytes of unnecessary data in my deep stack that I can’t delete.

I’ve only just noticed the following error occurring in the home-assistant.log over and over again for each person I have trained in Double Take:

ERROR (MainThread) [homeassistant.components.sensor] Platform mqtt does not generate unique IDs. ID double_take_jim_bob already exists - ignoring sensor.double_take_jim_bob

I don’t see anything of interest in the Double Take or Mosquitto logs, anyone any idea what could be causing it? Other than the error being written I doesn’t seem to be impacting anything.

Where can i find " key " for configuration ?

error: detectors.compreface requires property “key”

detectors:
  deepstack:
    url: http://192.168.0.109:5001
    key: ?
    timeout: 30

Home Assistant 2022.11.4
Supervisor 2022.10.2
Operating System 9.3
Frontend 20221108.0 - latest
Add-on Double Take
Current version: 1.13.1

I get this as well, it improves when i actually use low res images from rtsp in small face boxes, but then at some point it also is not able to make the distinction, and i do not ever get unknown faces.

I will try this custom build.

I am using home assistant core, do we have HACS integration for this by any chance!

1 Like

This should get you started.

  • Change payload to your frigate camera name.

  • Change xxxxxxxx http references to your domains

  • Repeat trigger section if you want to add more cameras.

  • alias: Notify when person detected on camera
    description: ‘’

    The trigger is mqtt and will fire whenever an object is detected by YOUR_CAMERA_NAME

    trigger:

    • platform: mqtt
      topic: frigate/events
      id: frigate-event

      choose the camera you want notifications for.

      payload: YOUR_CAMERA_NAME
      value_template: ‘{{ value_json[“after”][“camera”] }}’
      variables:
      camera: “{{ trigger.payload_json[“after”][“camera”] }}”
      id: “{{ trigger.payload_json[“after”][“id”] }}”
      label: “{{ trigger.payload_json[“after”][“label”] }}”

    Conditions must be used to filter out objects / detections that are not of interest

    condition:

    This condition is used to ensure that I am only notified of objects that are new. This is to avoid getting loads of updated notifications for the same dog in my backyard (as an example).

    • condition: or
      conditions:
      • condition: template
        value_template: “{{ trigger.payload_json[‘type’] == ‘new’ }}”
        action:
    • wait_template: “states(‘sensor.double_take_{{camera}}’)”
      timeout: 00:00:10
    • service: notify.pushover
      data_template:
      title: >-
      {{ label }} detected in {{ camera }}
      message: >-
      Time: {{ now().timestamp() | timestamp_custom(’%a %d-%m-%y %-I:%M %p’) }}
      {%- if label == ‘person’-%}
      {%- if state_attr(‘sensor.double_take_’+camera,‘counts’)[“person”] > 0 -%}
      {%- if state_attr(‘sensor.double_take_’+camera,‘counts’)[“match”] > 0 -%}
      {{-’\n’-}}Identification match:{{-’ ‘-}}
      {{ state_attr(‘sensor.double_take_’+camera,‘matches’)[0][“name”] }}
      {{-’\n’-}}Confidence level:{{-’ ‘-}}
      {{ state_attr(‘sensor.double_take_’+camera,‘matches’)[0][“confidence”] }}
      {%- elif state_attr(‘sensor.double_take_’+camera,‘counts’)[“miss”] == 1 -%}
      {{-’\n’-}}Identification guess: {{-’ ‘-}}
      {{ state_attr(‘sensor.double_take_’+camera,‘misses’)[0][“name”] }}
      {{-’\n’-}}Confidence level:{{-’ ‘-}}
      {{ state_attr(‘sensor.double_take_’+camera,‘misses’)[0][“confidence”] }}
      {%- elif state_attr(‘sensor.double_take_’+camera,‘counts’)[“unknown”] == 1 -%}
      {{-’\n’-}}identification unknown{{-’ ‘-}}
      {{ state_attr(‘sensor.double_take_’+camera,‘unknowns’)[0][“name”]|default(“zero”) }}
      {{-’\n’-}}Confidence level:{{-’ '-}}
      {{ state_attr(‘sensor.double_take_’+camera,‘unknowns’)[0][“confidence”]|default(“zero”) }}
      {%- endif -%}
      {%- endif -%}
      {%- endif -%}
      data:
      timestamp: true
      expire: 3600
      url: >-
      {%- if state_attr(‘sensor.double_take_’+camera,‘counts’)[“person”] > 0 -%}
      {%- if state_attr(‘sensor.double_take_’+camera,‘counts’)[“match”] > 0 -%}
      http://xxxxxxxxx:3000/api/storage/matches/{{ state_attr(‘sensor.double_take_’+camera,‘matches’)[0][“filename”] }}
      {%- elif state_attr(‘sensor.double_take_’+camera,‘counts’)[“match”] == 0 -%}
      https://xxxxxxxxx.com/api/frigate/notifications/{{id}}/snapshot.jpg
      {%- endif -%}
      {%- endif -%}
      attachment: >-
      https://xxxxxxxxx.com/api/frigate/notifications/{{id}}/snapshot.jpg
      mode: parallel
3 Likes

Is the min area under frigate conf section in double take the same as the min area setting in frigate yaml. It a bit confusing, as min area in frigate .yaml file is for the whole person object, not the face.

Question regarding double take sensors in HA which behave unexpectedly ,
i’m using latest HA and latest double take and frigate - all on dockers.

I monitor one of double take camera sensor while i stood in front of the camera ,
I expect to see a match - as in the double take gui i do see above 95% of match ,
in the sensor , i see 2 issues:

1.The match is in the “misses” attribute
2.Match attribute is always false

{{states.sensor.double_take_salon_cam.attributes}}

output:  
{
  "id": "1669756030.098279-ofmv03",
  "duration": 21.98,
  "timestamp": "2022-11-29T21:32:02.664Z",
  "attempts": 10,
  "camera": "salon_cam",
  "zones": [],
  "matches": [],
  "misses": [
    {
      "name": "avi",
      "confidence": 99.2,
      "match": false,
      "box": {
        "top": 223,
        "left": 249,
        "width": 31,
        "height": 33
      },
      "checks": [
        "box area too low: 1023 < 10000"
      ],
      "type": "latest",
      "duration": 1.17,
      "detector": "compreface",
      "filename": "3f12cbb7-7fb7-49e1-be50-64724ec93488.jpg",
      "base64": null
    }
  ],
  "unknowns": [],
  "personCount": 0,
  "counts": {
    "person": 0,
    "match": 0,
    "miss": 0,
    "unknown": 0
  },
  "icon": "mdi:camera",
  "friendly_name": "double_take_salon_desk"
}

@Avi_Cohen

Examine your JSON above again, the answer is…

        "box area too low: 1023 < 10000"

This is why match is false.

Reduce the box config in your double-take and see if match triggers true

How has your experience been so far with Arcface? Are you using the GPU version ?
I still get false positives or unknowns, to the point where i find it hard to use it for any useful automations. It is much better, that is true (and also slower, but that’s fine i guess).

What gpu does it need ?

image
can someone have this problem before am still stuck for a while

@naphanz Assuming the above is Double Take logs, your errors is saying that Double Take can’t connect to your MQTT server at ip address 192.168.1.48, port 1883.

Please confirm that your MQTT ip address, username, password are correct in Double Take config file.

For reference, here is what my config looks like:

mqtt:
  host: 192.168.11.213
  username: mosquitto
  password: <REDACTED>

Related, I use MQTT Explorer to manually connect to MQTT server to confirm connection details and see what’s happening: https://mqtt-explorer.com/

I found the documentation confusing but I figured it out eventually. Just copy and paste the compreface URL (from your config.yml) into a browser… for me it was http://homeassistant.local:8000 or http://(Home Assistant IP Addr):8000, since I installed the Add-On. Then register an account, create an application, and service. The service you create will get an API Key and that’s what I used in the key field.

PS: You’re showing your detector config for deepstack, which doesn’t need a value for key, blank is fine. Compreface seemed to require it and the above steps worked for me.

Can’t reached to http://homeassistant.local:8000/

Exadel CompreFace log error AVX not detected

Did you try with just your IP address?

Yes, same error.