Facial recognition & room presence using Double Take & Frigate

Is it possible to use this add-on without Frigate (corals are impossible to get) ?
I have CompreFace running well on a server. I have this addon running in Home Assistant,
and the two communicate fine with each other. If I upload a photo to DT, it uses CF to recognize it well.
I also use a doorbell camera from which I can capture a photo with a press and store it locally on HA.
Is there anyway to feed that local jpg image to DT so that it will be identified? Using MQTT looks nontrivial.

I can feed an image manually, so it seems like it should be possible.

EDIT: Figured it out, by reading above how to use the ONVIF bit. I use the Amcrest AD410, which can fetch an ONVIF image via http: http://username:password@ip/cgi-bin/snapshot.cgi.
First set up a camera that way in DT:

 cameras:
  front-door: 
    snapshot:
      url: http://camera_username:camera_password@camera_url/cgi-bin/snapshot.cgi

then made a shell_command in the configuration.yaml file:

shell_command:
  get_doorbell_snapshot: "curl --location --request GET 'http://ha_url:3000/api/camera/front-door'"

I can then run this shell command whenever the door button is pressed, and double take does the rest.
Like magic. Fantastic!

1 Like

Any ideas why I am receiving multiple notifications for the same event?

    - alias: Double Take
      trigger:
        - platform: state
          entity_id: sensor.double_take_user1
        - platform: state
          entity_id: sensor.double_take_user2
      action:
        - service: notify.mobile_app_user1_iphone
          data:
            message: |-
              {% if trigger.to_state.attributes.match is defined %}
                {{trigger.to_state.attributes.friendly_name}} is near the {{trigger.to_state.state}} @ {{trigger.to_state.attributes.match.confidence}}% by {{trigger.to_state.attributes.match.detector}}:{{trigger.to_state.attributes.match.type}} taking {{trigger.to_state.attributes.attempts}} attempt(s) @ {{trigger.to_state.attributes.duration}} sec
              {% elif trigger.to_state.attributes.unknown is defined %}
                unknown is near the {{trigger.to_state.state}} @ {{trigger.to_state.attributes.unknown.confidence}}% by {{trigger.to_state.attributes.unknown.detector}}:{{trigger.to_state.attributes.unknown.type}} taking {{trigger.to_state.attributes.attempts}} attempt(s) @ {{trigger.to_state.attributes.duration}} sec
              {% endif %}
            data:
              attachment:
                url: |-
                  {% if trigger.to_state.attributes.match is defined %}
                    http://192.168.141.17:3000/api/storage/matches/{{trigger.to_state.attributes.match.filename}}?box=true
                  {% elif trigger.to_state.attributes.unknown is defined %}
                     http://192.168.141.17:3000/api/storage/matches/{{trigger.to_state.attributes.unknown.filename}}?box=true
                  {% endif %}
              actions:
                - action: URI
                  title: View Image
                  uri: |-
                    {% if trigger.to_state.attributes.match is defined %}
                      http://192.168.141.17:3000/api/storage/matches/{{trigger.to_state.attributes.match.filename}}?box=true
                    {% elif trigger.to_state.attributes.unknown is defined %}
                       http://192.168.141.17:3000/api/storage/matches/{{trigger.to_state.attributes.unknown.filename}}?box=true
                    {% endif %}
      mode: parallel
      max: 10

Configuration:

    mqtt:
      host: 192.168.141.21
      username: USER
      password: REDACTED

    home_assistant:
      url: http://192.168.141.11:8123
      token: XXXXXXXX

    frigate:
      url: http://192.168.141.21:5000
        # only process images from specific cameras
      cameras:
        - driveway_cam
        - living_room_cam

    detectors:
      compreface:
        url: http://192.168.141.17:8000
        key: KEY

    detect:
      match:
        # minimum confidence needed to consider a result a match
        confidence: 70
        # minimum area in pixels to consider a result a match
        min_area: 2500

      unknown:
        # minimum confidence needed before classifying a name as unknown
        confidence: 40
        # minimum area in pixels to keep an unknown result
        min_area: 0

1 Like

Spoke to soon. Using the API call only works very occasionally. I have been testing it, and it has recognized it only twice, after a couple dozen attempts. I cannot find any logs to check, so am stuck at this point. Does anyone know how to get logs and debug this setup?

I’m having the same problem and can’t figure out how to configure this either in Frigate, or in DoubleTake. There are cases where I have nearly three full pages of thumbnails of basically same posture of me and my son getting ready for cycling in front of our one camera - all taken within probably 10 sec or so.

It is either Frigate that is sending events too often and not recognising this as a same event or some other setting I can’t figure out. Can’t see it is because of max_disappeared or so.

To clarify, in Frigate’s UI for this specific event only 1 image is stored/shown. Maybe it is that simple that I just lower the number of snapshots/latest in Doubletake, but I worry this would lower the quality of recognition at the same time.

No-one else has this?

I am trying to create a sensor that reports the confidence of the matches for a person, I then want to pass this sensor to a statistics sensor to average the readings from the last couple minutes. I cannot seen to get the confidence sensor working. Below is my yaml for the sensor.

mqtt:

  sensor:

    - name: "joe_face"

      state_topic: 'double-take/matches/joe'

      json_attributes_topic: 'double-take/matches/joe'

      value_template: '{{ value_json.confidence }}'

Hi, @Noy1122, based on your comments later, seems like you’ve moved forward and solved the below issue, could you please share how you fixed it? I’m having exactly the same issue, running the deepstack examples in python returns the expected result, but double-take is showing an red dot status for deepstack and docker logs double-take gives below error.

error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')

It stopped working for me a week ago and I don’t know why again :cry: :cry:

Hi !

My double take add-on stops running for whatever reason. At the begining it can connect to deepstack but after around 20seconds the double-take add-on stops. Putting log in debug dosen’t give any indications. I remover all unnecessary configuration like mqtt of frigate.

Is there any command line to start it and investigate the behavior ?

Finally I found that the independant docker runs fine with the same configuration used by the add-on. Perhap’s it is some HA orchestrator*, i’ll investigate.

Hey there,
I’ve been working on getting this all setup and finally got around my compreface issues but now I am getting “match: true” but “name : unknown” in double take despite compreface naming me correctly. I am not sure where else to check to coordinate between the detector and doubletake.

Any direction would be appreciated!

EDIT: Solved : I had my Unknown percentage too high. Thanks!

I am unable to get this going. I’ve had Frigate working for a while.
Everything looks to be okay:
1

MQTT looks to be working as it should based on MQTT explorer (with the exception that I’m uncertain what camera_1 is because I didn’t define that in Frigate):

Here’s my Double Take config:

Here’s my Double Take log:

And here’s what I see (i.e., nothing):

The only exception is that Frigate logs seem to indicate it’s not connecting to MQTT logs (but MQTT Explorer would seem to show otherwise), giving:
“frigate.mqtt ERROR : Unable to connect to MQTT server: MQTT Not authorized”

Any help would be very much appreciated. Thanks!

I finally found out why this is not working for me, the url for deepstack can’t end with a slash /, seems like the double take code is adding another slash / at the end, which is causing the rest api to fail.
hope this helps someone else in a similar situation.

@Jako could you please fix this one from the code?

i.e.:
instead of

detectors:
  deepstack:
    url: http://192.168.86.60:5050/

I changed the configuration to:

detectors:
  deepstack:
    url: http://192.168.86.60:5050

trying to repro using python with the deepstack API by calling the url with double slash gives error as well (though different one).

1 Like

I figured it out. My problem was that Frigate was not communicating with Mosquitto Broker. I was aware from other reports that a user and password are required in frigate.yml but thought that port was set by default to 1883. However, I needed to list everything at the beginning of frigate.yml (at least using my Hassio installation).

mqtt:
  host: your local host ip address
  port: 1883 # <-- required
  user: xxx
  password: xxx

Also set this user and password in the Mosquitto Broker configuration:

logins:
  - username: xxx
    password: xxx

Now everything’s working.

Hi, same here, the addons connects to all the services and then stop after 20s without any error in the log.

Hi Jako,

Great job. I love this and I got it working, but not 100% as I want too.

I have everything running in docker containers (Frigate, Deepstack, Double Take), but it needed to have frigate and deepstack running on a different server as they only seem to work from port 5000. Have you ran into this issue as well as I really want to run this all from one machine.

I used your example, but it just doesn’t work. Once I use a deepstack docker running on another machine I can get it working.

version: ‘3.7’

services:
double-take:
container_name: double-take
image: jakowenko/double-take
restart: unless-stopped
environment:
DETECTORS: deepstack
MQTT_HOST: 123.123.123.123
FRIGATE_URL: http://123.123.123.123:PORT
DEEPSTACK_URL: http://123.123.123.123:PORT
ports:
- 3000:3000

Any insight would be greatly appreciated.

Thanks,

Pim

Hey @verschup76, you should be able to run each of them on different ports.

You can have your local port be different than the port the container needs. For example this would run DeepStack on 5001.

deepstack:
    container_name: deepstack
    image: deepquestai/deepstack
    restart: unless-stopped
    ports:
      - 5001:5000
    environment:
      VISION-FACE: 'True'
      MODE: High

Thanks Jako, that did the trick. I should have figured it out myself, but I think I was just fixated on something else :wink:

Hello! I’m looking for some clarifications about the following default settings:

Assuming the point of mqtt is to use subscribe/publish to communicate – i.e. Double-take wouldn’t spam Frigate asking “are we there yet” in car analogy terms, but instead wait for Frigate to publish to mqtt something that Double-take is subscribed to look for (mqtt: true), What does latest and snapshot set to a non-zero value do here?

Like, what exactly does the value of 10 mean? I assume Double-take isn’t asking Frigate for the same latest and snapshot files 10 times in a row with a 0 second delay? The only thing else I can think of is that these settings exist to tell Double-Take to re-process the files with the hope there will be different outcomes, but I don’t think that makes much sense either, unless training is always on and we want some kind of positive feedback loop? But even if that were true, double-take should be able to just re-use the images it already downloaded, not re-download the same files 10 times?

Thanks!

2 Likes

Hi Jako,

long long long time ago I created this and now I see compreface has new containers. how do I update these:

CONTAINER ID   IMAGE                           COMMAND                  CREATED         STATUS         PORTS                                                      NAMES
c2b75222cc15   exadel/compreface-admin:0.6.1   "sh -c 'java $ADMIN_…"   2 minutes ago   Up 2 minutes                                                              compreface-admin
5affef49f93d   exadel/compreface-fe:0.6.1      "/docker-entrypoint.…"   3 minutes ago   Up 3 minutes   0.0.0.0:8000->80/tcp                                       compreface-ui
e45d37999516   jakowenko/double-take           "/bin/bash ./entrypo…"   4 minutes ago   Up 4 minutes   0.0.0.0:3000->3000/tcp                                     double-take
95b1ea0192b7   portainer/portainer-ce:latest   "/portainer"             7 minutes ago   Up 7 minutes   0.0.0.0:9000->9000/tcp, 8000/tcp, 0.0.0.0:9443->9443/tcp   portainer
e109b6d0015f   exadel/compreface-core:0.6.1    "uwsgi --ini uwsgi.i…"   3 months ago    Up 4 weeks     3000/tcp                                                   compreface-core
fd72d843981d   postgres:11.5                   "docker-entrypoint.s…"   3 months ago    Up 4 weeks     5432/tcp                                                   compreface-postgres-db
f95a0e1f2d54   exadel/compreface-api:0.6.1     "sh -c 'java $API_JA…"   3 months ago    Up 4 weeks                                                                compreface-api