Facial recognition & room presence using Double Take & Frigate

@ChrisHA Have you tried a tool called “MQTT Explorer”. There is delete capability.

1 Like

Could someone help me prevent Double Take from showing an unknown face if confidence is less than a certain level? One of my cameras is in the living room and when someone is on the couch the camera doesn’t always see a clear face. I end up with a lot of detections if unknown because the face isn’t really visible. I would like to not see those. I thought that if I did this:

detect:
  match:
    confidence: 80
  unknown:
    confidence: 80

then if confidence of unknown was less than 80 it would not detect it but I am still getting detections like this:
image
Is there a way to stop this?

Also, how can I get Double Take to not try to detect the same image so many times in a row? I tend to get the same exact image detected 3-5 times with the same person in view. Along those lines, anyway of it only detecting on the snapshot and not the latest or mqtt image?

Thanks in advance.

Unknown is everything below 80

In both match and unknown you can try increasing the min_area a bit to cut off faces too small, they will still show in the results but will not count as a hit.

If you want no match then increase det_prob_threshold: 80 a little at a time and re-test your matches… it will return nothing if it fails this test. I have been creeping this value up and it has really reduced the number of images accepted and false positives from blurry movement shots etc.

detectors:
  compreface:
    # minimum required confidence that a recognized face is actually a face
    det_prob_threshold: 0.98
2 Likes

Hey Simon. How did you get the triggers and image passed to DT? Could you share the configs?

I sort of parked it and moved onto something else, DT will take in a Ring Doorbell snapshot but its far from perfect.

  • use Ring-MQTT to integrate and publish topics to your MQTT server
  • download MQTT Explorer to determine the topics published and find the Ring one that has the snapshot image
  • added the following code into the DT config:
    cameras:
    doorbell:
    snapshot:
    topic: ring/fce1e764-7caf-4098-a1a7-xxxxxxxx/camera/54e01xxxxxx/snapshot/image
    quality: 100
    height: 1080
    crop: True

Limitations:

  1. Only takes one snapshot image into DoubleTake, could do with multiple of to determine the best facial image
  2. Doesn’t zoom in onto the face of the person so the box size is too small for reliable AI detection

I’m confident there is a better way to do it I’ve just spent my time fixing other HA challenges. My other thought process while not quite as slick but may work was to create an automation that takes 3-5 snapshot images upon motion detection with a timestamp and then import these into DT, in advance of the import you could run them through a tool to resize etc. It’s just available time to determine the best way to solve the problem.

If anyone else has a slick way that integrates Ring into DT we could use that would save a huge amount of time, there must be someone out there :slightly_smiling_face:

1 Like

Thanks! I landed on something similar but also change the interval to 10s when motion is detected then back to 1 hour when it clears.

It’s a battery cam with no subscription though so snapshots seem unreliable at best. Agreed, I could go down the route of starting the cam and snapshotting frames but this seems lead to a range of other restrictions. I have a number of other cams through Frigate covering the front of the house and this does work so may just leave as it is and move on to the next project

Hi there, newbie question. How do i create an automation to have a camera ‘re-scan’ the room. Im using Doubletake, Frigate, Compreface. Ideally i would like to run this automation when my motion sensors detect motion. Or at the very least on a schedule like every 30 minutes. It seems like my system does not fire as much as I would like.

Hey,

This has probably been covered but thought I would ask.

I have setup my Hikvision cameras to capture faces and these are FTP uploaded to a folder, I have home assistant running but I would like to drop the detected face’s in a folder and then use Double Take to monitor is this possible?

Want to avoid having to run loads of middleware if the faces are being pulled out - does that make sense?

Any help appreciated!

Please tell me how to set up automatic deletion of snapshots in Double Take?

Hi.

there is a way to save matches for more than 5 days? I´m currently using double take for a month at least and only can see the matches from 5 days to today. I would like to save a hole month or more time. if it is possible?

This is controlled in the config under the ‘purge:’ section. More details here: GitHub - jakowenko/double-take: Unified UI and API for processing and training images for facial recognition.

1 Like

Ohh thanks I didn’t see it

Can you add CodeProject.AI Server to next release. It seen to replacing DeepStack.

3 Likes

hello,
am having the same issue. Have you managed to solve it ?

Any news of adding CodeProject.AI Server to next release. It seen to replacing Blue iris is using this instead of DeepStack. Please let me know if this can be done.

Thanks

2 Likes

I would love to use CodeProject.AI as well. Thank you.

Team; could use some help - I’m really struggling with firing off automations based on matches.

What am I trying to achieve?
I’ve got a front doorbell camera (more to come). When that camera detects a face (above match threshold), I want to send a notification.
What do I have working?
I’ve got frigate hooked up to the camera, detecting people. I’ve got Compreface set up, and I’ve got double take setup. In Compreface, I’ll get multiple attempts and eventually a match. For example, as I’m walking up to the camera, the box might be too small for the first few, etc. MQTT events are firing. I’m getting updates in HassIO.
What isn’t working?
I can’t figure out what to trigger off of. I’ve got an example of the state attribute I get below (ignore that this one is an unknown or a miss. The same basic attributes come through for a hit.

id: 1678673212.371881-ghhgqw
duration: 0.87
timestamp: 2023-03-13T02:07:43.868Z
attempts: 27
camera: cam_name
zones: 
matches: 
misses: 
unknowns: 
- name: unknown
  confidence: 8.5
  match: false
  box:
    top: 140
    left: 273
    width: 99
    height: 128
  checks:
    - 'confidence too low: 8.5 < 40'
  type: snapshot
  duration: 0.24
  detector: compreface
  filename: xxxxxx-2dc4-473c-a2ea-1f3114f1db54.jpg
  base64: null

personCount: 0
counts: 
person: 0
match: 0
miss: 0
unknown: 0

icon: mdi:camera
friendly_name: double_take_cam_name

What’s the right way to trigger? The example of triggering on people doesn’t work for me - I want to trigger on cameras. State doesn’t reliably go from 0->1. I can’t get personcount to work properly either (and it doesn’t reset that quickly).

alias: Notify on double take match
description: ""
trigger:
  - platform: state
    entity_id:
      - sensor.double_take_cam_name
condition:
  - condition: template
    value_template: "{{ trigger.to_state.attributes.matches|length > 1 }}'"
    enabled: true
action:
  - service: notify.mobile_app_APPNAME
    data:
      message: test {{ trigger.to_state.attributes.matches|length }}
mode: parallel
max: 10

Doesn’t work reliably, as an example. Any suggestions?

To add to this, I get weird outcomes like the one below. Both a match and a miss for the same person, and nothing under the counts?

id: 1678674704.36796-ko6qak
duration: 0.46
timestamp: '2023-03-13T02:31:45.277Z'
attempts: 3
camera: cam_name
zones: []
matches:
  - name: person_1
    confidence: 82.11
    match: true
    box:
      top: 46
      left: 146
      width: 56
      height: 66
    type: snapshot
    duration: 0.2
    detector: compreface
    filename: xxxx-7ff1-4b04-af9d-a0ebb16835e7.jpg
    base64: null
misses:
  - name: person_1
    confidence: 46.88
    match: false
    box:
      top: 180
      left: 276
      width: 23
      height: 28
    checks:
      - 'confidence too low: 46.88 < 80'
      - 'box area too low: 644 < 1000'
    type: latest
    duration: 0.18
    detector: compreface
    filename: xxxx-3cd8-493a-99a4-c76af2880200.jpg
    base64: null
unknowns: []
personCount: 0
counts:
  person: 0
  match: 0
  miss: 0
  unknown: 0
icon: mdi:camera
friendly_name: double_take_cam_name

I installed and setup frigate, doubletake and compeface one week ago and everything were running smoothly in my first tests, but today when I started to add more persons in compreface and even if they are getting detected in doubletake, I cannot see any mqtt entities for them in home assistant.

I see sensor.double_take_cam1, sensor.double_take_cam2, sensor.double_take_unknown, sensor.doutble_take_person1 but for the person 2 and 3 that got detected today was not created a sensor.

Any ideas?

Edit: I see know that the new persons haven’t been a match and that’s why are not popping up as sensors.

is there a way via the addon in the config to change storage location, I have a nas mounted in HA and would rather use that for storage, its quck enough over the 10gb network my ha box has.