Facial recognition & room presence using Double Take & Frigate

Jako · November 27, 2021, 6:39pm

Hey @tbrausch, thank you for checking out Double Take.

I’ve attached a GIF of what should happen for #1. Can you look at the logs and see if there are any errors that occur right after the upload?

The people sensors for #2 only get created when there is a positive match for that person. I may tweak this in a future release, but if there have been no matches for the person you created, then the sensor won’t be created yet. In your case it seems like all the results have been for the “unknown” person. If you jump over to our Discord server, I can help you fine tune your config to give you more matches.

2021-11-27 13.37.01

Jako · November 27, 2021, 6:41pm

Happy to announce with the help of others we have a Home Assistant CompreFace Add-on which includes persistent storage.

Ben_Asraf · November 28, 2021, 1:04pm

first of all i want to say thank you it’s an amazing tool,
i have a problem with the new version it’s keep crashing ever since i upgraded

tbrausch · November 28, 2021, 1:33pm

Basically, after I select the image and click OK, the dialog box disappears and it acts like I cancelled the operation. There are no errors reported in the log. My home assistant is running on a separate linux installation, but my main system is windows based. This is where I have been trying to upload an image from. I tried the same thing from the linux server that home assistant is running on and the upload worked OK. Should this work from a browser in windows? I will get with you soon for help on my config - I tried the new Compreface add-on and am having some difficulty getting it running.

Edit: My bad - I was running the instance of home assistant on windows via https, as soon as I changed it to http the file upload worked fine. Sorry for the trouble.

WolfsW3lp3 · November 28, 2021, 8:01pm

Hello,

I’m pretty new to all the AI stuff. I’m currently running Blue Iris and have already purchased a Jetson Nano for DeepStack and compreface (once supported). DeepStack for BlueIris mainly, or will that become obsolete with Double Take? Now the question is how do I integrate Double Take with BlueIris? What exactly do I have to consider to be able to use the room detection or to detect if a stranger is in the house or to open the door when you are standing in front of the door? (Would probably be quite insecure since no depth scan, any image would probably work then?)

Sorry, this question is rather general. I don’t know yet exactly how to implement it at all.

danbutter · November 29, 2021, 12:40am

It depends on what you are running BlueIris for…as in how you are using it and what features you are relying on in BlueIris.
I run BlueIris (BI) to do 24/7 recording and I make use of some of its features that aren’t available elsewhere. I use frigate for object detection and since Jako just got a single container addon for compreface I’ll be looking into Double Take soon for facial recognition.

I’ve tried deepstack and BI and while it works I had a heck of a time with all the extra stuff I needed to run and the false positives with deepstack were just too many to be useable. I wouldn’t recommend it.
I’ve been running frigate for about a year now and I think I have had 4 hits that were recognized as a person or car that weren’t. Pretty amazing stuff there.

Ben_Asraf · November 29, 2021, 9:39am

i have a problem with the compare face add on and since i upgraded to 1.7 double take add on keep crushing,
and for the life of me i cant figure out why

Jako · November 29, 2021, 2:24pm

@Ben_Asraf Can you send me the logs of the CompreFace addon? Does the UI load up? Or does it not start at all?

WolfsW3lp3 · November 29, 2021, 2:54pm

I’m using BlueIris for Recording when I’m away.
Have you directly connected your cameras to frigate or over BlueIris?
Have you used the GPU or the CPU Version of DeepStack?

About the facial recognition, what are you using it for? Intruder Alerts? I can’t imagine an
automation for now that could be useful, presence detection yes but face?

danbutter · November 29, 2021, 10:22pm

Well if you are comfortable with the way frigate works you could use that for recording. I prefer being able to use BI like a more traditional NVR and don’t use frigate to record any video…just do object recognition.
I connected my cameras to frigate (2nd stream) as well as BI (main stream)…not one through the other.
I tried out both versions of deepstack, but not for over a year now.
I not using double take at all yet, but I plan to play with it. I think it could be useful to tell who is where at certain times.
If you don’t think facial recognition is worth much to you then I would suggest you go check out frigate and see if that works for your needs a bit better. It is more object detection and it is quite good at it.
HTH

Ben_Asraf · November 30, 2021, 8:05am

the ui starts i’m able to enter and than it always alert me this “Error during synchronization between servers: Connection refused (Connection refused) executing GET http://localhost:3000/status” when im trying to enter the app to get a key,

but the main problem really is that double take keeps crushing since i update to latest version

Jako · November 30, 2021, 3:10pm

Hey @Ben_Asraf, have you verified your system has AVX2? I’ve updated the add-on to check for that before starting, but it’s a requirement for CompreFace.

thundergreen · November 30, 2021, 6:50pm

I installed Double-Take latest beta 1.7.0 but I get gateway error. 1.6.0 works properly. I openene an issue on github . Maybe you can have a look @Jako ?

[BUG] Gateway Error in version 1.7.0 · Issue #173 · jakowenko/double-take (github.com)

Ben_Asraf · November 30, 2021, 9:40pm

thats my problem too

Jako · November 30, 2021, 10:19pm

I will take a look at it tonight and report back! For now it should work if you use port 3000 directly and don’t go through ingress.

Ben_Asraf · November 30, 2021, 10:23pm

i doesnt because the add on keeps stopping at least for me

thundergreen · November 30, 2021, 10:32pm

I had to change the port to another as 3000 is already in use but Still same issue. But another strange issue after trying to run the container. The Zwavejs addon and integration stops working. I will investigate this. It happened after my last Backup .(

Jako · December 1, 2021, 2:21am

What do the logs of your add on say?

Ben_Asraf · December 1, 2021, 8:11am

thats what i have in the log
info: Double Take v1.7.0-bf3e911
info: MQTT: connected
info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

Noy1122 · December 5, 2021, 12:19pm

Maybe you can tell me what I am doing wrong?

I see the cams on frigate and I see events in frigate.
I see detection on docker

but in double take its show me “no files”

this is the conf in double take

I have logs from double take

21-12-05 08:20:53 info: processing front: 704605e4-b3fc-469c-a73e-f624bfe24764
21-12-05 08:20:53 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:53 info: done processing front: 704605e4-b3fc-469c-a73e-f624bfe24764 in 0.01 sec
21-12-05 08:20:53 info: {
  id: '704605e4-b3fc-469c-a73e-f624bfe24764',
  duration: 0.01,
  timestamp: '2021-12-05T06:20:53.531Z',
  attempts: 1,
  camera: 'front',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:20:54 info: processing Door Camera: 1638685244.83641-xw0vbs
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 1)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 info: done processing Door Camera: 1638685244.83641-xw0vbs in 0.18 sec
21-12-05 08:20:54 info: {
  id: '1638685244.83641-xw0vbs',
  duration: 0.18,
  timestamp: '2021-12-05T06:20:54.269Z',
  attempts: 2,
  camera: 'Door Camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:21:02 info: processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9
21-12-05 08:21:02 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:21:02 info: done processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9 in 0.01 sec
21-12-05 08:21:02 info: {
  id: 'd343ebb2-3837-447c-91ba-19566bef2ad9',
  duration: 0.01,
  timestamp: '2021-12-05T06:21:02.125Z',
  attempts: 1,
  camera: 'door camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 10:51:37 info: Double Take v1.7.0-bf3e911
21-12-05 10:51:37 info: MQTT: connected
21-12-05 10:51:37 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:08:55 info: Double Take v1.7.0-bf3e911
21-12-05 11:08:55 info: MQTT: connected
21-12-05 11:08:55 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:15:58 info: Double Take v1.7.0-bf3e911
21-12-05 11:15:58 info: MQTT: connected
21-12-05 11:15:58 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

and that what I see in HA about the image_processing

this is the conf of image processing

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.199
    port: 80
    save_file_folder: /config/www
    save_file_format: jpg
    always_save_latest_file: True
    targets:
      - target: person
        confidence: 60
    source:
      - entity_id: camera.door_camera_2
      - entity_id: camera.front

and this is the conf of frigate

cameras:
  Door Camera:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


            
  Front:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


    
detectors: 
  cpu1:
    type: cpu
  cpu2:
    type: cpu

why its not working to me?