Facial recognition & room presence using Double Take & Frigate

What do the logs of your add on say?

thats what i have in the log
info: Double Take v1.7.0-bf3e911
info: MQTT: connected
info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

Maybe you can tell me what I am doing wrong? :slightly_frowning_face:

I see the cams on frigate and I see events in frigate.
I see detection on docker

but in double take its show me “no files”

this is the conf in double take

I have logs from double take

21-12-05 08:20:53 info: processing front: 704605e4-b3fc-469c-a73e-f624bfe24764
21-12-05 08:20:53 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:53 info: done processing front: 704605e4-b3fc-469c-a73e-f624bfe24764 in 0.01 sec
21-12-05 08:20:53 info: {
  id: '704605e4-b3fc-469c-a73e-f624bfe24764',
  duration: 0.01,
  timestamp: '2021-12-05T06:20:53.531Z',
  attempts: 1,
  camera: 'front',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:20:54 info: processing Door Camera: 1638685244.83641-xw0vbs
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 1)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 info: done processing Door Camera: 1638685244.83641-xw0vbs in 0.18 sec
21-12-05 08:20:54 info: {
  id: '1638685244.83641-xw0vbs',
  duration: 0.18,
  timestamp: '2021-12-05T06:20:54.269Z',
  attempts: 2,
  camera: 'Door Camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:21:02 info: processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9
21-12-05 08:21:02 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:21:02 info: done processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9 in 0.01 sec
21-12-05 08:21:02 info: {
  id: 'd343ebb2-3837-447c-91ba-19566bef2ad9',
  duration: 0.01,
  timestamp: '2021-12-05T06:21:02.125Z',
  attempts: 1,
  camera: 'door camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 10:51:37 info: Double Take v1.7.0-bf3e911
21-12-05 10:51:37 info: MQTT: connected
21-12-05 10:51:37 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:08:55 info: Double Take v1.7.0-bf3e911
21-12-05 11:08:55 info: MQTT: connected
21-12-05 11:08:55 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:15:58 info: Double Take v1.7.0-bf3e911
21-12-05 11:15:58 info: MQTT: connected
21-12-05 11:15:58 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

and that what I see in HA about the image_processing
4

this is the conf of image processing

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.199
    port: 80
    save_file_folder: /config/www
    save_file_format: jpg
    always_save_latest_file: True
    targets:
      - target: person
        confidence: 60
    source:
      - entity_id: camera.door_camera_2
      - entity_id: camera.front

and this is the conf of frigate

cameras:
  Door Camera:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


            
  Front:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


    
detectors: 
  cpu1:
    type: cpu
  cpu2:
    type: cpu

why its not working to me?

Hey @Noy1122, it looks like you have a key set for DeepStack but don’t have it configured in your Double Take config.yml file, which is resulting in DeepStack API to throw 401 errors.

I tried to set API KEY but I still got errors, I need to make the API-KEY here?
I just change “mysecretkey” to 112233

docker run -e VISION-DETECTION=True -e API-KEY="mysecretkey" -v localstorage:/datastore -p 80:5000 deepquestai/deepstack

and then in the conf.yaml

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.199
    port: 80
    api_key: 112233
    save_file_folder: /config/www/deepstack/snapshots/
    save_file_format: jpg
    always_save_latest_file: True
    targets:
      - target: person
        confidence: 60
    source:
      - entity_id: camera.door_camera_2
      - entity_id: camera.front

and in the double take conf

# Double Take
mqtt:
  host: 192.168.1.199
  username: XXXX
  password: XXXX
  
frigate:
  url: http://192.168.1.199:5000
  
  
detectors:
  deepstack:
    url: http://192.168.1.199:3002
    key: 112233

and I still got error

21-12-07 10:50:59 info: processing Door Camera: 1638867044.277462-wn6fkr
21-12-07 10:50:59 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 1)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-07 10:50:59 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-07 10:50:59 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-07 10:51:00 info: done processing Door Camera: 1638867044.277462-wn6fkr in 0.33 sec
21-12-07 10:51:00 info: {
  id: '1638867044.277462-wn6fkr',
  duration: 0.33,
  timestamp: '2021-12-07T08:51:00.019Z',
  attempts: 5,
  camera: 'Door Camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-07 10:51:06 info: processing door camera: 8d19bfb9-f9a3-45ed-a881-e949e35cb25e
21-12-07 10:51:06 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-07 10:51:06 info: done processing door camera: 8d19bfb9-f9a3-45ed-a881-e949e35cb25e in 0.03 sec
21-12-07 10:51:06 info: {
  id: '8d19bfb9-f9a3-45ed-a881-e949e35cb25e',
  duration: 0.03,
  timestamp: '2021-12-07T08:51:06.804Z',
  attempts: 1,
  camera: 'door camera',
  zones: [],
  matches: [],
  misses: []

but now I dont have error 401

and its look good now
1111

I will check it with the camera when I come home, maybe its works now but I still no see matches

stil not working :slightly_frowning_face: :slightly_frowning_face:

this is the log from double take

}
21-12-07 15:53:31 info: processing front: ab5f2aab-98d0-448a-be27-cc4d6f634756
21-12-07 15:53:31 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-07 15:53:31 info: done processing front: ab5f2aab-98d0-448a-be27-cc4d6f634756 in 0.01 sec
21-12-07 15:53:31 info: {
  id: 'ab5f2aab-98d0-448a-be27-cc4d6f634756',
  duration: 0.01,
  timestamp: '2021-12-07T13:53:31.359Z',
  attempts: 1,
  camera: 'front',
  zones: [],
  matches: [],
  misses: []
}
21-12-07 15:53:34 info: processing door camera: 31e84128-2cbd-4c1b-b07d-63ab728a6622
21-12-07 15:53:34 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-07 15:53:34 info: done processing door camera: 31e84128-2cbd-4c1b-b07d-63ab728a6622 in 0.01 sec
21-12-07 15:53:34 info: {
  id: '31e84128-2cbd-4c1b-b07d-63ab728a6622',
  duration: 0.01,
  timestamp: '2021-12-07T13:53:34.511Z',
  attempts: 1,
  camera: 'door camera',
  zones: [],
  matches: [],
  misses: []
}

@Noy1122 Can you go to localhost:3000/api/recognize/test and paste the output of the call here? That should be the raw DeepStack response.

yes of course. this is the log

[{"detector":"deepstack","status":200,"response":"<!DOCTYPE html><html lang=\"\"><head><meta charset=\"utf-8\"><meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><meta id=\"theme-color\" name=\"theme-color\" content=\"#20262e\"><link rel=\"icon\" type=\"image/svg+xml\" href=\"./favicon.svg\"><link rel=\"apple-touch-icon\" href=\"./apple-touch-icon.png\"><meta name=\"apple-mobile-web-app-capable\" content=\"yes\"><link href=\"./splash/iphone-12-pro-max.png\" media=\"(device-width: 428px) and (device-height: 926px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-12.png\" media=\"(device-width: 390px) and (device-height: 844px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-12-mini.png\" media=\"(device-width: 375px) and (device-height: 812x) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-11-pro-max.png\" media=\"(device-width: 414px) and (device-height: 896px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-11-pro.png\" media=\"(device-width: 375px) and (device-height: 812px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-11.png\" media=\"(device-width: 414px) and (device-height: 896px) and (-webkit-device-pixel-ratio: 2)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-xs-max.png\" media=\"(device-width: 414px) and (device-height: 896px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-xs.png\" media=\"(device-width: 375px) and (device-height: 812px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-xr.png\" media=\"(device-width: 414px) and (device-height: 896px) and (-webkit-device-pixel-ratio: 2)\" rel=\"apple-touch-startup-image\"><link href=\"./splash/iphone-x.png\" media=\"(device-width: 375px) and (device-height: 812px) and (-webkit-device-pixel-ratio: 3)\" rel=\"apple-touch-startup-image\"><link rel=\"manifest\" href=\"./manifest.json\"><link id=\"theme-link\" rel=\"stylesheet\" href=\"./themes/bootstrap4-dark-blue/theme.css\"><title>Double Take</title><link href=\"css/config.fb9b6e53.css\" rel=\"prefetch\"><link href=\"css/login.53072c61.css\" rel=\"prefetch\"><link href=\"css/match.783c2617.css\" rel=\"prefetch\"><link href=\"css/match~train.9952cffa.css\" rel=\"prefetch\"><link href=\"css/token.6d25f6b8.css\" rel=\"prefetch\"><link href=\"css/train.60c3ad3e.css\" rel=\"prefetch\"><link href=\"js/config.3d71a644.js\" rel=\"prefetch\"><link href=\"js/config~match~train.e6413dd2.js\" rel=\"prefetch\"><link href=\"js/login.8d1d24fe.js\" rel=\"prefetch\"><link href=\"js/match.ad7c4e70.js\" rel=\"prefetch\"><link href=\"js/match~train.f842f601.js\" rel=\"prefetch\"><link href=\"js/token.b4dfa748.js\" rel=\"prefetch\"><link href=\"js/train.f33d112b.js\" rel=\"prefetch\"><link href=\"css/chunk-vendors.1620483c.css\" rel=\"preload\" as=\"style\"><link href=\"css/index.9ac8d443.css\" rel=\"preload\" as=\"style\"><link href=\"js/chunk-vendors.0212d0c8.js\" rel=\"preload\" as=\"script\"><link href=\"js/index.e08325f8.js\" rel=\"preload\" as=\"script\"><link href=\"css/chunk-vendors.1620483c.css\" rel=\"stylesheet\"><link href=\"css/index.9ac8d443.css\" rel=\"stylesheet\"><script>\n        window.ingressUrl = '';\n        window.publicPath = '';\n      </script>\n      </head><body><noscript><strong>We're sorry but Double Take doesn't work properly without JavaScript enabled. Please enable it to continue.</strong></noscript><div id=\"app\"></div><script src=\"js/chunk-vendors.0212d0c8.js\"></script><script src=\"js/index.e08325f8.js\"></script></body></html>"}]

It looks like your DeepStack URL is wrong or something else is happening. That test endpoint should’ve returned the recognition results, instead it’s an HTML page from DeepStack.

Have you tried doing the DeepStack Python example scripts to make sure it’s working?

Hello Jako, since I updated to version 1.7.0, double take has stopped working. I have done every suggestion here with no luck. Is there a way to rollback to 1.6.0?

:slightly_frowning_face: :slightly_frowning_face: :slightly_frowning_face:
I dont kno what to do…
how can I test it with DeepStack Python example scripts?
when I send image_processing.scan from Developer Tools I see it on ubuntu

Same here . Would like to roll back .any ideas how as my backup got deleted by the system already

I am not going to make an argument for facial recognition as being the most useful thing. It’s not. People are doing cool things with it if you look up and down this thread. Where I use it and I have found it handy is on my doorbell. I have my usual Fedex/UPS and USPS people saved, I get a notification on my google homes when they are spotted saying deliveries have arrived. I have friends and family saved so I know who is at the door without looking. If I get an unknown person then I can assume its a salesperson and promptly ignore it unless I had ordered door dash.

I wish my success rate was that good with frigate. Don’t get me wrong its hit rate is great but I still get false alerts. I blame this on my camera way more than I blame frigate though.

Part of it may be environmental as well, but for me frigate works better than the full fat version of tensorflow ever did, better than deepstack did and…I don’t remember the other one I tried out before.
I use substreams on my cams for detection so the resolution is lower, but it still pulls people out of a corner in the pics. I know that there is a lot of tweaking you can do with it if you go to the frigate github page. I just have masks on places I don’t want it to look so my CPU doesn’t have to work too hard.

Hello

I just installed double-take (using a docker on Unraid). I am wondering why am I getting matches for a person under the camera MQTT feed, but the /matches topic always just says “unknown”

Here is an example of “unknown” MQTT message (it does actually show name)

{
  "id": "1639882306.561435-759155",
  "duration": 4.84,
  "timestamp": "2021-12-19T02:51:56.749Z",
  "attempts": 11,
  "camera": "test",
  "zones": [],
  "misses": [
    {
      "name": "frank",
      "confidence": 79.72,
      "match": false,
      "box": {
        "top": 33,
        "left": 411,
        "width": 48,
        "height": 73
      },
      "checks": [
        "box area too low: 3504 < 10000"
      ],
      "type": "latest",
      "duration": 0.36,
      "detector": "deepstack",
      "filename": "26d82ab3-0cb1-4832-8c9e-5112242d7c53.jpg",
      "base64": null
    }
  ],
  "unknown": {
    "name": "unknown",
    "confidence": 0,
    "match": false,
    "box": {
      "top": 22,
      "left": 391,
      "width": 68,
      "height": 73
    },
    "checks": [
      "confidence too low: 0 < 60",
      "box area too low: 4964 < 10000"
    ],
    "type": "latest",
    "duration": 0.36,
    "detector": "deepstack",
    "filename": "7a067d3e-3d2f-42fe-8c81-22053dab40ae.jpg",
    "base64": null
  }
}

Are there any news why double take 1.7.0 does not work anymore ? Any way to roll back to 1.6.x without Backup?

1 Like

Yes, I have the same… updated to
image

ad it just doesnt work anymore…

I also had to manually copy-paste my yaml in the frontend now since it did not pick it up from the previous location…

So does it work for you finally or not ?:thinking::face_with_monocle:

For Me? Not…