Facial recognition & room presence using Double Take & Frigate

Share your Projects!
604

On my system (Intel NUC NUC7PJYH, 8 gig ram) Doubletake keeps on crashing after a few minutes. There are no errors in the logs (see log below). Am running Deepstack CPU and Frigate. Doubletake runs on port 3010 as the default port was already taken.

Can anyone point me in the right direction to debug this problem?

Log:

info: Double Take v1.13.1-8e2728d
info: MQTT: connected
info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

Configuration:

frigate:
  url: http://192.168.1.10:5000
  
mqtt:
  host: 192.168.1.10:1883
  username: TOPSECRET
  password: TOPSECRET
  
detectors:
  deepstack:
    url: http://192.168.1.10:5001
    timeout: 15
605

Sergey Krashevich just added CodeProject.AI into his fork of double-take:

Step to implemement on docker-compose setup (linux):

  1. Docker-compose example for codeproject.ai. Add the following lines into your compose file
codeproject.ai:
    container_name: codeproject.ai
    image: codeproject/ai-server
    restart: unless-stopped
    volumes:
      - /<data>/codeproject.ai/data:/etc/codeproject/ai
      - /<data>/codeproject.ai/modules:/app/modules
    ports:
      - 32168:32168
  1. Pull image
    docker-compose pull
  2. To create this version of doube-take docker container
  • Clone this repo to a temp. folder
    mkdir dttemp 
    cd dttemp
    git clone https://github.com/skrashevich/double-take.git
  • Run the build command
    ./.develop/build
  1. Adapt the double-take configruation - detectors:
aiserver:
    url: http://<ip-adress>:32168
    # number of seconds before the request times out and is aborted
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
    # only process images from specific cameras, if omitted then all cameras will be processed
    # cameras:
    #   - front-door
    #   - garage
  1. Restart containers (restart double-take and start project.ai)
    docker-compose up -d
  2. Train codeproject.ai …
  3. Have fun
2 Likes
607

Hi @S-Przybylski , first big thank you for code. I was able to get your forked repo to run in Docker Desktop on Windows 10.

I can see that Double-Take is sending images to CodeProjectAI v2.0.8-beta to process and I am getting some matches.

But I see this error in logs:

23-04-01 14:59:36 error: 1 validation error
23-04-01 14:59:36 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]

Should I ignore this error. Or is there a fix?

double-take config
double-take config2880×1795 491 KB

codeprojectai
codeprojectai2867×1795 464 KB 

Double Take
23-04-01 14:59:01 info: Double Take v1.13.1
23-04-01 14:59:01 error: 1 validation error
23-04-01 14:59:01 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]
23-04-01 14:59:01 info: MQTT: connected
23-04-01 14:59:03 error: 1 validation error
23-04-01 14:59:03 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]
23-04-01 14:59:03 error: 1 validation error
23-04-01 14:59:03 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]
23-04-01 14:59:04 error: 1 validation error
23-04-01 14:59:04 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]
23-04-01 14:59:11 error: 1 validation error
23-04-01 14:59:11 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]
23-04-01 14:59:36 error: 1 validation error
23-04-01 14:59:36 error: detectors is not any of [subschema 0],[subschema 1],[subschema 2],[subschema 3],[subschema 4]

23-04-01 15:03:13 info: andy: queuing 2 file(s) for training
23-04-01 15:03:14 info: andy: training complete in 0.64 sec
23-04-01 15:03:37 info: processing manual: fae7b335-49d6-4542-8139-9f0f11278679
23-04-01 15:03:39 info: done processing manual: fae7b335-49d6-4542-8139-9f0f11278679 in 1.48 sec
23-04-01 15:03:39 info: {
  id: 'fae7b335-49d6-4542-8139-9f0f11278679',
  duration: 1.48,
  timestamp: '2023-04-01T15:03:39.024Z',
  attempts: 1,
  camera: 'manual',
  zones: [],
  counts: { person: 1, match: 1, miss: 0, unknown: 0 },
  matches: [
    {
      name: 'andy',
      confidence: 81.04,
      match: true,
      box: [Object],
      type: 'manual',
      duration: 0.25,
      detector: 'aiserver',
      filename: '706195d5-1711-4cb4-b9dd-ccbdf7c0419b.jpg'
    }
  ],
  misses: [],
  unknowns: []
}
608

I am not the owner of the fork and not the coder of this enhancement.

I do not have any error in my setup (I use deepstack and codeprojectai) and I am not sure where this error comes from. Your configuration screenshot seems to be ok. Do you have “detectors:” twice?
Please re-check the whole configuration and compre this with the updated documentation:
GitHub - skrashevich/double-take: Unified UI and API for processing and training images for facial recognition.

609

Thanks for reply @S-Przybylski . I re-checked, I only have 1 detector listed in my double-take config. Sorry, I mistook you for the forked repo author.

I am glad to hear you do not have errors on your side.

Here is my config in full, except for password:

# Double Take
# Learn more at https://github.com/jakowenko/double-take/#configuration

mqtt:
  host: 192.168.11.213
  username: mosquitto
  password: XXXX
  
detectors:
  aiserver:
    url: http://192.168.11.211:32168
    # number of seconds before the request times out and is aborted
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
telemetry: false
    
      
detect:
  match:
    # save match images
    save: true
    # include base64 encoded string in api results and mqtt messages
    # options: true, false, box
    base64: false
    # minimum confidence needed to consider a result a match
    confidence: 70
    # hours to keep match images until they are deleted
    purge: 168
    # minimum area in pixels to consider a result a match
    min_area: 16000
    #when using substream image, 3600

  unknown:
    # save unknown images
    save: true
    # include base64 encoded string in api results and mqtt messages
    # options: true, false, box
    base64: false
    # minimum confidence needed before classifying a name as unknown
    confidence: 69
    # hours to keep unknown images until they are deleted
    purge: 8
    # minimum area in pixels to keep an unknown result
    min_area: 16000
610

Observation, if I only have aiserver detector, I get the above error.

But if I add a 2nd detector like Compreface (that I had used previously), the above error goes away:

compreface:
    #url: http://192.168.11.211:8002
    #key: 9e7ad273-5d3c-4ea1-af7a-af633bd91604
    url: http://192.168.11.214:8002
    key: 3375fbdc-c739-4824-9871-89645f611259
    # minimum required confidence that a recognized face is actually a face 
    #https://community.home-assistant.io/t/facial-recognition-room-presence-using-double-take-frigate/290943/585?u=actran70
    det_prob_threshold: 0.98
611

This could be the reason, why i do not see the error. I have currently two enabled…

1 Like
612

I found the problem. There is a typo in the file double-take/api/src/schemas/index.js:

 Line 113:    { required: ['aiseerver'] }, #is wrong
              { required: ['aiserver'] },  #corrected

I send a info to Sergey Krashevich…

1 Like
613

Thank you @S-Przybylski

614

Ooops. I corrected the mistake. You can just re-pull the container

1 Like
615

I downloaded your repo fresh to new directory and did:

docker build -f ./.build/Dockerfile -t double-take .

and got this error below…any ideas?

> [frontend-builder 10/10] RUN --mount=type=cache,target=/root/.npm npm run build:
#36 0.750
#36 0.750 > [email protected] build
#36 0.750 > vite build --minify false
#36 0.750
#36 1.059 NODE_ENV=production is not supported in the .env file. Only NODE_ENV=development is supported to create a development build of your project. If you need to set process.env.NODE_ENV, you can set it in the Vite config instead.
#36 1.087 vite v4.2.1 building for production...
#36 1.225
#36 1.225 ./themes/bootstrap4-dark-blue/theme.css doesn't exist at build time, it will remain unchanged to be resolved at runtime
#36 1.229 transforming...
#36 1.452 ✓ 9 modules transformed.
#36 1.453 ✓ built in 364ms
#36 1.453 [vite]: Rollup failed to resolve import "socket.io-client" from "/build/src/App.vue".
#36 1.453 This is most likely unintended because it can break your application at runtime.
#36 1.453 If you do want to externalize this module explicitly add it to
#36 1.453 `build.rollupOptions.external`
#36 1.456 error during build:
#36 1.456 Error: [vite]: Rollup failed to resolve import "socket.io-client" from "/build/src/App.vue".
#36 1.456 This is most likely unintended because it can break your application at runtime.
#36 1.456 If you do want to externalize this module explicitly add it to
#36 1.456 `build.rollupOptions.external`
#36 1.456     at onRollupWarning (file:///build/node_modules/vite/dist/node/chunks/dep-79892de8.js:46368:19)
#36 1.456     at onwarn (file:///build/node_modules/vite/dist/node/chunks/dep-79892de8.js:46138:13)
#36 1.456     at Object.onwarn (file:///build/node_modules/rollup/dist/es/shared/node-entry.js:25149:13)
#36 1.456     at ModuleLoader.handleInvalidResolvedId (file:///build/node_modules/rollup/dist/es/shared/node-entry.js:23784:26)
#36 1.456     at file:///build/node_modules/rollup/dist/es/shared/node-entry.js:23744:26
------
executor failed running [/bin/sh -c npm run build]: exit code: 1
616

Can someone please tell me how to get the CompreFace key for Double Take?

617

CompreFace has its own web UI when you set it up and when you setup an account it sets up a key for you.

618

Anyone with a picture of your face can unlock your door. I also have it set so this automation only works within the first 10 minutes of someone getting home.

619

Hello. I use
Double Take
MQTT
Frigate
Deep Stack

The year works without problems. I recently discovered that Frigate is marked in red in the configuration! Which means it’s not connected. BUT the system works, the camera sees the person and detects him.

What could be the problem?

2023-04-06_08-22-55
2023-04-06_08-22-55335×536 44.2 KB

2 Likes
620

Tell me please. One of the cameras stopped working in Frigate.

2023-04-10 15:10:53.431563271  [2023-04-10 18:10:53] frigate.video                  ERROR   : camera_1: Unable to read frames from ffmpeg process.
2023-04-10 15:10:53.432104632  [2023-04-10 18:10:53] frigate.video                  ERROR   : camera_1: ffmpeg process is not running. exiting capture thread...
2023-04-10 15:11:00.780775066  [2023-04-10 18:11:00] watchdog.camera_1              ERROR   : Ffmpeg process crashed unexpectedly for camera_1.
2023-04-10 15:11:00.780850148  [2023-04-10 18:11:00] watchdog.camera_1              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-04-10 15:11:00.780965677  [2023-04-10 18:11:00] ffmpeg.camera_1.detect         ERROR   : [rtsp @ 0x564644ef6e40] method DESCRIBE failed: 401 Unauthorized
2023-04-10 15:11:00.780997434  [2023-04-10 18:11:00] ffmpeg.camera_1.detect         ERROR   : rtsp://*:*@192.168.1.62:554/user=admin&password=XXX%211&channel=1&stream=0.sdp?: Server returned 401 Unauthorized (authorization failed)

The code in yaml looks like this:

mqtt:
  host: 192.168.1.134
  user: Alexey
  password: Alexey!1
  
cameras:
  camera_1: # <------ Name the camera
    mqtt:
      crop: True
      height: 500
    rtmp:
      enabled: false
    ffmpeg:
      input_args: -rtsp_transport tcp
      inputs:
        - path: rtsp://root:[email protected]:554/user=admin&password=XXX&channel=1&stream=0.sdp? # <----- Update for your camera
          roles:
            - detect
            - rtmp
            - record
            - clips
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
      fps: 14
    objects:
      track:
        - person
        - bicycle
        - car
        - motorcycle
        - bird
        - cat
        - dog
      filters:
        person:
          min_score: 0.6 # min score for object to initiate tracking (default: 0.5
          threshold: 0.8 # min decimal percentage for tracked object's computed score to be considered a true positive (default: 0.7)
        car:
          threshold: 0.85
          min_score: 0.6
      mask:
        - 880,73,884,32,1280,26,1280,67
        - 112,175,556,134,554,327,123,360
    zones:
      parking:
        coordinates: 470,315,264,334,190,569,413,685,710,680,883,540,1236,525,1241,307,875,150,461,175
        objects:
          - car
    record:
      enabled: True
      retain:
        days: 3
        mode: motion
      events:
        pre_capture: 15
        post_capture: 15
        objects:
          - person
        retain:
            default: 10
            mode: motion
621

How would I set false positives to unknown?

2 Likes
622

I getting the same error. I running window version of CodeProjectAI. Do I need to change something in their configuration file to make this work.

Greg

623

Where do I add this?

624

This is the yml file for your docker container. Typically in your docker-compose.yml or a specific one.