Facial recognition & room presence using Double Take & Frigate

I have a video doorbell and would like to add face recognition. I have double take running along with mqtt and frigate on an rpi 4b 8gb. Is it possible to get deepstack, compreface or face box running. Any help would be much appreciated. Thanks :grinning_face_with_smiling_eyes: .

I have an Odroid N2+/HA Blue. Frigate will definetely run, Deepstack typically times out for me, so Double Take will not receive the (MQTT) images. Video Hardware acceleration for Pi is better, so this might work depending on how many cameras you use…

… but maybe I’m just doing something sub optimal myself. Just give it a try?!

Yes I have but I can’t figure out to get deep stack working
When I write the image processing config I get an error saying that the deep stack platform doesn’t exist

Can you share your config with us, here?

mqtt:
  host: 192.168.0.219
  username: homeassistant
  password: *******************************

frigate:
  url: http://192.168.0.219:5000

detectors:
  deepstack:
    url: http://192.168.0.219
    key:

image
Deepstack Log:

DeepStack: Version 2022.01.01
/v1/vision/face
---------------------------------------
/v1/vision/face/recognize
---------------------------------------
/v1/vision/face/register
---------------------------------------
/v1/vision/face/match
---------------------------------------
/v1/vision/face/list
---------------------------------------
/v1/vision/face/delete
---------------------------------------
/v1/vision/detection
---------------------------------------
---------------------------------------
v1/backup
---------------------------------------
v1/restore

Double Take cannot find DeepStack. Try adding :portnumber here, or add a screenshot of the configuration-page of the DeepStack AddOn that specifies the port number, in the end it’ll be similar to how you’ve indicated the Frigate portnumber.

Good morning everyone, I have been reading for a long time and now I have to tell you my problem.
No matter where I searched, I found nothing about this problem.
To the point:
I have everything set up. MQTT, Double-Take.
At the beginning everything ran normally, I have given the matter no more attention, because I was busy with other things. Yesterday I set up my Reolink Doorbell. Another Reolink Camera I have removed, this was also set up with Homeassistant.
Now for the problem I get on the user interface of Double-take directly displayed:
cannot read property of undefined (reading ‘id’)
Recognize it does everything even with names but I am shown a match, in the filter I can also select only Miss.
I have MQTT, double-take also reinstalled, but without success. The sensor by case sensor.marco… is not created at all unknown however already.
Can it be that the no longer paired camera makes problems.
I really do not know what to do.

@shasta this has been answered. See Facial recognition & room presence using Double Take & Frigate - #647 by Townsmcp

1 Like

Thanks for the message, I have looked at this and also installed the repo, the error is no longer displayed but I still have the problem because of the Match and Miss error the sensor “Me” is not created and thus still have the error that only Miss is displayed.
deepstack recognizes me, but the sensor is not created.

I also just noticed that no images are read in automatically at all, only manually although the config file was created correctly and everything is displayed in green.

Trying to use double take to open the gate. Have the following automation

- id: vto_open_door
  alias: VTO Open door
  mode: single
  trigger:
  - platform: state
    entity_id: sensor.double_take_kalitka
  condition:
  - condition: or
    conditions:
      - condition: template
        value_template: '{{trigger.payload_json["matches"][0]["name"] == "andrew"}}'
      - condition: template
        value_template: '{{trigger.payload_json["matches"][0]["name"] == "julia"}}'
  action:
  - service: dahua.vto_open_door
    data:
      door_id: 1
    target:
      device_id: 8cdf104fd3e6b65ba6380c4c222edxxx

In the logs get the following error and automation never fires up

Error evaluating condition in 'VTO Open door': In 'condition': In 'or' (item 1 of 2): In 'template' condition: UndefinedError: 'dict object' has no attribute 'payload_json' In 'or' (item 2 of 2): In 'template' condition: UndefinedError: 'dict object' has no attribute 'payload_json'

Any help appreciated

They say to put masks on. And what to do with this? :rofl: :rofl: :rofl:


Defines the car as a person. Is he a Transformer? :grinning_face_with_smiling_eyes:

verify training images for Slava and mother-in-law. try to re-train (“Sync” button in “train” tab) if all images don’t looks like as vehicle )

1 Like

Am running codeproject and doubletake addons by @skrashevich with a minimal config (see below). Double Take however crashes seconds after being started with no real clue on why. This is, by the way, the same behaviour as when I tried the original version from Jakowenko.

Does anyone have a clue on what’s happening and how this can be prevented?

mqtt:
  host: 192.168.1.10
  username: homeassistant_mqtt
  password: homeassistant_mqtt

frigate:
  url: http://192.168.1.10:5000/
  
detectors:
  aiserver:
    url: http://192.168.1.10:5003
    timeout: 15
    # require opencv to find a face before processing with detector
    opencv_face_required: false
    # only process images from specific cameras, if omitted then all cameras will be processed
    # cameras:
    #   - front-door
    #   - garage

cameras:
  deurbel:
  
logs:
  level: debug

Relevant log:

23-08-12 08:21:08 info: Double Take v1.13.10
23-08-12 08:21:08 verbose: {
  telemetry: false,
  auth: false,
  detect: {
    match: {
      save: true,
      base64: false,
      confidence: 60,
      purge: 168,
      min_area: 10000
    },
    unknown: {
      save: true,
      base64: false,
      confidence: 40,
      purge: 8,
      min_area: 0
    }
  },
  time: { timezone: 'UTC' },
  frigate: {
    attempts: { latest: 10, snapshot: 10, mqtt: true, delay: 0 },
    image: { height: 500 },
    labels: [ 'person' ],
    update_sub_labels: false,
    stop_on_match: true,
    min_area: 0,
    url: 'http://192.168.1.10:5000/'
  },
  mqtt: {
    protocol: 'mqtt',
    tls: {},
    topics: {
      frigate: 'frigate/events',
      matches: 'double-take/matches',
      cameras: 'double-take/cameras',
      homeassistant: 'homeassistant'
    },
    host: '192.168.1.10',
    username: 'homeassistant_mqtt',
    password: '********'
  },
  logs: { level: 'debug' },
  ui: {
    path: '',
    theme: 'bootstrap4-dark-blue',
    editor: { theme: 'nord_dark' },
    logs: { lines: 500 },
    pagination: { limit: 50 },
    thumbnails: { quality: 95, width: 500 }
  },
  server: { port: 3000 },
  detectors: {
    aiserver: {
      timeout: 15,
      opencv_face_required: false,
      url: 'http://192.168.1.10:5003'
    }
  },
  cameras: { deurbel: null },
  storage: {
    path: '/config/double-take',
    config: { path: '/config/double-take' },
    secrets: { path: '/config', extension: 'yaml' },
    media: { path: '/config/double-take' },
    tmp: { path: '/dev/shm/double-take' }
  },
  version: '1.13.10'
}
23-08-12 08:21:09 verbose: api listening on :3000
23-08-12 08:21:09 info: MQTT: connected
23-08-12 08:21:09 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
23-08-12 08:21:10 verbose: Incominng event from frigate: {"before": {"id": "1691817339.236351-x1w2p5", "camera": "deurbel", "frame_time": 1691821209.91682, "snapshot_time": 1691819097.594738, "label": "car", "sub_label": null, "top_score": 0.84375, "false_positive": false, "start_time": 1691817339.236351, "end_time": null, "score": 0.84375, "box": [922, 364, 1074, 433], "area": 10488, "ratio": 2.2028985507246377, "region": [842, 222, 1162, 542], "stationary": true, "motionless_count": 19346, "position_changes": 1, "current_zones": [], "entered_zones": [], "has_clip": true, "has_snapshot": false}, "after": {"id": "1691817339.236351-x1w2p5", "camera": "deurbel", "frame_time": 1691821270.207675, "snapshot_time": 1691819097.594738, "label": "car", "sub_label": null, "top_score": 0.84375, "false_positive": false, "start_time": 1691817339.236351, "end_time": null, "score": 0.84375, "box": [922, 364, 1074, 433], "area": 10488, "ratio": 2.2028985507246377, "region": [842, 222, 1162, 542], "stationary": true, "motionless_count": 19647, "position_changes": 1, "current_zones": [], "entered_zones": [], "has_clip": true, "has_snapshot": false}, "type": "update"}
23-08-12 08:21:10 verbose: 1691817339.236351-x1w2p5 - car label not in (person)

Relevant log from Mosquitto:

2023-08-12 07:35:49: Client 2QhVSE2EzrGW0mNCh496d5 closed its connection.
2023-08-12 07:36:03: New connection from 172.30.32.1:41597 on port 1883.
2023-08-12 07:36:03: New client connected from 172.30.32.1:41597 as 1Up3s6Gr3t5mNA39HwshRh (p2, c1, k60, u'homeassistant_mqtt').
2023-08-12 07:38:13: New connection from 172.30.32.1:36908 on port 1883.
2023-08-12 07:38:13: New client connected from 172.30.32.1:36908 as double-take-962f52ab (p2, c1, k60, u'homeassistant_mqtt').
2023-08-12 07:38:19: Client double-take-962f52ab closed its connection.
2023-08-12 07:50:53: New connection from 172.30.32.1:54378 on port 1883.
2023-08-12 07:50:53: New client connected from 172.30.32.1:54378 as double-take-36416b73 (p2, c1, k60, u'homeassistant_mqtt').

Running supervised? Double take crashes on my installation too. Never could figure what is wrong. In the end I installed it on a different host system. Not of much help, I know.

Yep! Running it Supervised on a fresh, and supported, Debian 12 system.

You could try and use the container from the GitHub - skrashevich/hassio-addons repro. I haven’t tried that, but it now runs fine for me.

Already tried, but same result

I believe I managed to make it work thanks to a cryptical error message from codeporject.ai in supervisor logs.

If you look at the configuration of the module it displays 2 options (see below)

But if you edit it in yaml you will see the third one (data dir)

I manually created the mentioned folders and restarted codeproject addon and now also my doubletake addon continues to work :smiley:

EDIT: the above indeep keeps Doubletake up and running, but as soon as something needs to be done it crashes again… Still believe that might also be something with folders/permissions to folders…

EDIT2: decided to install DoubleTake via Docker. Works perfectly, only drawback is that my installation is now no longer supported (but at least I have an unsupported, working system instead of a not working supported one)

Has anyone been able to get a successful automation in HA. I’m trying to set one up that triggers when my wife or I come home and double-take confirms with 90% or higher facial recognition and the action is to unlock the front door.

Hi. Everything seems to be working. Just need to pick up a good camera.)))