Facial recognition & room presence using Double Take & Frigate

Jako · June 30, 2021, 2:41am

@ppluti007 I also updated the config page on the UI to show the error when you hover over the icon. Here’s an example with the incorrect API key.

Eoin · July 2, 2021, 6:25am

@Jako This might be a silly question, but I have read contradicting information about DeepStack. If I associate 1 image to a person in double-take then the next day associate a new image does it add to the existing person profile or does old profile get wiped and will only consist of the new image?

Hoping that make sense, adding images extends the profile or profile will only consist of the latest images associated.

sender · July 2, 2021, 7:33am

“what” should I upgrade to see this and where?

ppluti007 · July 2, 2021, 2:42pm

Hi
i did some test and it works without any quote or doublequote around the mysecret.

i have in same docker-compose.yaml both deepstack and deepstack-ui a simple GUI for deepstack. They are in the same yaml file and in every combination they works fine toghether, but double-take only works if i omit the quotes:

---------------------------1st test => KO---------------------------
docker.compose.yaml

[...]
services:
  deepstack:
    environment:
      - API-KEY="mypassword"
[...]
  deepstack_ui:
    environment:
      - DEEPSTACK_API_KEY="mypassword"

double-take config.yml

[...]
detectors:
  deepstack:
    key: "mypassword"

---------------------------2nd test => KO---------------------------
docker.compose.yaml

[...]
services:
  deepstack:
    environment:
      - API-KEY='mypassword'
[...]
  deepstack_ui:
    environment:
      - DEEPSTACK_API_KEY='mypassword'

double-take config.yml

[...]
detectors:
  deepstack:
    key: 'mypassword'

---------------------------3th test => IT WORKS---------------------------
docker.compose.yaml

[...]
services:
  deepstack:
    environment:
      - API-KEY=mypassword
[...]
  deepstack_ui:
    environment:
      - DEEPSTACK_API_KEY=mypassword

double-take config.yml

[...]
detectors:
  deepstack:
    key: mypassword

Do the password model have lower performance or there is low impacts? As we said we are in local lan, if it impact performance maybe i will avoid the password using.

ppluti007 · July 2, 2021, 3:56pm

for testing the beta version of double-take you have to use the beta docker image

image: jakowenko/double-take:beta

ppluti007 · July 2, 2021, 4:10pm

Sorry, i read it only just now.

i take this docker-compose as a sample:

github.com

robmarkcole/deepstack-ui/blob/e515d4d6670f2818a44fc009a5a1efc8ccbc1eb2/docker-compose.yml

version: "3.9"
services:
  deepstack:
    container_name: deepstack
    restart: unless-stopped
    image: deepquestai/deepstack
    ports:
      - '5000:5000'
    environment:
      - VISION-DETECTION=True
      - API-KEY=""
    volumes:
      - /Users/robin/Data/pytorch-models:/modelstore/detection
  deepstack_ui:
    container_name: deepstack_ui
    restart: unless-stopped
    image: deepstack-ui
    environment: 
      - DEEPSTACK_IP=deepstack
      - DEEPSTACK_PORT=5000

This file has been truncated. show original

Bye

Jako · July 2, 2021, 4:14pm

Hey @Eoin, I haven’t heard about that contradicting information. My understanding is that every image you add will get added to the user’s collection. So you should be fine adding one at a time.

@sender, the jakowenko/double-take:beta image that ppluti007 referenced above contains all the latest changes I’ve been posting about. v0.8.0 is building right now on Github, so the latest Docker tag should have all the updates within the hour.

@ppluti007 glad you got it working. I don’t think the password should impact performance, but I don’t know what DeepStack does on the backend with it. I run mine without a password since it’s only accessible locally.

I also created a Discord server if anyone ever wants help or has questions

zil0g · July 8, 2021, 1:42pm

Not sure what I’m doing wrong… installed a doorbell cam yesterday, 2 new people were visible (postman, and a friend) but I can’t see their snapshots in the AI? (already recognised people snapped by the doorbell cam are visible). I click on the “settings” button, and there doesn’t seem to be any option to filter by “unknown” (just the 2 people already registered on the system)?

Jako · July 8, 2021, 7:58pm

Hey @zil0g, the UI currently only shows the last 100 images and the dropdown filters only show the users found within those images displayed. I should fix this though so you can filter on everything even ones that aren’t displayed. If you want to open an issue on Github, I can try to prioritize this.

I’m wondering if you have more than 100 images and the unknowns are just further down on the list. Try deleting the current ones displayed, there’s a check all button, and see if they show up.

If that doesn’t work, I can try to help you on Discord.

zil0g · July 8, 2021, 8:12pm

thanks for the reply! Yes, I’ve already worked out you only get 100 images at a time. I’ve just gone through your docs again and realised that, following the example config, that unknown images are purged after only 8 hours which means they’d have been deleted before I got a chance to go through them! I’ve increased this to 48hrs now, I think that should solve the problem! Yes it would be great though to filter ALL of the images and quickly display any unknowns, that’s a good idea, I will open an issue.

Jako · July 8, 2021, 8:35pm

Glad you figured it out, I forgot about the purge time, so good catch on checking that.

Thanks for your support with this project, if you have any feature requests feel free to let me know!

zil0g · July 13, 2021, 12:46pm

@Jako would it be possible to include the Frigate event ID as part of the Double-Take message? This would allow tighter integration with Frigate and solve a few problems I think, e.g. I’ve had scenarios where Compreface returns different names from different snapshots during the same motion event from Frigate, but the name with the highest % is usually the correct one. If I could tie all of these detections to the same event, the lower-scoring incorrect “matches” could be discarded.

Jako · July 14, 2021, 4:19am

@zil0g, the Frigate ID should be included in the API response and MQTT messages. Is it not coming through for you?

I like the idea of taking the best result from each Frigate ID to increase the changes of a correct match. This could make some detections slower though as it wouldn’t be as close to real time since the entire Frigate event would have to end.

zil0g · July 14, 2021, 2:18pm

@Jako thanks, I looked at the docs and the IDs looked like different formats, so I assumed it was a separate, DT generated one, although I didn’t check. Will have a play about with it later. Yes it isn’t necessary the best approach for every situation, but would be good for some (including presence detection I think). The other idea I had was for a HA wall display (or Magic Mirror) with a camera, that could display relevant information depending on who looked at it. Then by tracking Frigate’s “End” event associated with a particular Face detection, it would know when the person who activated it had left, so the screen could go blank again, or return to normal display, etc.

Jako · July 14, 2021, 5:44pm

You’re right, the docs show a UUID v4 which is generated if the API is hit directly. I’ll make sure to include a note that the Frigate ID is used for all Frigate events.

I’ll have to think about the best way to handle your magic mirror example. I feel for something like that multiple images should be processed for an event and taking the best result before considering it a match. Then when the end event is fired it would need to clear out that result from the camera to allow the mirror to go back to an idle state.

If you want to discuss or brain storm, feel free to message me on Discord.

Jako · July 15, 2021, 1:27pm

The newest beta build includes a lot of cool features for the training page. I would love some feedback and suggestions if there are any.

This is how it’s currently setup.

The red trash button on the right side untrains / deletes images from multiple subjects. If the image is currently trained it will remove that subject entirely and retrain it with the other trained images.

Screen Shot 2021-07-15 at 3 07 25 AM

If the image is untrained by doing the previous step and you click the same button it will then remove it.

Screen Shot 2021-07-15 at 3 07 48 AM

So a trained image goes from trained → untrained → deleted

If you select a name from the dropdown you can do one of three things.

Upload a new image from your file system. You can even use your phone camera which will allow you to use some high quality photos for training.
Sync and retrain the model. This deletes the current user from the configured detectors and then takes all the images from the /train/${name} folder to retrain the model.
Ability to completely untrain a user. This will not remove the images, but they will go into the untrained state and then can be selected and deleted if wanted.

Jako · July 21, 2021, 7:25am

Just published v0.9.0 with these noteworthy additions.

Sensors will automatically be created in HA now as long as the MQTT integration is enabled.
Train page with functionality to mass train, untrain, or upload new images. This makes adding / removing images for users very easy now from the UI.

sender · July 21, 2021, 8:06am

Hi Jako, is this also including the fix to show the images again (currently in beta)?

Eoin · July 21, 2021, 9:35am

Amazing release, very cool! Thx for all your effort.

ravensteijn · July 21, 2021, 12:55pm

Thnxs for this amazing release! it’s working when i upload an image. But when i select an image(green selected) from the matchpage and add it to a person the front end says. 1 file trained for … and everything looks ok.

But when i look at the logs it says:

error: connect ECONNREFUSED 127.0.0.1:80
info: MYNAME: queuing 0 file(s) for training

i’m using ip adresses like 192.168.1.7 etc. with portnumbers range : 5000 etc. no 80. Where do i need to change this portnumber?