Facial recognition & room presence using Double Take & Frigate

@juan11perez would the current MQTT topic payloads work for the HTTP payload? Do you have some flexibility on the gotify side based on what the content is? If so, that should be a pretty easy feature to get added in.

Anything in the matches folder will be accessible from localhost:PORT/api/storage/matches/, but the images from here will be deleted based off the purge.matches and purge.unknown values.

@Jako thank you again.
For gotify if your proposal is not difficult to add, perhaps you can add it and i’ll test it. I’m not very knowledgeable on the subject and I’ve been learning like most enthusiast; by research, trial and error. ​
Please forgive my limited knowledge.

For the jpgs Im aware of localhost:PORT/api/storage/matches/ and it works provided you’re in the local network.
When outside the notification will arrive without the jpg.
The way it’s addressed in HA is by setting up a reverse proxy, so I’ve setup a https://doubletake.mydomain.com so I can access it externally, but it’s obviously password protected via nginx.

If I substitute

localhost:PORT/api/storage/matches/ with https://doubletake.mydomain.com/api/storage/matches/

it needs a password to access.

Like I said I’m not knowlegeable enough and may be missing something obvious.

According to the Gotify API docs, it looks like if a title and message are POSTed to a the URL, then it should work. Maybe I’ll add support for Gotify directly, rather than a generic HTTP endpoint, since Gotify expects a certain payload. I just don’t have any Android devices, so I’ll need your help to test this. Can you look at that POST /message endpoint and see if that will accomplish what you need?

I have my /matches folder public to the outside network right now, but the images get deleted within an hour. That allows me to get the image in the notification. But I’m sure there’s others way you can accomplish it too if you want to add more security to it.

@Jako Thank you
I can definitely test it. I was using a scrip with zoneminder.
Basically I put the below script in say the config folder and it fires upon receipt of info from the end point.

#!/bin/bash
ZM_TOKEN='<>'
EVENT_ID=`echo ${6} | awk -F'/' '{ print $8 }'`
MESSAGE=`echo ${4} | sed -e 's/.*] \(.*\)Motion.*/\1/'`

if [[ "$MESSAGE" != "Motion: All" ]] && [[ "$3" == "Doorbell" ]]; then

  curl --request POST \
    --url 'https://goti.<>.com/message?token=<token>' \
    --header 'content-type: application/json' \
    --data "{
        \"title\": \"zm136 - $3\",
        \"message\": \"$MESSAGE\n\n ![Camera Image](https://zm/zm/index.php?view=image&eid=${EVENT_ID}&fid=objdetect&popup=1&token=${ZM_TOKEN})\",
        \"priority\": 6,
          \"extras\": {
        \"client::display\": {
          \"contentType\": \"text/markdown\",
        \"client::notification\": {
          \"click\": \"{ 'url': 'https://zm/zm/cgi-bin/nph-zms?mode=jpeg&frame=1&replay=none&source=event&event=${EVENT_ID}&connkey=77493&token=${ZM_TOKEN}'}\"
        }
      }
    }
  }"

thank you for the advise on the matches folder. I’ll review what Im doing wrong.

1 Like

Thanks, I’ll work to include this in the 0.7.0 release. Do you think the gotify notifications should be global or configured to work on a per camera / zone basis?

@Jako
I’d suggest global and then the user can apply some filter, like in my example, if they only want a specific notification.

In reality this is not needed as the HA notifications work fine (don’t want to waste your valuable time). The advantage of having gotify is that you can get direct notifications independent of HA.

Sorry to ask about matches folder again, but could you explain how you actually expose that folder to the outside? Thank you

@juan11perez, I use Traefik as a reverse proxy to expose the images.

I just pushed a new beta build (jakowenko/double-take:beta) to include the Gotify service support. I was able to test that the notifications work, but can’t test that the image is included since it appears to only be supported on Android devices. Let me know if you can test this and the results.

I’ve included options for only specific cameras or zones.

Here’s a basic example and a more complex one. Let me know what you think!

notify:
  gotify:
    url: http://localhost:8080
    token: XXXXXXXX
notify:
  gotify:
    url: http://localhost:8080
    token: XXXXXXXX
    priority: 3
    cameras:
      - living-room
      - backyard
    zones:
      - camera: driveway
        zone: zone-1
      - camera: driveway
        zone: zone-2

I wasn’t sure what to put for the default message in the notification, but open to suggestions. Maybe I can make it configurable too.

wow, nice. The Gotify web ui will render a jpg if provided along with the message.

I tested the beta and I’m getting similar notifications to yours; i.e. without jpg. Neither on the phone nor webrowser.

To send a jpg in gotify it uses the extra parameters. Check yellow highlighted text in the link.

For example when sending a doubletake message via HA using gotify my message is as follows:

 message: >
   {{trigger.payload_json['match']['name']|title}} detected with confidence {{trigger.payload_json['match']['confidence']}}%
   ![Camera Image](http://192.168.1.100:3020/api/storage/matches/{{trigger.payload_json['match']['filename']}})

Top line is a the message. Bottom line is the jpg in the format ![](img url)

This is how the gotify component is set up in the notify.yaml in HA

- platform: rest
  name: juan_gotify
  resource: https://goti.<>.com/message
  method: POST_JSON
  headers: 
    X-Gotify-Key: !secret gotify_frigate_key
  message_param_name: message
  title_param_name: title
  data:
    priority: 6  
    extras:
      client::display:
        contentType: "text/markdown"

The method needs to be POST_JSON in HA for the jpg to download.

I hope I’ve help clarify.

I think your basic message is sufficient. I appreciate the more options you introduce the more maintenance work.

I apologise for repeating the subject, but the matches folder accessibility is a limitation.
As far as I can understand the file can only be made accessible externally via a reverse proxy with https, unless the port is forwarded and open, which would give full access to doubletake and would stop working when your ip changes.

I use nginx proxy manager and I can proxy doubletake, but then I have to set up an nginx password to prevent unwanted access to the ui.

When i do this, the message cant download the jpg.

In home assistant, the site is secured itself, but the the www folder is exposed regardless.

@juan11perez

I do pass the extra params with the request. Here’s the actual code for it, but I wasn’t able to get the image to show up. So I can look into that more.
Code Ref: double-take/gotify.js at 434d5b4a61a1792c74b3c685e5799fd97ab16b55 · jakowenko/double-take · GitHub

Feel free to message me on Discord, Jako#0593 and I can try to help you more with the reverse proxy as well.

@Jako
That’s because you’re passing the image in the wrong place.
Your code has it in “client::notification”:
That’s not for images, that’s to open urls when clicking the notification.

the image has to go in the message section, like in my above example

  \"message\": \"$MESSAGE\n\n ![Camera Image](https://zm/zm/index.php?view=image&eid=${EVENT_ID}&fid=objdetect&popup=1&token=${ZM_TOKEN})\",

I think your code would need to be something like this:

  return axios({
    method: 'post_json',
    url: `${GOTIFY.URL}/message?token=${GOTIFY.TOKEN}`,
    data: {
      message: message ![Camera Image](http://0.0.0.0:${SERVER.PORT}/api/storage/matches/${filename}?box=true),
      priority: GOTIFY.PRIORITY,
      extras: {
        'client::display': {
          contentType: 'text/markdown',
        },
      },
    },
  });
};

Ah I see, thank you. So I can keep it in client::notification to make it open the image when you click it, but it also needs to be passed in the message prop. I’ll try to get that fixed tonight.

yes. That’s correct.

1 Like

@juan11perez pull the latest beta and give it a try now. I’m using base64 encoded images now. I think this will fix your reverse proxy problem too since there wouldn’t be a need for it. The image in the notification and the link are both base64.

@Jako
Perfect! this is fantastic. works great.
And you’re right, no need to reverse proxy. I get the image locally and remotely.
It’s practical to be able to get notifications regardless of HA availability.

Thank you very much for considering and implementing this suggestion.

Awesome! I’m glad it’s working for you. Thank you for the great suggestion and let me know if you have any others.

Once again thank you. Now I just need to find a nice icon for your app. You can change the icon in the gotify web ui, where you copied your key.

One more question, I read in your related frigate discussion post, you’ve had better experience with compreface?

Is that the MQTT image that’s being processed? One thing I would suggest is to alter the defaults from Frigate. This is what I have configured for each camera. Making it larger will produce better results.

mqtt:
   timestamp: False
   bounding_box: False
   crop: True
   height: 500

Yes, CompreFace has been the best for me so far, then probably DeepStack. I do like how DeepStack supports object recognition, I’d like to eventually add support for that down the line. CompreFace seems to be pretty actively developed too which is a plus.

I would run a couple of them at the same time if I were you and see what works best. Then once you’re happy you can just run the one.

Yes, it’s the MQTT. I think the problem is that this is a 720p resolution camera. So not that great.
What I’ve noticed is that if you’re closer, the frame is wider/better.
I’ll try your suggested settings. Thank you.
And thank you for the detector tip.

1 Like

Hi @Jako , just managed to get this up & running (a lot of trial & error as I don’t really know what I’m doing :rofl: ) Very impressed so far though.

Can you clarify where I’m supposed to put the section you quoted, to alter the defaults from Frigate? In the frigate.yml file or the Double Take config file? (if the latter, could you show it in context as I’m not sure where about it should go?) Thanks!

Hey @zil0g, glad you got it working. If you run into any problems let me know!

The MQTT section above is from the Frigate config.yml file.

Here’s what it would look like

garage:
    objects:
      track:
        - person
    ffmpeg:
      inputs:
        - path: rtsp://stream-url.com
          roles:
            - detect
    width: 1280
    height: 720
    fps: 5
    snapshots:
      enabled: True
      bounding_box: True
    mqtt:
      timestamp: False
      bounding_box: False
      crop: True
      height: 500