Facial recognition & room presence using Double Take & Frigate

@thundergreen here’s the two nodes that are responsible for the MQTT in and creating the HA entity and updating it for a specific person.

Then this is what the flow looks like that’s responsible for sending the notification.

The first node is just a HA state change for the entity you created above. The function node contains this. Replace the http://double-take-url.com:port part with the URL and port you’re running the container on.

const { friendly_name, name, id, camera, confidence, type, detector, totalduration, attempts } = msg.data.new_state.attributes;
msg.data.url = `http://double-take-url.com:portstorage/matches/${name}/${id}-${type}.jpg`;
msg.message = `${friendly_name} is near the ${msg.payload} @ ${confidence}% by ${detector}:${type} taking ${attempts} attempt(s) @ ${totalduration} sec`;
return msg;

My service node looks like this and contains this for the data.
{"message":"{{message}}","data":{"attachment":{"url":"{{{data.url}}}"}}}

Let me know if you have any issues setting this up.

you can share the homeassistant profile for room presence

My sensor looks like this:

image

my config:

Don’t find the error :frowning:

error log Node-Red:

image
What does that mean ?

I get this as sensor if I change my node sensor state to: msg.payload only iso msg.payload.room

Make sure your MQTT in looks like this. Guessing your output needs to be a “parsed JSON object”

BNGO! :slight_smile:

image

Thanks a lot! I will watch notification now and try an automation to open my door if my face has been detected :slight_smile: If I am right the sensor will show the “room” … like this if sensor shows froont-door it will fire an automation to open the door . Another question just to make this awesome pice of softare more generic… would you consider to rename it to something like “location” or “area” iso room as I think many people would use it more for locations than rooms as i can hardly imagine that people have cameras in each room :slight_smile: (just an idea)

3 Likes

To show my respect and contribute to your project, I created an updated README . Feel free to merge :slight_smile: You can also maybe add it to your first post?

3 Likes

Glad you got it working! Yes, the sate of the entity in HA will just be the camera name in title case which is currently called room. i.e. front-door > Front Door.

I do like your idea of changing room to location. That’ll be easy to do, but will require you to update your Node-Red flow to set the HA entity to msg.payload.location.

Do you think I should remove the - and title case it still? I wanted to display the camera name in a friendly way, but I could offload that logic to HA or the consumer of the API and make them responsible for formatting.

Thank you so much! I will check it out this weekend and get it merged in.

If you have any other ideas let me know!

hey @lupac, my newest build includes the option to specify exactly which cameras you want to use for facial recognition.

See FRIGATE_CAMERAS in the options https://github.com/jakowenko/double-take#options

It would look something like this.

environment:
    FRIGATE_CAMERAS: living-room, office, basement

I didn’t get this part :stuck_out_tongue: well I thank you for being that open. I might come up with some ideas in the near future… To rename your App into location would make it more generic and just changing the payload won’t be that difficult :slight_smile: The Name of the room is completely ok for me and formatting won’t really impact the function. That said I mean that the sensor state will only be likely used as trigger. We dn’t care that much of this state name I think.

If you really want to blow your app up I’d have the idea that you App will be an Hassio Addon and maybe a custom component in addition with functions like:

→ chosing camera you want to use ( could also be cinfigurable via addon configuration )
→ automatically create entities for each camera
→ (overkill!!!) adding option to track persons / Objects only if person.xxx is home
→ adding configuration (or custom_component) to create notifications

… more to come :slight_smile:

In addition and compliance to frigate I’d rename your app Frigate compagion or Frigate Person Detection Addon or something in this way.

Hope it doesn’t overwhelm you but as I am no programmer but I would call me harcore Home Assistant User :slight_smile:

Thanks for all your ideas! I’ll take them all into consideration. A HA add-on is definitely something I’d like to look into.

The project was originally called frigate-events, but I want to expand this beyond just Frigate. It currently works with any URL of an image, so it’s not dependent on Frigate to use, Frigate is more of an integration for my project.

Great @Jako, thanks a lot!

I am going to test it now and report back.

1 Like

Added in support for DeepStack facial detection :rocket:

I had to stop the app as it crashed frigate as mentioned in the issue (connection …) … Since I stopped it no crashes anymore. did you have a look this too ? Would love to use this Appa again

@Jako don’t you want to merge my pull request? Updating readme with nodered example?

@thundergreen I haven’t been able to find a clear way to cause Frigate to crash. I have experienced it before, but for the last two weeks Frigate has been running very smoothly and hasn’t crashed once.

What’s your Frigate setup like? Are you using any of the accelerators or doing the processing on your CPU?

I will get your PR merged in, don’t worry. I just wanted to clean things up a little and expand on it a little before doing so.

Hello using coral pcie as detector .but dunno if this is related? Thanks for merging my or.but take your time and better make things clean and in order :slightly_smiling_face: I deactivated your app and since then no crashes anymore.so it must be related I think somehow.

I’ve made quite a few UI updates over the past few weeks and the root of the API at http://YOUR-IP:PORT will now serve a frontend that shows all matches and misses from the detectors. You can either retrain the models with the images or delete them. More updates to follow!

2 Likes

:+1:

I got my Jetson Nano today to play about with Double Take to bridge Frigate and Deepstack :slight_smile:

1 Like

Did you also get a look into the problem where frigate would crash using your app?

Not much I can do about it since it’s on the Frigate side. Seems like downgrading Frigate fixes it, I just restart frigate when I see it happen. Hoping their 0.9.0 release fixes it.