Person and face detection using image_processing with Sighthound

Share your Projects!
1

Hi all
just released a custom component for face and person detection with sighthound.com. Adds an entity where the state of the entity is the number of faces detected in an image. Person and face data are accessible as attributes, and events are fired that can be used to trigger automations.

To use this component you must register with sighthound to get an api key. The developer tier (free) allows 5000 requests per month, therefor you are advised to set a long scan_interval and call the scan service when you want to process an image, otherwise you will quickly burn through your 5000 requests as the default scan interval is 10 seconds. Please read the developer docs.

image

image
The image with bounding boxes is just to show what the component is recognising. If you are curious how to view bounding boxes see this repo.

5 Likes
2

How accurate are you finding the api?

I downloaded the free trial of Sighthound Video but found it did not work well with my wide angle fish-eye cameras.

3

That’s surprises me, you should raise the topic on their forums. Myself I’ve only briefly experimented with it but found it to be similar accuracy to other classifiers out there

4

I think it was the lens distortion that gave it trouble. I can get other corrected streams out of the camera. Might give it another go but the trial has expired on my test PC now.

1 Like
5

@robmarkcole I see that they offer vehicle model and color detection. Can we get this component to work with that?

6

I would create a seperate component for cars but since these components (all rest APIs) are very similar to one-another it is trivial to knock out another

7

@robmarkcole if you can that would be great. Would be nice to get notified when a sensor picks up a particular model car sitting in driveway. On a separate note how would you compare this component to your facebox one? Tried reading the documentation but its a bit beyond me. Do you have inx how to train faces for this component?

8

I see recognition of faces is possible, but functionality is identical (from what I can tell) to the Microsoft component - any reason which you would need this? My main interest in Sighthound is for person detection.

https://www.sighthound.com/docs/cloud/recognition/

9

Events
On each image processing, an image_processing.detect_face event is fired for each detected face, and an image_processing.detect_persons event is fired with the total number of detected persons. The events can be used to trigger automations, for example the following:

  • id: ‘11200961111’
    alias: Notify on person detection
    trigger:
    platform: event
    event_type: image_processing.detect_persons
    action:
    service: notify.pushbullet
    data_template:
    title: People detected
    message: ‘Alert: {{ trigger.event.data.total_persons }} persons detected by {{ trigger.event.data.entity_id }}’
    I put on automation
    not working
10

log errors?

11

I configured the same in the automation section

12

I cant offer any advice without an error or knowing your complete config…

13

Can this identify a person by name? Like Jason is found.
I have used sighthound before and it seems to do a good job.

14

As for the car, ALPR does this, the issue I ran into was there is no scan interval like this one so the cap is hit quite quickly.

15

@jasondreher ALPR does number plates only (from what I can tell), whilst Sighthound does License Plate, Make, Model, and Color of car.
Cheers

16

@robmarkcole I’ve set this up and its working well, email alerts are also being sent when a face is detected.

Any thoughts on how the image can be attached to the alert being sent? The below is what i have configured for attaching files generated by the folder_water integration, i’ve tried to adapt it to sighthound but to no avail

– id: SightHound
alias: SightHound Notify on person detection
trigger:
platform: event
event_type: image_processing.detect_persons
action:
- service: notify.email_steve
data_template:
message: “Alert: {{ trigger.event.data.total_persons }} people detected by {{ trigger.event.data.entity_id }}”
data:
images:
- “{{ trigger.event.data.path }}”

With this config the alert is emailed, but no attachment, i’m assuming i’m not passing through the image after the detection?

17

I’m overseas without computer access but checkout https://www.hackster.io/robin-cole/motion-activated-image-capture-and-classification-of-birds-6ef1ce

1 Like
18

Thanks alot, I reckon the last step in those instructions is what I missed doing. Will check it tomorrow and post back.

1 Like
19

I struggled with this a bit but couldn’t wrap my head around the template section. Got it working slightly differently through, with this flow ;

  1. Motion captures a image when motion is detected (picture_output best)
  2. Automation that watches for a new image and then sends that to be scanned for people
  • id: ‘88’
    alias: “Scan image after motion detected”
    initial_state: ‘on’
    trigger:
    platform: event
    event_type: folder_watcher
    event_data:
    event_type: created
    action:
    - service: image_processing.scan
    data_template:
    entity_id: “image_processing.sighthound_motion_{{ trigger.event.data.folder.split(‘/’)[2] }}”
  1. Automation that sends a telegram notification after the scan has completed, if a person is found. The file section is hardcoded to suit my setup, which is what more advanced HA user would have used a template for i guess…?
  • id: ‘1120092824611’
    alias: “Notify that a person has been detected”
    initial_state: ‘on’
    condition:
    condition: template
    value_template: “{{ trigger.event.data.total_persons > 0 }}”
    action:
    service: notify.telegram_steve
    data_template:
    message: “A Person has been detected”
    data:
    photo:
    - file: “/storagemotion/{{ trigger.event.data.entity_id.split(‘.’)[1].split(‘‘)[2] }}/lastmotion.jpg"
    caption: "We have detected {{ trigger.event.data.total_persons }} people at {{ trigger.event.data.entity_id.split(’.‘)[1].split(’    ’)[2] }} camera”
    trigger:
    platform: event
    event_type: image_processing.detect_persons

Here is what the notification looks like. The red box is drawn by motion, not by sighthound.

PNG

I’m still messing with it but it’s working nicely now. Thanks @robmarkcole for your efforts with this and the other image classification services, been really interesting learning more about them

2 Likes
20

I cant tell why this is not working,
-below is my config
-my camera is working
-my token is active in sighthound
-HA creates a entity named image_processing.sighthound_frontdoorcam
-the .py file is saved in custom_components\image_processing

here it is in a group
image

image_processing:
  - platform: sighthound
    api_key: AD63I7IkQxxxxxxxxxxxxxxxxxxxxx
    mode: dev
    state_display: faces
    scan_interval: 1000
    source:
      - entity_id: camera.FrontDoorCam

and when I look at sighthound settings it shows 0/5000 usage
Any ideas?