Help with Dlib facial recognision

Configuration
1

Hi guys,

I am trying to set facial recognition in my HA setup. I prefer to do it with an add-on running on HA over using another docker or externally.

I got the initial setup working with one camera and one face to recognize. it work pretty good.

Now I wish to add more cameras and more faces but I have two problems:

  1. Having one camera and multiple faces, results in recognizing only the first one.
  2. If I add more cameras, HA core crash every time I try to do that. I think it’s spiking the CPU and HA keeps restarting.

The documentation is not so informative in the CPU excessive section (the sensor suggested just don’t work, maybe I got it wrong), so I come to you guys for help.

There are 3 parts in the config: the platform in configuration.yaml, the cameras and the faces.

When I try to have them all in the same platform declaration, HA crash and reboot every few minutes.

image_processing:
  - platform: dlib_face_identify
    scan_interval: 10000
    source:
      - entity_id: camera.camera5
        name: DoorCamIdentify
      - entity_id: camera.camera1
        name: EntranceCamIdentify
      - entity_id: camera.camera2
        name: LivingroomCamIdentify
    faces:
      First: /config/www/FamilyPics/FirstPortrait.jpeg
      Second: /config/www/FamilyPics/SecondPortrait.jpeg
      Third: /config/www/FamilyPics/ThirdPortrait.jpeg

And than call the service once needed (I will set it later, maybe motion based decision)

- alias: "test face recognition"
  id: 'testreco'
  trigger:
    platform: state
    entity_id: input_boolean.trigger
    to: 'on'
  action:
    - service: image_processing.scan
      target:
        entity_id:
          - image_processing.doorcamidentify
          - image_processing.entrancecamidentify
    - service: input_boolean.turn_off
      entity_id: input_boolean.trigger

And have one of the following for each camera:

- alias: "Known person is detected"
  id: 'cam5persondetect'
  trigger:
    platform: event
    event_type: image_processing.detect_face
    event_data:
      entity_id: image_processing.doorcamidentify
      name: "{{ 'First' or 'Second' or 'Third' }}"
  action:
    - service: notify.mobile_app_naors_iphone13pmax
      data:
        message: >
          {{ trigger.event.data["name"] }}, Cam5 Gotcha!

What is the correct way to this? should I somehow split the configs?

Thank you guys in advanced!

2

If ha crash with more camera it’s possible that’s HA is having difficulty detecting even single face(takes long time) and miss detection if multiple face

In docker you can limit memory and cpu use of containers. This may help crashes if it’s caused by host server crash vs container crash.

3

Let me check if i follow…
Using a docket might fix the resources issue. And if a docker is now an option, I should also consider another facial recognition method? :worried:

4

Maybe
Not clear if Host crashing or container is crashing
My guess is if you not limit resources the host is crashing

This will definitely stop HA crashing. You are always going to be limited by your host performance

5

this does not make sense.

you should have 3 triggers defining each name or you can attempt to list them out but it may not work. If you use a condition, it will work

condition:
- condition: template
  value_template: "{{ trigger.event.data.name in ['First', 'Second', 'Third'] }}"
6

As a last resort, I might consider. but i wish to make sure that its not something that I do wrong

7

That might be it… because I reduced back to one camera and commented out the name field in event data section and it looks like it works with 3 faces (I just fired the trigger for any face and filtered it in the message).
So, I will add the template you suggested (to avoid complex multiple triggers), and see if it runs.
after that, ill try it with more cameras.

Thank you all for helping. Ill update after testing

Edit:
Now that I am thinking… in the configuration I declare the faces… so the event will only fire if they are detected. right?
So, I only need the name for deciding what to do (like sending a message with the detected name or opening a door).

In that case, I only need to add more cameras and see if it holds

8

After a test, I think that another automation is helping it crash…
After running the scan on multiple cameras, it immediately crashed.
I am separating the scan into 2 different automations, each one with only one camera, but I will try to fire them at the same time with a single trigger.
This will tell me if it is the multiple cameras in the config or the fact i am scanning two cameras at once

Edit:
OK, it is the resources… even when firing them in two different automations, the host crashed.

This means I need to keep it simple with one camera, or think on a different solution

9

How is HA installed?

10

@tmjpugh

its installed on a virtual box VM with allocation of 3 processors and 4GB memory