Face and person detection with Deepstack - local and free!

  1. Build a Ubuntu20.04 VM
    2.Install Docker
    Installieren und Verwenden von Docker unter Ubuntu 20.04 | DigitalOcean
    Compose V2 | Docker Documentation
  2. Install Portainer (optional)
    docker volume create portainer_data
    docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce
  3. Install HA Supervised
    GitHub - home-assistant/supervised-installer: Installer for a generic Linux system
  4. Create a 2nd VM or run Deepstack on a machine with gpu. Frigate consumes a lot of cpu without gpu.

hi all,

deepstack-ui is working very well so deepstack should be working fine, but deepstack_object
and deepstack_face won’t work in homeassistant.

i have done everything like robmarkcole has descriped on github and i watched the great video from everything smart home. but no detections on my image.

i think my problem is when i go in developer tools → service in homeassistant and start the image_processing.scan service manually…nothing happens, no detections, no response, nothing…

it would be great if someone could help me to get it working.

homeassistant supervised and deepstack are running in docker on debian 11 in an vm

edit: i have tried to use my camera and not the camera.local_fle and then it works ^^ but why not with the image filles?

make sure your local file camera is pointing to the config folder or to a directory included in the ā€œmedia_dirs:ā€ setting

thx for the reply, the local file was pointing on /config/www folder. very strange…but it works with my cams so i can ignore it i think ^^

Maybe you can tell me what I am doing wrong? :slightly_frowning_face:

I see the cams on frigate and I see events in frigate.
I see detection on docker

but in double take its show me ā€œno filesā€

this is the conf in double take

I have logs from double take

21-12-05 08:20:53 info: processing front: 704605e4-b3fc-469c-a73e-f624bfe24764
21-12-05 08:20:53 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:53 info: done processing front: 704605e4-b3fc-469c-a73e-f624bfe24764 in 0.01 sec
21-12-05 08:20:53 info: {
  id: '704605e4-b3fc-469c-a73e-f624bfe24764',
  duration: 0.01,
  timestamp: '2021-12-05T06:20:53.531Z',
  attempts: 1,
  camera: 'front',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:20:54 info: processing Door Camera: 1638685244.83641-xw0vbs
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 1)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 info: done processing Door Camera: 1638685244.83641-xw0vbs in 0.18 sec
21-12-05 08:20:54 info: {
  id: '1638685244.83641-xw0vbs',
  duration: 0.18,
  timestamp: '2021-12-05T06:20:54.269Z',
  attempts: 2,
  camera: 'Door Camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:21:02 info: processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9
21-12-05 08:21:02 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:21:02 info: done processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9 in 0.01 sec
21-12-05 08:21:02 info: {
  id: 'd343ebb2-3837-447c-91ba-19566bef2ad9',
  duration: 0.01,
  timestamp: '2021-12-05T06:21:02.125Z',
  attempts: 1,
  camera: 'door camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 10:51:37 info: Double Take v1.7.0-bf3e911
21-12-05 10:51:37 info: MQTT: connected
21-12-05 10:51:37 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:08:55 info: Double Take v1.7.0-bf3e911
21-12-05 11:08:55 info: MQTT: connected
21-12-05 11:08:55 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:15:58 info: Double Take v1.7.0-bf3e911
21-12-05 11:15:58 info: MQTT: connected
21-12-05 11:15:58 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

and that what I see in HA about the image_processing
4

this is the conf of image processing

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.199
    port: 80
    save_file_folder: /config/www
    save_file_format: jpg
    always_save_latest_file: True
    targets:
      - target: person
        confidence: 60
    source:
      - entity_id: camera.door_camera_2
      - entity_id: camera.front

and this is the conf of frigate

cameras:
  Door Camera:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


            
  Front:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


    
detectors: 
  cpu1:
    type: cpu
  cpu2:
    type: cpu

why its not working to me?

Deepstack addon?

I have Deepstack running as a docker container in HA supervised on a Debian host, so until recently that was a supported setup but now supervisor is complaining about unsupported software.

Converting the deepstack container to an add-on would fix that is my understanding. Has anyone tried this - unsure how difficult it is to add the existing image as a source to build an addon? One of the first hurdles would be to have volume mapping so it finds my custom model, though a deepstack folder in /share would work.

Many thanks!
Jhhbe

1 Like

in the terminal window you see |401| error.
for troubleshooting you can download the snapshot images in the frigate ui and try to use these with the deepstack ui application

image
i get the 400 error when the face detection can’t detect a face on the image

I just started using deepstack.
I’m finding the same results.

A lot of the time there are false positives. Even with high confidence levels sometimes.
I trained just 2 people with 20+ images, but there are still false positives in the sense that person 1 gets recognized as person 2. Does anyone know how to improve this performance?

Hey, i’m trying to set it up with Synology Docker too and it doesn’t seem to work. could you explain more on how you did it?

I installed the DeepStack docker pack on my Synology Docker, and added the image_processing details to HA, with the IP (of synology) and the port (of the deepstack docker), also tried to change to different port numbers and it didn’t change anything.

When I call the service-scan, I see no change in the logs/terminal on docker

I installed it on Synology Docker and added the config to HA.
Now the Object detection works well, the Face detection sensor remains ā€œunknownā€ instead of the face number.
When I send the command to scan the face counter, I can see the new line adding to the terminal on docker. so home assistant does communicate with it about face detection, it’s just that it doesn’t get the information back or the sensor doesn’t get updated for some reason…

any ideas?

same, do you have error in ha log ?

I ended up using another machine instead of synology, using Intel Nuc with dockers in Proxmox, works perfect

This fixed it for me thank you very much.

1 Like

Thank you Rob for the amazing work.

I managed to Install deep stack on HASS machine (since it already have docker) … I run it with face and object detection … all is good. it recognize faces, people after training … next step is various automations.

Hi all

I have deepstack running in docker and Motioneye integration setup in HA. I also have HASS deepstack object running from HACS. Below is config.yaml for image processing.

If I use developer tool to manually call image processing service, I do see the event fired in deepstack along w/ all the snapshots from cameras. However, it won’t automatically detect object; please HELP as I spent last couple of days searching thru Google and Youtube w/ not much luck

image_processing:

  • platform: deepstack_object
    ip_address: 10.0.19.241
    port: 80
    save_file_folder: /config/snapshots/
    save_file_format: png
    save_timestamped_file: True
    always_save_latest_file: True
    scale: 0.75
    roi_x_max: 0.8
    roi_y_max: 0.8
    targets:
    • target: person
    • target: vehicle
      confidence: 60
    • target: car
      confidence: 40
      source:
    • entity_id: camera.garage_cam

Excited to discover this addon which supports using google coral and integration with my deepstack components

I’ve been beating my head against the wall with Deepstack on synology (since that was the easiest docker place near my Hass NUC). Good to hear that I’m not the only one who finds it fails on that config. Will look to install on the NUC I guess…I have Frigate and DoubleTake working well, for the 10 minutes that Deepstack runs.

Hello,

I have Deepstack running only on PORT 80. If I change the port e.g. to PORT 81 the communication does not work anymore.

Do you have the same experience?

Hey Everyone,

For the last while I’ve been getting a 401 Unauthorized returned by the deepquestai/deepstack container when an image processing scan is called. Nothing as changed in my configuration and I’ve verified my API key is defined in the docker_compose.yaml as well as in the image_processing.yaml.

Docker log:

HASS log:

This error originated from a custom integration.

Logger: custom_components.deepstack_object.image_processing
Source: custom_components/deepstack_object/image_processing.py:349
Integration: deepstack_object (documentation)
First occurred: February 27, 2022, 12:00:29 AM (8 occurrences)
Last logged: 11:24:04 AM

Deepstack error : Error from Deepstack request, status code: 401

docker_compose:

version: "3.6"

services:
  deepstack:
    image: deepquestai/deepstack
    container_name: deepstack
    environment:
      - API-KEY="APIKEY_REDACTED"
      - VISION-DETECTION=True
      - VISION-FACE=True
    ports:
      - 5000:5000
    volumes:
      - /etc/docker/deepstack:/datastore
    restart: always

image_processing.yaml:

- platform: deepstack_object
  ip_address: 192.168.xx.xx
  port: 5000
  api_key: !secret deepstack_key
  save_file_folder: /config/www/snapshots/
  save_timestamped_file: True
  # roi_x_min: 0.35
  roi_x_max: 0.8
  #roi_y_min: 0.4
  roi_y_max: 0.8
  scan_interval: 10000
  targets:
    - target: person
      confidence: 80
    - target: car
      confidence: 65
    - target: truck
      confidence: 65
  source:
    - entity_id: camera.driveway

Anyone else run into this?

For me those image.processing sensors also seemed to stop working after upgrading to the newest release.

I found this thread https://www.reddit.com/r/BlueIris/comments/tmlxtc/bi_and_deepstack_nothing_found_after_move_to_new/

So i thought i should switch deepstack back to version 2021.09.01. But unfortunately the deepstack container on my jetson nano times out after 1 minute even if i trigger my image.processing sensors the api doesn’t seem to process those any more.

Homeassistant Logs just tell me that i reachted the 10 seconds timeout.

2022-04-14 20:42:04 WARNING (MainThread) [homeassistant.helpers.entity] Update of image_processing.deepstack_object_garage_cam is taking over 10 seconds
2022-04-14 20:42:05 ERROR (SyncWorker_1) [custom_components.deepstack_object.image_processing] Deepstack error : Timeout connecting to Deepstack, the current timeout is 10 seconds, try increasing

But all of this worked fine previously.
Can someone help me with this?

Regards,