Face and person detection with Deepstack - local and free!

  1. Build a Ubuntu20.04 VM
    2.Install Docker
    Installieren und Verwenden von Docker unter Ubuntu 20.04 | DigitalOcean
    Compose V2 | Docker Documentation
  2. Install Portainer (optional)
    docker volume create portainer_data
    docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce
  3. Install HA Supervised
    GitHub - home-assistant/supervised-installer: Installer for a generic Linux system
  4. Create a 2nd VM or run Deepstack on a machine with gpu. Frigate consumes a lot of cpu without gpu.

hi all,

deepstack-ui is working very well so deepstack should be working fine, but deepstack_object
and deepstack_face wonā€™t work in homeassistant.

i have done everything like robmarkcole has descriped on github and i watched the great video from everything smart home. but no detections on my image.

i think my problem is when i go in developer tools ā†’ service in homeassistant and start the image_processing.scan service manuallyā€¦nothing happens, no detections, no response, nothingā€¦

it would be great if someone could help me to get it working.

homeassistant supervised and deepstack are running in docker on debian 11 in an vm

edit: i have tried to use my camera and not the camera.local_fle and then it works ^^ but why not with the image filles?

make sure your local file camera is pointing to the config folder or to a directory included in the ā€œmedia_dirs:ā€ setting

thx for the reply, the local file was pointing on /config/www folder. very strangeā€¦but it works with my cams so i can ignore it i think ^^

Maybe you can tell me what I am doing wrong? :slightly_frowning_face:

I see the cams on frigate and I see events in frigate.
I see detection on docker

but in double take its show me ā€œno filesā€

this is the conf in double take

I have logs from double take

21-12-05 08:20:53 info: processing front: 704605e4-b3fc-469c-a73e-f624bfe24764
21-12-05 08:20:53 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:53 info: done processing front: 704605e4-b3fc-469c-a73e-f624bfe24764 in 0.01 sec
21-12-05 08:20:53 info: {
  id: '704605e4-b3fc-469c-a73e-f624bfe24764',
  duration: 0.01,
  timestamp: '2021-12-05T06:20:53.531Z',
  attempts: 1,
  camera: 'front',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:20:54 info: processing Door Camera: 1638685244.83641-xw0vbs
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 1)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:20:54 info: done processing Door Camera: 1638685244.83641-xw0vbs in 0.18 sec
21-12-05 08:20:54 info: {
  id: '1638685244.83641-xw0vbs',
  duration: 0.18,
  timestamp: '2021-12-05T06:20:54.269Z',
  attempts: 2,
  camera: 'Door Camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 08:21:02 info: processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9
21-12-05 08:21:02 error: TypeError: deepstack process error: Cannot read properties of undefined (reading 'flatMap')
    at Object.module.exports.normalize (/double-take/api/src/util/detectors/deepstack.js:68:39)
    at module.exports.normalize (/double-take/api/src/util/detectors/actions/index.js:8:25)
    at Object.module.exports.process (/double-take/api/src/util/process.util.js:142:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async Object.module.exports.start (/double-take/api/src/util/process.util.js:120:17)
    at async module.exports.polling (/double-take/api/src/util/process.util.js:53:25)
    at async Promise.all (index 0)
    at async module.exports.start (/double-take/api/src/controllers/recognize.controller.js:148:7)
21-12-05 08:21:02 info: done processing door camera: d343ebb2-3837-447c-91ba-19566bef2ad9 in 0.01 sec
21-12-05 08:21:02 info: {
  id: 'd343ebb2-3837-447c-91ba-19566bef2ad9',
  duration: 0.01,
  timestamp: '2021-12-05T06:21:02.125Z',
  attempts: 1,
  camera: 'door camera',
  zones: [],
  matches: [],
  misses: []
}
21-12-05 10:51:37 info: Double Take v1.7.0-bf3e911
21-12-05 10:51:37 info: MQTT: connected
21-12-05 10:51:37 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:08:55 info: Double Take v1.7.0-bf3e911
21-12-05 11:08:55 info: MQTT: connected
21-12-05 11:08:55 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot
21-12-05 11:15:58 info: Double Take v1.7.0-bf3e911
21-12-05 11:15:58 info: MQTT: connected
21-12-05 11:15:58 info: MQTT: subscribed to frigate/events, frigate/+/person/snapshot

and that what I see in HA about the image_processing
4

this is the conf of image processing

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.199
    port: 80
    save_file_folder: /config/www
    save_file_format: jpg
    always_save_latest_file: True
    targets:
      - target: person
        confidence: 60
    source:
      - entity_id: camera.door_camera_2
      - entity_id: camera.front

and this is the conf of frigate

cameras:
  Door Camera:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


            
  Front:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:554/Streaming/Channels/101
          roles: 
            - detect
            - clips
    objects:
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2


    
detectors: 
  cpu1:
    type: cpu
  cpu2:
    type: cpu

why its not working to me?

Deepstack addon?

I have Deepstack running as a docker container in HA supervised on a Debian host, so until recently that was a supported setup but now supervisor is complaining about unsupported software.

Converting the deepstack container to an add-on would fix that is my understanding. Has anyone tried this - unsure how difficult it is to add the existing image as a source to build an addon? One of the first hurdles would be to have volume mapping so it finds my custom model, though a deepstack folder in /share would work.

Many thanks!
Jhhbe

1 Like

in the terminal window you see |401| error.
for troubleshooting you can download the snapshot images in the frigate ui and try to use these with the deepstack ui application

image
i get the 400 error when the face detection canā€™t detect a face on the image

I just started using deepstack.
Iā€™m finding the same results.

A lot of the time there are false positives. Even with high confidence levels sometimes.
I trained just 2 people with 20+ images, but there are still false positives in the sense that person 1 gets recognized as person 2. Does anyone know how to improve this performance?

Hey, iā€™m trying to set it up with Synology Docker too and it doesnā€™t seem to work. could you explain more on how you did it?

I installed the DeepStack docker pack on my Synology Docker, and added the image_processing details to HA, with the IP (of synology) and the port (of the deepstack docker), also tried to change to different port numbers and it didnā€™t change anything.

When I call the service-scan, I see no change in the logs/terminal on docker

I installed it on Synology Docker and added the config to HA.
Now the Object detection works well, the Face detection sensor remains ā€œunknownā€ instead of the face number.
When I send the command to scan the face counter, I can see the new line adding to the terminal on docker. so home assistant does communicate with it about face detection, itā€™s just that it doesnā€™t get the information back or the sensor doesnā€™t get updated for some reasonā€¦

any ideas?

same, do you have error in ha log ?

I ended up using another machine instead of synology, using Intel Nuc with dockers in Proxmox, works perfect

This fixed it for me thank you very much.

1 Like

Thank you Rob for the amazing work.

I managed to Install deep stack on HASS machine (since it already have docker) ā€¦ I run it with face and object detection ā€¦ all is good. it recognize faces, people after training ā€¦ next step is various automations.

Hi all

I have deepstack running in docker and Motioneye integration setup in HA. I also have HASS deepstack object running from HACS. Below is config.yaml for image processing.

If I use developer tool to manually call image processing service, I do see the event fired in deepstack along w/ all the snapshots from cameras. However, it wonā€™t automatically detect object; please HELP as I spent last couple of days searching thru Google and Youtube w/ not much luck

image_processing:

  • platform: deepstack_object
    ip_address: 10.0.19.241
    port: 80
    save_file_folder: /config/snapshots/
    save_file_format: png
    save_timestamped_file: True
    always_save_latest_file: True
    scale: 0.75
    roi_x_max: 0.8
    roi_y_max: 0.8
    targets:
    • target: person
    • target: vehicle
      confidence: 60
    • target: car
      confidence: 40
      source:
    • entity_id: camera.garage_cam

Excited to discover this addon which supports using google coral and integration with my deepstack components

Iā€™ve been beating my head against the wall with Deepstack on synology (since that was the easiest docker place near my Hass NUC). Good to hear that Iā€™m not the only one who finds it fails on that config. Will look to install on the NUC I guessā€¦I have Frigate and DoubleTake working well, for the 10 minutes that Deepstack runs.

Hello,

I have Deepstack running only on PORT 80. If I change the port e.g. to PORT 81 the communication does not work anymore.

Do you have the same experience?

Hey Everyone,

For the last while Iā€™ve been getting a 401 Unauthorized returned by the deepquestai/deepstack container when an image processing scan is called. Nothing as changed in my configuration and Iā€™ve verified my API key is defined in the docker_compose.yaml as well as in the image_processing.yaml.

Docker log:

HASS log:

This error originated from a custom integration.

Logger: custom_components.deepstack_object.image_processing
Source: custom_components/deepstack_object/image_processing.py:349
Integration: deepstack_object (documentation)
First occurred: February 27, 2022, 12:00:29 AM (8 occurrences)
Last logged: 11:24:04 AM

Deepstack error : Error from Deepstack request, status code: 401

docker_compose:

version: "3.6"

services:
  deepstack:
    image: deepquestai/deepstack
    container_name: deepstack
    environment:
      - API-KEY="APIKEY_REDACTED"
      - VISION-DETECTION=True
      - VISION-FACE=True
    ports:
      - 5000:5000
    volumes:
      - /etc/docker/deepstack:/datastore
    restart: always

image_processing.yaml:

- platform: deepstack_object
  ip_address: 192.168.xx.xx
  port: 5000
  api_key: !secret deepstack_key
  save_file_folder: /config/www/snapshots/
  save_timestamped_file: True
  # roi_x_min: 0.35
  roi_x_max: 0.8
  #roi_y_min: 0.4
  roi_y_max: 0.8
  scan_interval: 10000
  targets:
    - target: person
      confidence: 80
    - target: car
      confidence: 65
    - target: truck
      confidence: 65
  source:
    - entity_id: camera.driveway

Anyone else run into this?

For me those image.processing sensors also seemed to stop working after upgrading to the newest release.

I found this thread https://www.reddit.com/r/BlueIris/comments/tmlxtc/bi_and_deepstack_nothing_found_after_move_to_new/

So i thought i should switch deepstack back to version 2021.09.01. But unfortunately the deepstack container on my jetson nano times out after 1 minute even if i trigger my image.processing sensors the api doesnā€™t seem to process those any more.

Homeassistant Logs just tell me that i reachted the 10 seconds timeout.

2022-04-14 20:42:04 WARNING (MainThread) [homeassistant.helpers.entity] Update of image_processing.deepstack_object_garage_cam is taking over 10 seconds
2022-04-14 20:42:05 ERROR (SyncWorker_1) [custom_components.deepstack_object.image_processing] Deepstack error : Timeout connecting to Deepstack, the current timeout is 10 seconds, try increasing

But all of this worked fine previously.
Can someone help me with this?

Regards,