Face and person detection with Deepstack - local and free!

fhedstrom · November 21, 2020, 12:39pm

Hi,

I am having a problem getting the home assistant integration to work. Deepstack works fine and when I do a curl from HA it returns exactly what expected.

But when I run the integration in Home Assistant I get no detection

I can’t understand what I am doing wrong

My configuration.yaml

camera:
  - platform: local_file
    file_path: /config/www/image.jpg
        

image_processing:
  - platform: deepstack_object
    ip_address: localhost
    port: 5000
    api_key: mysecretkey
    save_file_folder: /config/snapshots/
    save_timestamped_file: True
    # roi_x_min: 0.35
    roi_x_max: 0.8
    #roi_y_min: 0.4
    roi_y_max: 0.8
    scan_interval: 5
    targets:
      - person
      - car
    source:
      - entity_id: camera.local_file
        name: Deepstack_object

johnolafenwa · November 24, 2020, 11:31am

DeepStack on Jetson!

Hello everyone. We are excited to share the release of DeepStack GPU Version on the Nvidia Jetson. With support for the full range of Jetson Devices, from the 2GB Nano edition to the higher end jetson devices.

This supports the full spectrum of DeepStack features.
You can run DeepStack on the Jetson with the command below.

sudo docker run --runtime nvidia -e VISION-DETECTION=True -p 80:5000 deepquestai/deepstack:jetpack-x1-beta

To run with the face apis, simply use -e VISION-FACE=True instead, for scene, use -e VISION-SCENE=True.

We are super excited to finally bring a stable version of DeepStack that runs on ARM64. We strongly recommend using this over the Raspberry + NCS version, as it is faster, more stable and the Jetson Nano is also less costly than the RPI + NCS combination.

We are working towards full open source release before December, With support for custom detection models and the Windows Native Edition all scheduled for this week.

Thanks for all your feedbacks, we are excited to build the future AI platform with you all.

robmarkcole · November 24, 2020, 12:11pm

@fhedstrom in your config you have included api_key but on your curl I can see you do not have it set

fhedstrom · November 24, 2020, 12:39pm

Yes. Somehow it works without the API key when doing curl, the integration doesn’t.

Here is a screen from the container config…

sunglass_guy · November 24, 2020, 1:20pm

Hi rob what’s the best way to update a deep stack container?

radinsky · November 24, 2020, 3:35pm

hi! is there anything new regarding custom models? thanks!

Unthred · November 24, 2020, 5:08pm

Wowsers super cool
Is face detection only possible on humans? I have at least 8 squirrels visiting my garden and I would love to be able to identify them

dorremez · November 25, 2020, 7:59am

Hi need some help please…
I’m running Hass.io,
Done with installing all the components(HACS)/Container(Portainer)/config lines (Configuration.yaml)…

When I run the “image.processing.scan” service, I get a results in the State of “image_proccessing.mydeekstack_detection”…

But, I don’t get any files saved to the folder.
And was wondering if I need to some how create an automation that will call the Service every while to see if anything was detected?

Plus, when I try the Curl command, I’m getting an “api error”…?

Please help!

Cavekeeper · November 27, 2020, 3:20pm

While looking for a binary_sensor for the deepstack image, I noticed your post. I’ve adapted the code accordingly. But unfortunately it doesn’t work as expected. The binary trigger is triggered, but it only drops out very late. This is probably also due to the fact that the image process for this channel is still above the value 0. How could I set up a binary sensor that works well here?

binary_sensor:
  - platform: template
    sensors:
      deepstack_flur_sensor:
      friendly_name: "Deepstack Flur Sensor"
      device_class: motion
      value_template: "{{ states('image_processing.deepstack_object_diskstation_flur')|float > 0 }}"

Baker_Benjamin · December 2, 2020, 6:00pm

long time lurker, first time poster

I have tried my best to read through this and can’t find the answer to my question.

I want to know if/ how to get this to learn people as they are coming and going and then make an object for said person so I can set triggers based on if that person is seen. Is this a thing? Did I miss it as I was scanning through comments?

The ultimate goal is to greet people and auto unlock the door based on their face and to have a custom greeting per person.

Any help pointing me in the right direction would be greatly appreciated.

lsaranto · December 2, 2020, 8:30pm

The face teaching part is here https://github.com/robmarkcole/HASS-Deepstack-face#service-deepstack_teach_face .

I don’t use face recognition, so I don’t know if you can use the whole video image for teaching or if you need to manually crop the images first. But I would run the recognition for a while. Then I would have a bunch of images which I would then teach to the AI. But like I said, I’m not sure if that’s the correct procedure.

Baker_Benjamin · December 2, 2020, 8:47pm

Thx I will definitely take a look at this tonight.

danb1 · December 8, 2020, 4:35pm

Hi
I’m trying to create a script that will remove old deepstack images to stop the images taking up too much space. I’ve got the below added in my configuration.yaml which works but I’d like to exclude the deepstack_camname_latest.jpg file as I use these files as cameras to see when the last person was detected. Can anyone help with this?

shell_command:
delete_camera_images: find /config/www/deepstack/ -type f -mtime +2 -exec rm “{}” ;

danb1 · December 8, 2020, 5:10pm

Replying to myself as someone on Facebook has kindly helped confirmed the command. This will delete all files older than 14 days except the deepstack_camname_latest.jpg file

shell_command:
  delete_camera_images_new: find /config/www/deepstack/ -type f -mtime +1 | grep -v \._latest\.jpg | xargs -i rm "{}"

robmarkcole · December 10, 2020, 3:20pm

Any developers on this thread might be excited to know that deepstack is now a public repo, and open for contributions

SummerRain · December 10, 2020, 6:28pm

I just started playing with DeepStack. Can anyone point me to the curl syntax for registering multiple pictures for one person? Thanks

Jafterdark · December 11, 2020, 3:40am

Coral USB Accelerator
or
Intel Neural Compute Stick 2
or
stick with my old Nvidia 960 graphics card

I prefer to use one of these USBS because I rather go to a server that would fit in a rack but if the card is better I will stick with it

I want to build a new NVR and looking for recommendations for AI detection on devices above

Thank you for your time and any information you can give me about the above two USB devices

starob · December 12, 2020, 11:39am

Hi,
for test purposes I have Deepstack-Object Docker running on 2 different systems:

Synology DS218+ NAS (Intel CeleronJ3355 (Dual Core), 6 GB RAM), Image: NOAVX, non-GPU and container limited to 3GB RAM
Notebook with Windows 10, Core i3-4010U (2 cores with HT) and 4 GB RAM. docker image: AVX, non-GPU version

I do a curl -X POST -F [email protected] ‘http://192.168.1.x/v1/vision/detection’ against those 2 systems by changing the IP address (same test.jpg file. It includes a person, garden plants and potted plants.)

The Synology NAS takes 6-10 seconds (!) to process the image and returns:

{"success":true,"predictions":[{"confidence":0.7821689,"label":"person","y_min":217,"x_min":945,"y_max":494,"x_max":1045},{"confidence":0.7728115,"label":"potted plant","y_min":361,"x_min":731,"y_max":489,"x_max":842},{"confidence":0.77145827,"label":"potted plant","y_min":410,"x_min":559,"y_max":522,"x_max":634},{"confidence":0.6516304,"label":"potted plant","y_min":376,"x_min":1068,"y_max":474,"x_max":1157},{"confidence":0.64147574,"label":"potted plant","y_min":426,"x_min":641,"y_max":504,"x_max":702},{"confidence":0.62971485,"label":"potted plant","y_min":413,"x_min":413,"y_max":526,"x_max":494},{"confidence":0.5387862,"label":"potted plant","y_min":350,"x_min":410,"y_max":535,"x_max":518},{"confidence":0.4051743,"label":"potted plant","y_min":329,"x_min":623,"y_max":477,"x_max":732},{"confidence":0.40153322,"label":"vase","y_min":400,"x_min":753,"y_max":489,"x_max":820}]}

The Notebook replies in ~650 ms and returns:

{"success":true,"predictions":[{"confidence":0.46187398,"label":"potted plant","y_min":754,"x_min":166,"y_max":979,"x_max":470},{"confidence":0.46211874,"label":"potted plant","y_min":472,"x_min":1260,"y_max":920,"x_max":1637},{"confidence":0.4783537,"label":"potted plant","y_min":231,"x_min":351,"y_max":368,"x_max":454},{"confidence":0.60136586,"label":"potted plant","y_min":368,"x_min":1073,"y_max":473,"x_max":1156},{"confidence":0.62832135,"label":"potted plant","y_min":334,"x_min":617,"y_max":506,"x_max":729},{"confidence":0.7375296,"label":"potted plant","y_min":357,"x_min":740,"y_max":493,"x_max":844}]}p

So the NAS identifies a person whereas the Notebook identifies a potted plant but no person.

Any ideas why the results can be so different?

One observation I made: On the NAS the datastore is used. It includes 2 files: activate and faceembedding.db. On the Notebook it is not used.

Regards

rafale77 · December 12, 2020, 4:07pm

That’s awesome news! I am too far along with my own project running retinaface+arcface and YoloV4+mish cuda on pytorch integrated into HA which is docker free and seems more accurate but requires a GPU. Maybe I’ll give deepstack another look. I am looking to improve efficiency to accommodate more inferences than my current 8 cameras (1536x2048) at 4fps each.

Eatoff · December 14, 2020, 2:59am

Is the raspberry pi version moving to docker? And will it still require the NCS2?