Face and person detection with Deepstack - local and free!

Ok - the use case here is car detection at at gate. I have to fire the detection multiple times sometimes for it to recognize something as simple as a car, even though I have played with confidence levels all over the place. 75-80% seems about where it needs to be to not just show the entire frame as being detected as a “car”… so, that means that the delays stack up every time I fire the event. Firing it 3 times can take over 15 seconds, or longer. I might as well get out of the car and punch in the numbers on the keypad to open the gate!

When a REAL car appears, it’s 95.3% sure it’s a car. Whew!

So obviously it’s not a critical use case, but there is something going on Deep inside causing processing delays… and it’s not my Pi being overloaded. Glances shows overall 27-30% CPU usage pretty much at all times.

Here’s me sending the curl -X trigger from my Mac to the Coral Pi rest server in succession:

~/Downloads/:> curl -X POST -F [email protected] 'http://192.168.1.208:5000/v1/vision/detection'
{"predictions":[{"confidence":0.58203125,"label":"car",
"x_max":2688,"x_min":25,"y_max":1520,"y_min":36}],"success":true}

~/Downloads/:> curl -X POST -F [email protected] 'http://192.168.1.208:5000/v1/vision/detection'
{"predictions":[{"confidence":0.58203125,"label":"car",
"x_max":2688,"x_min":25,"y_max":1520,"y_min":36}],"success":true}

~/Downloads/:> curl -X POST -F [email protected] 'http://192.168.1.208:5000/v1/vision/detection'
{"predictions":[{"confidence":0.58203125,"label":"car",
"x_max":2688,"x_min":25,"y_max":1520,"y_min":36}],"success":true}

Interesting that it thinks my empty driveway jpg I submit is consistently identified as a “car” with 58.20% confidence each time I submit it :slight_smile: (could it be the shadows/sun outline?) Hence my need for a number greater than this for confidence levels so I eliminate false positives.

And the resulting log:

2020-05-13 09:01:56,722 INFO werkzeug Thread-17325 : 192.168.1.200 - - [13/May/2020 09:01:56] "POST /v1/vision/detection HTTP/1.1" 200 -
2020-05-13 09:01:57,638 INFO werkzeug Thread-17326 : 192.168.1.200 - - [13/May/2020 09:01:57] "POST /v1/vision/detection HTTP/1.1" 200 -
2020-05-13 09:01:58,880 INFO werkzeug Thread-17327 : 192.168.1.200 - - [13/May/2020 09:01:58] "POST /v1/vision/detection HTTP/1.1" 200 -
2020-05-13 09:02:00,017 INFO werkzeug Thread-17328 : 192.168.1.200 - - [13/May/2020 09:02:00] "POST /v1/vision/detection HTTP/1.1" 200 -
2020-05-13 09:02:01,071 INFO werkzeug Thread-17329 : 192.168.1.200 - - [13/May/2020 09:02:01] "POST /v1/vision/detection HTTP/1.1" 200 -

as you can see from the timestamps, there is no delay, and I’m sending the 1MB image above for processing.

The good that comes from investigating all of this is my “new” deep understanding of this component and how it all works together! Not time wasted at all… Jeff

Just published v3.1 which allows filtering objects using a region of interest

Also a comparison of deepstack and rekognition, win for deepstack!

Hi all, Let me start by saying this is a such a great idea Rob you ahve done an amzing job with such a detailed instruction set up.

I would like to ask though has anyone tried installing deepstack on their hassio instance? I am having a problem that the component is running and the server is running (I can see the landing page at local ip) and i can see in portainer that the deepstack container is receiving a request every time I activate the service request but I am not getting an output and don’t know where to start to troubleshoot. I see that you can run the curl statement in the readme but I am not running a linux setup other than Hassio. Any tips would be appreciated. I am running Hassio on a VM machine on a windows laptop with an old core I7 and installed the deepstack:latest rather than novrm version as I believe virtual box and core i7 allow for vrm.

EDIT - I loaded up the windows based deepstack server and it worked seamlessly so it must be the portainer/docker deepstack server which has a problem. Still any tips would be welcomed.

EDIT - the problem was the I was looking at too many install documents and used VISION-SCENE=True -v localstorage:/datastore -p 80:5000 deepquestai/deepstack and didn’t realise i hadn’t put VISION- DETECTION =True . Thought I would leave this post in case it is helpful but maybe no one is as silly as I am.

1 Like

Strange. I’m also running into the calls apparently not being made. When I first set it up, I got it to capture/analyze one frame. Worked great! This was to the pi/coral setup. Unfortunately, calling the service hasn’t worked since, though it DOES seem to run an analysis on startup. I also set it up on an intel nuc, and that isn’t seeming to be called either. Any tips for that situation?

How are you calling the service from HomeAssistant? Via Developer Tools or through an automation? I have found that calling the Coral Python flask app in quick succession can result in results not being returned…

Post your relevant config and may be able to help troubleshoot

It is getting confusing when people talk about deepstack but are actually referring to my coral integration. For coral please use this thread instead, and we keep this one to discussion about the official deepstack docker container. Thanks

Hey sorry rob. I’m really talking about the custom component, because I can’t get it to fire against either endpoint. Thanks for all you awesome work btw. Even the little bit i got going was really cool.

BTW: Here’s my config:

image_processing:
  - platform: deepstack_object
    ip_address: skull.local
    port: 5000
    api_key: !secret deepstack_key
    save_file_folder: /config/snapshots/
    # save_timestamped_file: True
    # roi_x_min: 0.35
    # roi_x_max: 0.8
    #roi_y_min: 0.4
    # roi_y_max: 0.8
    targets:
      - person
    source:
      - entity_id: camera.loft

And im trying to trigger it from the services dev tool

EDIT:
Nothing shows up in DevTools-> Log. Is there somewhere else to look?

put logger in debug mode, this will show the api requests

If anyone wants to run a deepstack clone on an rpi and not need a movidius or coral stick, I have updated the following project to be compatible with deepstack integration:

Not seeing any requests for the most part. Oddly, i seem to be able to make maybe one request, but then future requests don’t go through.

So here’s the weird bit. It seems to be making calls (see the 1:26 websocket call), but it keeps setting the state back to the original one at 12:56.
It’s also not writing the latest image files to disk. It sometimes does, but I cant see any reason why/why not.

Log snippet here:
https://pastebin.com/XD9MYbbW

@blackgold9 can you create an issue on the repo with all the information in, thanks

Hi Guys,

I’m trying to automate face detection.
From what i’m seeing, there are 2 main available solutions :

  • integrated Dlib (using FaceLib and on-premise)
image_processing:
 - platform: dlib_face_identify
   source:
    - entity_id: camera.camera_parking
   faces:
     Thomas: /config/www/images/Thomas.jpg
     Michel: /config/www/images/Michel.png
  • deepstack (online with key API)

I’m disappointed i succeed to get my face recognized with any issue with DLIB, this is working well. Thus, i’m able to access at the time time detection though the UI to the information “faces > name > thomas, total_faces=1”. However i spend a few days, i’m not able to access to these attributes though templates neither to get history (like detected faces by date and time).

Capture d’écran 2020-05-12 à 11.56.07

Can you please guide me ? What is the best solution ? Do you succeed to access to these json attributes ? Why do you all move to deepstack, is this not possible to get it working with Dlib??

@robmarkcole look like you are dedicated to image recognition too, i’ve seen a lot of your posts and your VM ? Did you give up with Dlib ?

Thanks in advance.
Take care.

Thomas.

deepstack is a local solution. Cannot comment on dlib, other than to say when I looked at it installation was a faff

Are you able with deepstack to :

  • teach faces
  • detect faces, get cropped images on the UI or by notification

I think i will share all my configs on my wiki, i lost time just to find examples and implements them.

Furthermore, i’ve seen your comments, i’m wondering if it will work on my RaspiberryPi4 or only on my Mac VM. I’ve seen your comment that it was not possible.

Except your VM, do you have pre-working examples or wiki :
configuration.yaml
automation.yaml
Lovelace UI

Thanks again for helping.
Thomas.

The answer to these questions is yes, as documeted in the readme here. There are a few example automations on this thread surely, will take some searching

1 Like

Just curious, How does the rPi + neural compute stick 2 perform? Does it feel sluggish? Maybe hard to compare but say vs a similarly priced celeron/pentium class dedicated mini PC or laptop?

Having some trouble with this setup. Here is hoping someone can see what I’m missing.
I can’t get any image of what was captured.
I really just copied my config from the readme for object detection and added my camera.

- platform: deepstack_object
    ip_address: localhost
    port: 5000
    save_file_folder: /config/snapshots/
    save_timestamped_file: True
    # roi_x_min: 0.35
    roi_x_max: 0.8
    #roi_y_min: 0.4
    roi_y_max: 0.8
    targets:
      - person
      - car
      - truck
    source:
      - entity_id: camera.driveway

I have done the whitelist.

whitelist_external_dirs:
    - /config

and if I go to dev tools - services I can manually run the scan and get a result

 image_processing.deepstack_object_driveway	0	ROI person count: 0
ALL person count: 0
ROI car count: 0
ALL car count: 0
ROI truck count: 0
ALL truck count: 1
summary: 
truck: 1

objects: 
- bounding_box:
    height: 0.41
    width: 0.373
    y_min: 0.296
    x_min: 0.625
    y_max: 0.706
    x_max: 0.998
  box_area: 0.153
  centroid:
    x: 0.811
    'y': 0.501
  name: truck
  confidence: 97.279

Also tried running the event listener and listened for deepstack.file_saved and get nothing.

Anyone have a clue what I’m missing?

Thanks

So i have multiple security cams set up tracking motion. When motion is detected it calls image_process, I have 3 targets, person, dog, car. when deepstack.object_detected I call to notify my phone. I am no longer seeign what object it detect in the notification. For example when i trigger my garage camera, I get 3 notifications because i have 3 cars in the garage. The notification is not show what was detected, it is only showing the secion part with the confidence. It once did. I looked thru all the breaking changes, I fear I missed something

1 Like