Face and person detection with Deepstack - local and free!

Share your Projects!
826 
[{"id":"f47ff4fa.790eb8","type":"tab","label":"Flow 2","disabled":false,"info":""},{"id":"d6db80b9.75fda","type":"server-state-changed","z":"f47ff4fa.790eb8","name":"cat motion","server":"7dbb919d.bdcf9","version":1,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"binary_sensor.cat_pir_alarm","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"on","halt_if_type":"str","halt_if_compare":"is","outputs":2,"output_only_on_state_change":true,"x":80,"y":140,"wires":[["5f32488e.7205e8"],[]]},{"id":"ca1f37d1.ac86a8","type":"delay","z":"f47ff4fa.790eb8","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":500,"y":140,"wires":[["d3c5ef9e.2ee75"]]},{"id":"d3c5ef9e.2ee75","type":"api-call-service","z":"f47ff4fa.790eb8","name":"Deepstack Cat Scan","server":"7dbb919d.bdcf9","version":1,"debugenabled":false,"service_domain":"image_processing","service":"scan","entityId":"image_processing.deepstack_object_deepstack_cat","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":280,"y":240,"wires":[["18c4fd29.4d73a3"]]},{"id":"5f32488e.7205e8","type":"api-call-service","z":"f47ff4fa.790eb8","name":"Grab cat Image","server":"7dbb919d.bdcf9","version":1,"debugenabled":false,"service_domain":"shell_command","service":"cat_snapshot","entityId":"","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":300,"y":140,"wires":[["ca1f37d1.ac86a8"]]},{"id":"18c4fd29.4d73a3","type":"delay","z":"f47ff4fa.790eb8","name":"","pauseType":"delay","timeout":"4","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":480,"y":240,"wires":[["86efdf82.aba3d"]]},{"id":"86efdf82.aba3d","type":"api-current-state","z":"f47ff4fa.790eb8","name":"Cat Check","server":"7dbb919d.bdcf9","version":1,"outputs":2,"halt_if":"0","halt_if_type":"num","halt_if_compare":"gt","override_topic":false,"entity_id":"image_processing.deepstack_object_deepstack_cat","state_type":"str","state_location":"","override_payload":"none","entity_location":"","override_data":"none","blockInputOverrides":false,"x":710,"y":240,"wires":[["a4ade475.6865e8"],[]]},{"id":"a4ade475.6865e8","type":"api-call-service","z":"f47ff4fa.790eb8","name":"Telegram cat photo","server":"7dbb919d.bdcf9","version":1,"debugenabled":false,"service_domain":"shell_command","service":"cat_msg","entityId":"","data":"","dataType":"json","mergecontext":"","output_location":"","output_location_type":"none","mustacheAltTags":false,"x":970,"y":240,"wires":[[]]},{"id":"7dbb919d.bdcf9","type":"server","z":"","name":"Home Assistant","legacy":false,"addon":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":false}]

camera:
  - platform: local_file
    file_path: /config/www/deepstack_cat.jpg
    name: deepstack_cat
  - platform: local_file
    file_path: /config/www/snapshots/deepstack_object_deepstack_cat_latest.jpg
    name: deepstack_cat_latest

image_processing:
  - platform: deepstack_object
    ip_address: localhost
    port: 5000
#    api_key: mysecretkey
    save_file_folder: /config/www/snapshots/
    save_timestamped_file: false
    # roi_x_min: 0.35
    roi_x_max: 0.8
    #roi_y_min: 0.4
    roi_y_max: 0.8
    targets:
      - cat
    source:
      - entity_id: camera.deepstack_cat

shell_command:
  cat_snapshot: '(curl -s -X GET http://<ip of my hikvision picture stream>/ISAPI/Streaming/channels/2/picture > deepstack_cat.jpg && mv deepstack_cat.jpg www )'
  cat_msg:  '(curl -s -X POST "https://api.telegram.org/bot<your telegram bot token>/sendPhoto" -F chat_id=<cat chat id> -F photo="@/config/www/snapshots/deepstack_object_deepstack_cat_latest.jpg" -F caption="Cat Detected")'
827

If this helps anybody
setup

2 Likes
828

@Hitesh_Singh detect_only: True is a required parameter for the deepstack face integration when using tensorflow-lite.

@alpat59 no docker yet but that is a nice idea, should be possible

829

Just released v0.5 of the deepstack face integration, adding ability to save files (same functionality in deepstack object). Check it out :slight_smile:

3 Likes
830

Hi,

I run Home Assistant (Hassio / HassOS) on a Pi4 4GB and have just purchased a Coral USB stick. I currently use DOODS (with DOOBS Add On) but really want to use Deepstack or Frigate. Ideally, I would like to run my image_processing on the same device as HA with my Coral stick to help with speed of detection etc…

As I understand it, there is no Hassio/HassOS integration for Frigate and I tried to install a workaround with Portainer but as I have never used linux/docker it was too difficult.

I would really like to use your integration on my current RPi4 with the benefits of the Coral stick. Is this possible with very limited coding/docker/linux expertise?

I noticed there was a coral-pi-rest-server but that I think that uses tensorflow-lite which wouldn’t give me any advantages over DOODS which uses the same or something similar.

Sorry for replying to you directly as I saw your post way up in the thread but its pretty confusing for a NOOB because there are SO MANY githubs/ threads of yours/others that talk about very similar things.

Thanks

831

I am no longer working with the coral stick, it is simply not required unless you are doing high FPS. Furthermore it creates extra barrier to using docker owing to the USB interfacing. I am not a docker expert, but hopefully one arrives with interest in getting my tflite server running as an addon, no additional hardware required!

832

Hi, not sure if you meant me here. Not a clue where to start sorry, was just hoping you could just replace the sampled image with a transparent PNG so only the boxes are there. No worries if it’s a big job.

833

@Holdestmade please create a feature request on the repo, I am intrigued by this suggestion

834

Thanks for the great updates on the plugins. Both are working great together for me.

1 Like
835

@Gio76 I just deployed a custom classification model, you can see it here. Will do some more work on the server and do a write up soon

836

That’s amazing @robmarkcole! Thank you very much. Looking forward to be able to enter custom models in the object detection!

837

Just realeased v0.6 of the deepstack face integration, adding bounding boxes (optional):

image
image992×1208 322 KB

838

Building the automations I have in node-red would require a lot more patience and coding/scripting skills than I have.

Basically I’m having a simple tail on a ftp log that

  1. waits for the camera to login
  2. starts to grab screenshots every 0,5 seconds or so, then after 5 screenshots it gets them only every few seconds (because when motion begins its more probable that whatever you want to found in the images are found in the few seconds of motion) and
  3. while it still grabs screenshots it’s already running tensorflow/deepstack/doods (I’m still testing which is the most reliable one) until it finds something I’m interested in (say, a person) and then
  4. sends the processed image to me through Telegram + stops the image grabbing loop and sets a flow variable that something has been found, and then after:
  5. the camera uploads a motion video, it sends it to me through Telegram if something of interest was found. And last:
  6. It deletes the jpgs and mp4s from taking space. Also if nothing was found those are deleted earlier.

I’m using /run to save the temporary images and videos as to not have unnecessary wear on my SSD, because it’s doing writing a lot (whenever any of my cameras detect motion).

There’s also some other stuff I didn’t bother explaining here because otherwise this post would be even more huge…

1 Like
839

So I just benchmarked my tflite server running on RPi4 against deepstack on Mac pro, and the results are surprising. Processing 15 images, the RPi4 is significantly faster. This is owing to the optimisation of tflite models, but note accuracy will not be as good as deepstack on a Mac.

Platform Speed (sec) Predictions
Mac Pro with deepstack 51.9 91
RPi4 with tflite-server 9.33 159
1 Like
840

my GPU is finally here and I will be doing some installation and setup to switch from dlib to deepstack. Looking forward to some test this! I just wish it was not a container installation… as I think it will make the simultaneous GPU passthrough to both HA and deepstack impossible.

841

when deepstack is open sourced you will be able to run it however you like

1 Like
842

May I ask what else you would use the GPU for in HA?
I’m planning on using the GPU for deepstack someday…just wondering what else it could be used for.

843

Not specifically for HA, but I am for example looking at the camera components and have them send livestream instead of snapshots. The main application remains object and facial recognition but I am looking to do so on multiple streams simultaneously.
Further down the road, I may be looking at deep learning for home automation.

844

Is there some sort of compression happening after adding the bounding boxes? Very difficult to read the percentages… any way to make this more legible? image

845

It turns out to be difficult to create a function which will correctly annotate with text of appropriate size given the wide variety of shapes and sizes that images come in. Think I will remove that feature. Suggest you use the deepstack-ui to check what that thing is.