I kind of already do the high fps processing @Alex_Pupkin using this:
https://github.com/danecreekphotography/node-deepstackai-trigger
Essentially what i do is record all my cameras all the time with zoneminder. point the deepstack-ai-trigger towards where zoneminder is saving all of the images, then have it send off triggers to mqtt once it finds a match. I’ve run 2 cameras at 15fps (1080p) so far with a nvidia gtx1060 without any difficulty.
@mist i have unifi system with 10 cameras going 1080p/4K depending on the camera. The main one i want to automate is one facing long driveway and it is 4k. I am getting >400ms on deepstack inference on 1 model and i actually want to run a cascade of models (object detection > classifiaction) so it is too slow. Fastrcnn gives me 6FPS object detection and densenet121 10FPS classifiaction of crops. NVidia TitanX Pascal hardware.
Hi, I have Home Assistant installed on Docker on an Intel NUC running Ubuntu 20.04.
I have Motioneyeos addon running on the Intel as a controller for remote Motioneye installations.
Separately the two webcam builds are both RPI Zero W running raspbian lite.
One has Motioneyeos installed (a), the other has RPi Cam Control running (b) and built as a testbed for this project
I’ve got one docker instance running Deepstack and the HACS modules installed on Home Assistant.
Whenever (b) detects motion, it runs a bash script and publishes an MQTT message. This triggers an automation on Home Assistant to run the Scan for a person target. This works fine.
I now want (a) to do the same. Local detection and publishing of message work fine, I have a separate docker deepstack instance with different port number for (a) but it seems the images are just not being grabbed.
Has anyone any suggestions as to why RPi Cam works, but Motioneye doesn’t
Setup details below
Thanks,
Camera configuration :
camera:
- platform: mjpeg
mjpeg_url: http://x.x.x.x:8081/
name: camera motioneye
- platform: mjpeg
mjpeg_url: http://x.x.x.x/html/cam_pic_new.php
name: camera RPi CAM
Deepstack Configuration
image_processing:
- platform: deepstack_object
ip_address: x.x.x.x
port: 5000
save_file_folder: /media/images/
save_file_format: png
save_timestamped_file: True
always_save_latest_file: True
scale: 1
roi_x_min: 0
roi_x_max: 1
roi_y_min: 0.2
roi_y_max: 1
targets:
- target: person
confidence: 70
source:
- entity_id: camera.camera_RPi_CAM
- platform: deepstack_object
ip_address: x.x.x.x
port: 5010
save_file_folder: /media/images/
save_file_format: png
save_timestamped_file: True
always_save_latest_file: True
scale: 1
roi_x_min: 0
roi_x_max: 1
roi_y_min: 0.2
roi_y_max: 1
targets:
- target: person
confidence: 70
source:
- entity_id: camera.camera_motioneye
Can you share a little bit info on how did you integrated torchserve to Home Assistant?
And can I use another architecture like YOLO?
I would love to read more about how to do that
I have some experience with transfer learning and CNNs
Anyone used this lib in Home Assistant?
That lib is exactly for face recognition
upd
I got my hans on it, seems to work fine (added 7 faces, tested my father’s face and my face together - works good)
That being said, model is on my laptop right now. And I am not getting a lot of fps… But anyways, how to connect it to Home Assistant? Should I use flask to make API calls from script and then send requests from Home Assistant?
Hello,
I hope someone will help me. I already registered a set of images for recognition. When I triggered the scan it showed the logs in my Deepstack Server below but the problem was In the entity it showed null. I have been trying to fix it but to no avail. Thanks a lot
[GIN] 2021/03/14 - 07:38:01 | 500 | 1m0s | 172.17.0.1 | POST /v1/vision/face/recognize
[GIN] 2021/03/14 - 07:40:10 | 500 | 1m0s | 172.17.0.1 | POST /v1/vision/face/recognize
[GIN] 2021/03/14 - 07:48:00 | 500 | 1m0s | 172.17.0.1 | POST /v1/vision/face/recognize
[GIN] 2021/03/14 - 07:51:54 | 500 | 1m0s | 172.17.0.1 | POST /v1/vision/face/recognize
faces:
total_faces: null
total_matched_faces: 0
matched_faces:
{}
friendly_name: deepstack_face livingroom_camera
device_class: face
image_processing:
I have this problem as well. I used this integration a few years back and remember it working well. However, i stopped using it and just come back to it yesterday and am getting matches mixed up with low confidence for myself and my girlfriend, even after training 10+ images of both of us.
Did you manage to get it better?
No, haven’t really had time to spend on this lately (and also, my plan is to use it for face recognition when someone presses the doorbell at my front gate - and the gate isn’t installed yet, so no real hurry)
Be interesting to see if it gets better, as I would like to be able to rely on it!
I have just been reading up this thread and there seems to be a general confusion over if multiple requests to register a picture against a name add to the profile, or just override the profile.
I have so far saved ten photos of each person and registered each in an individual call
Looking at https://forum.deepstack.cc/t/calling-vision-face-register-more-than-once/496 though it seems I should register all 10 faces in one call. There is a script there to help with the process so I’ll try that out this afternoon.
Had you been training faces in the same way as me?
Are you running deepstack on a VM? 1m is the max time the recognise process runs for and means its timed out.
Yeah, I’m registering all faces in one call. That thread you referenced is where I got conclusive confirmation from the deepstack team about that. (there was some conflicting information on it over various threads, so was got to get it clarified)
As mentioned, I haven’t spent a lot of time on it yet - but once my gate gets installed, I’ll probably end up with more test images to start training 8-10 faces, and we’ll see if it works any better.
Yes I have the same experience. I took 10 pictures for learning, and from the same camera, took 10 pictures for recognition in similar positions/backgrounds and they all came back null instead of unknown, which I seem to recall means it didn’t recognize a person in the image.
So after a day of tinkering, I’m left thinking that I’m not getting the benefit from Deepstack I know it can give.
I registered 8 different people on both Deepstack and Facebox.
The 8 people include myself and girlfriend who are the only ones I care to track, but I registered other males/females with same hair/skin to us to just check that matching worked well.
Myself and partner I registered about 50 images of (using DSM.py and the teach_facebox.py scripts) and for the other people I registered 5-10 pictures each.
When running a script to call the image_processing for both integrations at the same time when looking at a live camera feed of myself/girlfriend I found;
Deepstack
Often matched myself or partner correctly, sometimes with low 60% confidence.
Sometimes matched us as the wrong person and against on of the other learnt faces. Sometimes (often in fact) didn’t even get the sex right!!
Facebox
Did not once get an incorrect match.
Sometimes had low mid 50% confidence but still had the correct person.
Did not ever match against the other faces I loaded just for testing.
I’m sure I’m missing something as there is a reason you all use Deepstack. But why am I not having success with face matches?
Thank you
This is an excellent component. I have successfully got HA and deepstack working for object detection with a custom model. What is the correct syntax to add more custom models to the HA config? Is it simply:
custom_model: model1, model2, model3
I wanted to jump in here and thank you guys for this thread and the work here, I was able to use the information here to get a pretty cool automation set up, thought maybe i would post it here just in case.
I’m using the wyze cams with the “NFS hack” (credit to HclX at https://github.com/HclX/WyzeHacks) This allows the wyze cam to write to an NFS drive instead of SD Card.
With that and the help of this thread, i was able to set up the following:
This all seems to work pretty smoothly thus far, and allows me to turn off all notifications on the wyze cams (and thus remove almost all false alarms).
Hopefully someone can tell me what im doing wrong, I have deepstack setup in ha. I can run a automation for do a imageprocess but it never detects anything or outputs a image.
Here is my config.yaml
image_processing:
- platform: deepstack_object
ip_address: 172.30.16.2
port: 5000
api_key: f4S88y5sPgvz3ZMUE3WeZ
save_file_folder: /config/snapshots/
save_file_format: png
save_timestamped_file: True
always_save_latest_file: True
targets:
- target: person
- target: vehicle
confidence: 60
- target: car
confidence: 40
source:
- entity_id: camera.blueiris_front_right
I know it working in deepstack because i can see output when i run manual scan
[GIN] 2021/03/31 - 08:52:58 | 200 | 4.329813ms | 172.30.16.25 | POST /v1/vision/detection
[GIN] 2021/03/31 - 08:53:24 | 200 | 19.591µs | 172.30.16.25 | POST /v1/vision/detection
[GIN] 2021/03/31 - 08:53:24 | 200 | 19.591µs | 172.30.16.25 | POST /v1/vision/detection
[GIN] 2021/03/31 - 08:53:26 | 200 | 77.167µs | 172.30.16.25 | POST /v1/vision/detection
[GIN] 2021/03/31 - 08:53:28 | 200 | 17.9µs | 172.30.16.25 | POST /v1/vision/detection
Also tried with
allowlist_external_dirs:
- "/config/snapshots"
no luck
Just to be clear, by “see outputs”, do you mean you also see the json response? For example:
$ curl -X POST -F [email protected] 'http://192.168.1.26:5000/v1/vision/detection'
{"success":true,"predictions":[{"confidence":0.5986328,"label":"dining table","y_min":234,"x_min":2,"y_max":461,"x_max":614},{"confidence":0.77685547,"label":"cup","y_min":299,"x_min":219,"y_max":384,"x_max":288},{"confidence":0.7783203,"label":"cup","y_min":256,"x_min":447,"y_max":326,"x_max":518},{"confidence":0.8852539,"label":"person","y_min":45,"x_min":311,"y_max":285,"x_max":520},{"confidence":0.93115234,"label":"person","y_min":24,"x_min":50,"y_max":391,"x_max":339}]}
Well I found out the issue but I don’t know why. Using deepstack on unraid I’m getting 403 error. If I run the deepstack on windows I have no issues. No even sure how to go about troubleshooting the 403 error.
[GIN] 2021/03/31 - 0801:5324:24 54 | 200 403 | 19543.591178µs | 172.30.16.25
Im running deepstack on unraid as well, and HA on a Pi4. Can you access the web UI for deepstack at all? Try changing port number?
Oh wow! It was that I had object detection to false!! Jeez!