Bah… no that’s a bug… Change it to
labels:
- name: person
- name: car
- name: truck
and it will work now. I’ll push a fix tonight.
Ok. Looks like these detection times were for RPI4. To get Coral working one needs to specify a coral-compartible model file - e.g. mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite
With such a model specified and with “hwAccel: true” detection times are very different like “duration”: 0.014071279.
Question - what would be the best model for detecting cars with a high degree of precision and even at night? Anything available off the shelf?
So it’s a bit of a mixed bag. The EdgeTPU models are really fast but not as accurate. I use http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz for accuracy. It’s however very heavy and may not even be able to run on a Raspberry Pi. Youll have to experiment.
Thanks! Update fixed the problem.
Would love some suggestions on this. I’m running DOODs on an ESXi VM on a Dell R710 (2x Intel Xeon E5630 2.53Ghz). I’ve given the VM 4x vCPUs and 16GB of memory…
When I use the default detector, I get the following:
2019-11-06T17:32:08.537Z INFO tflite/detector.go:273 Detection Complete {"package": "detector.tflite", "id": "", "duration": 0.151961344, "detections": 0}
2019-11-06T17:32:08.538Z INFO server/server.go:137 HTTP Request {"status": 200, "took": 0.269121395, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d73edf98f0f0/XnCvfuMEJD-000174", "remote": "192.168.2.4:38310"}
If I try to use the more accurate TensorFlow detector, the duration jumps:
2019-11-06T17:38:19.925Z INFO tensorflow/tensorflow.go:268 Detection Complete {"package": "detector.tensorflow", "id": "", "duration": 20.579554513, "detections": 1}
2019-11-06T17:38:19.925Z INFO server/server.go:137 HTTP Request {"status": 200, "took": 20.674139447, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d73edf98f0f0/XnCvfuMEJD-000180", "remote": "192.168.2.4:38568"}
Is there any way to reduce the detection time with TensorFlow? I’ve seen some posts about using the Google Coral, however that appears to require a different detector and isn’t as accurate as TensorFlow. I’ve tried bumping up the number of vCPUs, but they don’t appear to change anything.
Have you noticed the same times for all detections? There are the number of threads and concurrent. You should have the threads set to 4 and the concurrent set to 1-2. This basically creates 1-2 instances of tensorflow and each has 4 threads. The first detection typically takes quite a while but then the model gets cached and it speeds up.
Run a detection a few times and see if it speeds up. There is also 2 images. Latest points to the noavx image which is the most compatible. If you pick the amd64 tag it should be faster if your processor supports avx and sse4.2. I am not sure the 5630 supports avx though.
The edge TPU only supports models that are compiled for it which seems to be simplers ones.
The other thing you can try is resizing your image somehow before getting it to doods. The larger the image the longer it will take obviously.
Actually, change threads to zero and it should auto-select the number of threads. set concurrent to the number of cameras you have.
I’ll give that a try and see what happens… one question: what are the width/height attributes for under TensorFlow in the config?
The width and height is for the model. Some models have a fixed input width and height. If you don’t resize the image for the images will automatically be resized for you. Most of the mobilenet ones have a 300x300 image size or 224x224
Do you know what it does on aspect ratio?
Doods does not maintain aspect ratio. It just resizes at will. The idea is that any sort of image manipulation should be done before you pass it to Doods. That’s why you can see the width/height in the detectors call. It’s not ideal, but at the same time, in my experience, messing up the aspect ratio still produces okay results. I tend to prefer models like inception which take full size images, at the cost of massive CPU use. It’s a trade off that you need to play with a little. The other option would be to keep the aspect ratio but then you’re effectively loosing even more fidelity as it turns into an image with black bars at the top and bottom and even less detail. Perhaps myself or someone else can work on an enhancement to the component that if you provide a global detection area, it crops before sending to DOODS so you could specify a square area perhaps and aspect would be maintained. It could also have the benefit of being faster.
Changing the threads to 0 and concurrent to my # of cameras made a huge difference! Almost 50% faster:
2019-11-07T18:56:30.586Z INFO tensorflow/tensorflow.go:268 Detection Complete {"package": "detector.tensorflow", "id": "", "duration": 11.999233115, "detections": 0}
2019-11-07T18:56:30.587Z INFO server/server.go:137 HTTP Request {"status": 200, "took": 12.10396248, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "590e28c3b5b2/hldmG1tUvH-000206", "remote": "192.168.2.4:56454"}
Am going to work on width/height now. I’m not event sure what size snapshot I’m sending in…
which docker you using? noavx, amd64?
1 Like
I’m using the “latest” image. My processor doesn’t support the fancier features.
Slightly off topic… please forgive me.
Some background first. I’m using DOODs as follows:
- Camera detects motion, notifies HA via MQTT
- HA, after verifying automation hasn’t been run in last 60s, calls
image_processing.scan - If DOODs has returned a detection, then:
3.1. Send me a notification with the analyzed picture
3.2. Callcamera.recordwith duration of 20s, lookback of 10s.
I don’t believe that lookback is working. I have added the stream: component, and “ticked off” “pre-load stream” in Lovelace. It doesn’t seem to matter what value I put for lookback, I still get the same delay between the initial picture saved during image_processing.scan and the video.
I’m looking for a better way to do this. My initial thought is that I should start recording a video when the camera detects motion. If DOODs returns 0 detections, I can then delete the video. Seems easy in practice, but I think I would need to write a script that HA would call to accomplish this.
Is anyone else doing something similar? Have you got lookback working for your? Any other suggestions?
For completeness, here is my .yaml to do the above:
- platform: doods
scan_interval: 10000
url: "DOODS_URL"
detector: tensorflow
file_out:
- "/opt/homeassistant/config/www/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "/mountpoint/Homeassistant/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
source:
- entity_id: camera.frontdoor
confidence: 70
labels:
- name: person
- name: car
- name: truck
- alias: "camera motion on frontdoor"
trigger:
platform: state
entity_id: binary_sensor.dahua_frontdoor
to: 'on'
condition:
- condition: template
value_template: "{{ as_timestamp(now()) - as_timestamp(states.automation.camera_motion_on_frontdoor.attributes.last_triggered) | int > 60 }}"
action:
- service: image_processing.scan
entity_id: image_processing.doods_frontdoor
- alias: "tensorflow frontdoor"
trigger:
platform: state
entity_id: image_processing.doods_frontdoor
condition:
condition: template
value_template: "{{ 'person' in state_attr('image_processing.doods_frontdoor', 'summary') }}"
action:
- service: notify.MY_PHONE
data:
title: "Tensorflow"
message: "frontdoor"
data:
attachment:
content-type: jpeg
url: "https://MY_NABU_CASA_URL/local/tmp/frontdoor_latest.jpg"
- service: camera.record
data:
entity_id: camera.frontdoor
filename: "/mountpoint/Homeassistant/frontdoor_{{ now().strftime('%Y%m%d_%H%M%S') }}.mp4"
duration: 20
lookback: 10
2 Likes
i’ve been using the default detector, and it is super quick on my docker instance (host is i5, 12GB ram machine) but not so accurate (configwise i’m not playing with confidence scores at this point and just trying to see what the system is detecting). i have the faster_rcnn_inception_v2_coco_2018_01_28.pb model in the models folder. how can i try using this?
do i just switch the detector in my config to “tensorflow” or something else? i’m pretty new to this and a bit lost trying to get past this basic/default config
this is what i see in my doods docker log file for a detection currently:
2019-11-08T19:02:02.338Z INFO tflite/detector.go:266 Detection Complete {"package": "detector.tflite", "id": "", "duration": 0.028414947, "detections": 5}
and my hass config is
- platform: doods
scan_interval: 30
url: !secret doods_url
#detector: default
detector: default
file_out:
- "/config/www/img_proc/{{ camera_entity.split('.')[1] }}_latest.jpg"
#- "/config/www/img_proc/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
source:
- entity_id: camera.front_door_cam
- entity_id: camera.backyard_cam
- entity_id: camera.south_gate
- entity_id: camera.cars
#confidence: 50
labels:
- name: person
- name: car
- name: truck
- name: dog
- name: cat
wow that seems awesome, so do you record all the time, or only on motion? i dont fully understand the camera.record, but it seems awesome!
if i try to switch to the tensorflow detector inside of my configuration.yaml in i get the following error in hass logs
error converting image channels attribute 3 does not match bits per pixel from file 73646915
[[{{node DecodeBmp}}]]
where can i go to find the root cause?
that node DecodeBmp, makes me think that when switching to tensorflow that its looking for bmp files instead of the jpg files that my cameras in hass are sending?
hi @snowzach.
how do i know if im using my coral edgeTPU ?
set the config.yaml at following:
detectors:
- name: default
type: tflite
modelFile: models/coco_ssd_mobilenet_v1_1.0_quant.tflite
labelFile: models/coco_labels0.txt
numThreads: 4
numConcurrent: 4
hwAccel: true
and my docker-compose.yaml look like this:
services:
doods:
image: snowzach/doods:latest
container_name: doods
restart: unless-stopped
environment:
- TZ=Europe/Copenhagen
volumes:
- /etc/localtime:/etc/localtime:ro
- /root/docker/doods/config.yaml:/opt/doods/config.yaml
devices:
- /dev/bus/usb:/dev/bus/usb
ports:
- "8080:8080"
and in configuration.yaml:
image_processing:
- platform: doods
url: "http://192.168.1.4:8080"
detector: default
source:
- entity_id: camera.hoveddor
- entity_id: camera.indkorsel
- entity_id: camera.terrasse_nord_vest
- entity_id: camera.terrasse_syd_vest
- entity_id: camera.terrasse_ost
file_out:
- "config/www/image_processing/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "config/www/image_processing/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
confidence: 70
labels:
- name: person
- name: car
- name: truck
but my CPU is spiking and that shouldn’t be the case?
doods | 2019-11-09T21:52:18.721+0100 INFO tflite/detector.go:277 Detection Complete {"package": "detector.tflite", "id": "", "duration": 0.564877156, "detections": 0}
doods | 2019-11-09T21:52:18.722+0100 INFO server/server.go:137 HTTP Request {"status": 200, "took": 1.210405839, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "afbf0334d5d2/KWMjnTV8fu-000430", "remote": "192.168.1.10:49102"}
@champ23 I am not totally sure what that means. It sounds like it’s sending a format that it doesn’t understand. What format is your camera sending in? Doods will try to convert to bmp if it’s not a png, gif, jpeg or bmp.