understood, and the solution sounds good enough 
thanks for the tip @Tinkerer!
just out of curiousity: I’m guessing that turning off the scan_interval is also out of the game because it would probably need some serious changes in the service?
Yes, you can’t set it to zero, but you can set it to some crazy large amount of time. I did look up the maximum value once, and the answer was many weeks if not more. I usually pick either 24 hours or 7 days as feels appropriate.
yeah, I’ve already have 24h for other cameras [that can report motion detection to trigger automations], just that one that is “dumber” brought me to the whole idea thanks again.
Hey Snowzach - first, thank you so much! This is such a great component. Setup was surprisingly easy and I went from zero to tensorflow in like 20 minutes. I actually stood it up in a fairly beefy VM and it looks like it’s not nearly as resource hungry as I might have guessed.
I set it up to see if we could find the nighttime (animal) visitor on our back deck. That’s admittedly overkill, but isn’t that the reason most of us do these things?
I’m finding that Doods is reliable in detecting people, but it is getting a lot of other things (hilariously) wrong.
It thinks all of our chairs are toilets, our potted plants (with flamingos) are laptops, and our small dog is a horse.
These all come back with relatively high confidence.
Here’s my setup:
- platform: doods
scan_interval: 60
url: "http://10.15.1.55:8080"
detector: tensorflow
source:
- entity_id: camera.kitchencam
- entity_id: camera.hallwaycam
file_out:
- "/mnt/snowmass/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "/mnt/snowmass/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
confidence: 75
(The kitchen cam is actually what’s mounted on the deck currently)
Is there a way to exclude things? like can I do:
exclude:
- laptop
- toilet
Or is there a way to retrain the model to improve accuracy somehow?
Thanks again for a great project!
Long-time user, frequent lurker here. Thank you for this amazing integration! I have 5x Dahua cameras that I’m using with the Amcrest integration to send any motion events through DOODs. Working pretty well, but I have a couple of questions:
I do have a Coral attached and configured (?)
image-processing:
image: snowzach/doods:latest
restart: always
networks:
public:
ports:
- 8080:8080
devices:
- /dev/bus/usb:/dev/bus/usb
volumes:
- /hdd12tbpool/docker/askrask/doods/models:/opt/doods/models
- /hdd12tbpool/docker/askrask/doods/example.yaml:/opt/doods/config.yaml
environment:
TZ: America/Toronto
How do I know that it is actually being seen and used?
I frequently see the following messages in the log, what does “missing label” mean?
WARN tensorflow/tensorflow.go:214 Missing label {"package": "detector.tensorflow", "name": "tensorflow", "index": 33},
Many thanks,
J.
Hey @spacebass there is no way to exclude things from detection (currently) the alternative is to include things that you are interested in. Include everything in the labels file that you might be interested in… or perhaps look like what you are interested in. Include dog and cat if you are interested in say a racoon. The default list of labels is here: https://dl.google.com/coral/canned_models/coco_labels.txt I added a ticket for myself to add an exclude option.
1 Like
Hi @Juggler That typically means your config is wrong or you’re using the wrong text file for the models file you are using. I have seen a few cases where the file is just wrong. I’ve seen several variations of the coco text file as well. Essentially the models produce an output with a number that indicates the line of the text file for the label. If it says it’s missing, you might be using the wrong text file. There are 2 variants of the coco labels. You could try switching them.
There’s this one: https://dl.google.com/coral/canned_models/coco_labels.txt that is used for the mobilenet models.
And this one: https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt that is used for the inception models.
Thanks. I’ve verified that my coco_labels1.txt (I’m using faster_rcnn_inception… model) is identical to the one you shared.
Here is my image_processing.yaml:
- platform: doods
scan_interval: 10000
url: "http://192.168.2.10:8080"
detector: tensorflow
file_out:
- "/config/www/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "/mountpoint/Homeassistant/{{ camera_entity.split('.')[1] }}_{{ state_attr('input_datetime.lastmotion_'~camera_entity.split('.')[1], 'timestamp') | timestamp_custom('%Y%m%d_%H%M') }}.jpg"
source:
- entity_id: camera.driveway
- entity_id: camera.frontdoor
- entity_id: camera.garage
- entity_id: camera.playset
- entity_id: camera.pool
confidence: 80
labels:
- name: person
- name: car
- name: truck
Nothing obviously incorrect in there? Watching the logs, I only see the messages very sporadically. Could there be a bug somewhere?
It’s not likely a bug. It’s just that whatever label number the model is spitting out is not part of the labels file. You can see there are 80 labels in that file. It’s most likely spitting out a number greater than 80. If the rest of the detections look correct, I suspect it’s just some additional thing the model was trained for. There might be more information about the model that says what the value is used for that I am not aware of. I kinda think models are closely guarded (along with documetation) because the good ones are worth a lot of money I guess.
1 Like
Hey snowzach - any idea why my docker has started crashing?
I switched from using Tensorflow as my detector to using a Coral and edgetpu - the good news is before it crashes, it is much more accurate. The bad news is that it crashes pretty quickly. Unfortunately this box is USB2 so I thought it might be a speed thing, but I set the rate to 120 seconds and it still crashes.
administrator@bigsky:~$ sudo docker run -it --device /dev/bus/usb -v /home/administrator/doods/models:/opt/doods/models -v /home/administrator/doods/example.yaml:/opt/doods/config.yaml -p 8080:8080 snowzach/doods:latest
[sudo] password for administrator:
ERROR: Could not open 'models/coco_ssd_mobilenet_v1_1.0_quant.tflite'.
ERROR: The model is not a valid Flatbuffer file
2020-05-20T17:40:14.435Z ERROR detector/detector.go:73 Could not initialize detector default: could not load model models/coco_ssd_mobilenet_v1_1.0_quant.tflite {"package": "detector"}
2020-05-20T17:40:17.868Z INFO detector/detector.go:78 Configured Detector {"package": "detector", "name": "edgetpu", "type": "tflite-edgetpu", "model": "models/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite", "labels": 80, "width": 300, "height": 300}
2020-05-20 17:40:18.379676: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3
2020-05-20T17:40:18.391Z INFO detector/detector.go:78 Configured Detector {"package": "detector", "name": "tensorflow", "type": "tensorflow", "model": "models/faster_rcnn_inception_v2_coco_2018_01_28.pb", "labels": 65, "width": -1, "height": -1}
2020-05-20T17:40:18.393Z INFO server/server.go:274 API Listening {"package": "server", "address": ":8080", "tls": false, "version": "v0.1.5-0-g99f5768"}
2020-05-20T17:40:45.644Z INFO server/server.go:138 HTTP Request {"status": 200, "took": 0.004948739, "request": "/detectors", "method": "GET", "package": "server.request", "request-id": "1de0ae177b0c/zjLEJJcgZd-000001", "remote": "10.15.1.30:44936"}
2020-05-20T17:40:45.659Z INFO server/server.go:138 HTTP Request {"status": 200, "took": 0.00085173, "request": "/detectors", "method": "GET", "package": "server.request", "request-id": "1de0ae177b0c/zjLEJJcgZd-000002", "remote": "10.15.1.30:44938"}
2020-05-20T17:42:46.693Z INFO tflite/detector.go:431 Detection Complete {"package": "detector.tflite", "name": "edgetpu", "id": "", "duration": 0.379114038, "detections": 0, "device": {"Type":1,"Path":"/sys/bus/usb/devices/2-1"}}
2020-05-20T17:42:46.694Z INFO server/server.go:138 HTTP Request {"status": 200, "took": 0.46854004, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "1de0ae177b0c/zjLEJJcgZd-000004", "remote": "10.15.1.30:45202"}
2020-05-20T17:42:46.858Z INFO tflite/detector.go:431 Detection Complete {"package": "detector.tflite", "name": "edgetpu", "id": "", "duration": 0.16469957, "detections": 2, "device": {"Type":1,"Path":"/sys/bus/usb/devices/2-1"}}
2020-05-20T17:42:46.859Z INFO server/server.go:138 HTTP Request {"status": 200, "took": 0.632989922, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "1de0ae177b0c/zjLEJJcgZd-000003", "remote": "10.15.1.30:45204"}
2020-05-20T17:44:47.461Z INFO tflite/detector.go:431 Detection Complete {"package": "detector.tflite", "name": "edgetpu", "id": "", "duration": 0.182944442, "detections": 0, "device": {"Type":1,"Path":"/sys/bus/usb/devices/2-1"}}
2020-05-20T17:44:47.461Z INFO server/server.go:138 HTTP Request {"status": 200, "took": 0.274593207, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "1de0ae177b0c/zjLEJJcgZd-000006", "remote": "10.15.1.30:45402"}
2020-05-20T17:44:47.637Z INFO tflite/detector.go:431 Detection Complete {"package": "detector.tflite", "name": "edgetpu", "id": "", "duration": 0.175928323, "detections": 3, "device": {"Type":1,"Path":"/sys/bus/usb/devices/2-1"}}
2020-05-20T17:44:47.637Z INFO server/server.go:138 HTTP Request {"status": 200, "took": 0.450978551, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "1de0ae177b0c/zjLEJJcgZd-000005", "remote": "10.15.1.30:45404"}
2020-05-20T17:47:18.298Z ERROR tflite/detector.go:296 Detector timeout {"package": "detector.tflite", "name": "edgetpu", "device": {"Type":1,"Path":"/sys/bus/usb/devices/2-1"}}
Hey @spacebass I have seen others with similar issues. I think honestly it has something to do with power. The USB2 might not be able to provide enough power to use the EdgeTPU. I haven’t been able to figure out anything other than that. Sorry. 
hello snowzach, is PCIE coral supported or only USB version will work?
It should be supported assuming it uses the same drivers. You will need to figure out what device it shows up as and pass that through to the container.
Thank you for this integration. I resolved my issue
It seems I have (2) problem.
First, there is some issue where gasket-dkms not install properly. So I basically reinstall this.
Next, while dealing with first issue I readd USB coral to server so I can continue detecting until I fix. Well, it seems Doods didnt like the USB + PCIe running together.
I created container with device /dev/apex added and device /dev/bus/usb removed but still get error
detector/detector.go:73 Could not initialize detector edgetpu: could not initialize edgetpu /sys/bus/usb/devices/3-1
USB was priority and although device removed from docker, the fact it still physically connected to server, Doods want to make the connection. Simply disconnecting physical device again make it work.
@snowzach just moved from tensorflow. really makes it more modular and efficient. awesome work. would you mind sharing the status of event implementation? thanks.
Hi, since I updated to v110.1 DOODS make crashed mi home assistant, if I stop the addon it keep alive fine
Someone more with this trouble?
Sorry for my english and thanks
Hi @tmjpugh, I will see if I can work on adding an option to select which hardware device to use. Currently it tries to use them all. What is the name of the device that shows up?
Hi @cpuram, DOODS is actually a subset of Home Assistant image detection. I believe to fire events, it would need to be added there. Are you thinking that it should be an event when it is non-zero detections or perhaps when the number of detections increase?
Hi @pollinolas, can you provide some logs? It’s really hard to tell what might be causing the crash.
Hi, thanks for answer, I don’t know how to depure it, I only know that if I have the addon running home assistant crashed or it’s reseted with a recorder ended unfinished session advise in the log
Before update, it was working good like always