The models used are trained using normal cameras. I am not sure that it would work with a lidar camera.
DOODS needs to understand what the inputs and outputs of your models are. If it doesn’t understand the types, you get this error. You can’t just use any model. DOODS is designed to work with object detection models. If you set the environment variable LOGGER_LEVEL=debug and open a case in the git repo GitHub - snowzach/doods: DOODS - Dedicated Open Object Detection Service I could take a look.
I have been wondering about newer models also.
Not necessarily training my own, but newer than the default as I’ve been running this wonderful addon for a while now. I often get people detected in clumps of snow or reflections in a window or a birdhouse.
Are there other (than the included one) models that will work with DOODS?
Most of the mobilenet models work. The included ones are probably the best though in my opinion. It is what it is though, sometimes it sees random stuff. you can update the detection area and the confidence percentage to help prevent false positives.
Hi there. Just found DOODS and been playing around with it a few days…I’ve gotten it to work just fine using non-hw-accel but then it takes up a bunch of my CPU and I didn’t really want that…processing time was about 0.3 seconds…
I have a Nvidia P4000 installed and therefore I wanted to try the HW accelerated feature.
I’ve changed to the “cuda” release and I’ve added the GPU to my docker-compose file and selected a tensormodel with “hwAccel=true”
DOODS starts up fine and takes a bit to load the model selected (tensorflow) but then not much more happens…it takes 90 seconds for the image processing to timeout and that’s all that happens…
What am I doing wrong here? Do I need to select another detector?
CUDA support is kind of random… It can take DOODS a LONGGGG time to start up. I think what is happening is that the CUDA library has to compile tensorflow support for your card the first time it starts up. It can take a long time… Like annoyingly long… several minutes from my experience with the GTX970 I test with. It looks like DOODS is ready but it’s really not. I should add a message in there stating it could take a while and then another when it’s done. So my suggestion is wait… like a good 10 minutes before assuming DOODS is running. (sorry, it’s annoying I know) I tried to figure out how to cache the compiled code so it doesn’t take so long after the first time but I was unsuccessful. It’s not documented very well.
Hey snowzach and thanks for replying, amazing effort btw.
Yeah I read somewhere that it takes a long time before it’s ready (aka API ready), but it’s as if it never gets going really…It takes a good 3-5 minutes before it’s ready and then I can see an attempt to process images coming from HomeAssistant…however these just timeout after 90 sec… Before the “API ready” nothing is attempted to be processed.
I’m running the CUDA image and in my docker-compose file I have the following:
devices:
- /dev/dri:/dev/dri # For H/W transcoding
environment:
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=all
And then I’ve selected the tensorflow model to use…
Am I missing anything?
Ah! I see - So after API ready it just takes a while before it actually starts up, like 10 minutes as you said…
Now running on the GPU, processes an image in ~0.2-0.3 seconds…that’s good!
Last thing - If I wanna limit the GPU mem DOODS use, how do I do that? Currently it just steals all remaining 7.2GB of mem 
Thanks!
It looks like it’s possible to limit the GPU memory with some Tensorflow options… I can look into doing it. Is there a reason you want to limit the memory? Are you using the GPU with something else?
Hi again
Yeah, I use the GPU for Plex transcoding and plan on using it for Zoneminder streams as well…It’s a pretty heavy card, so I might as well get good use out of it 
Thanks
Hi, thanks for perfect work @snowzach.
Now I test CUDA version (in docker). I use GTX950 with 2G (level 5.2, 750 CUDA), CPU is i3 2nd Gen. Processing times are 0.2-0.3 seconds. I saw a warning about low memory (probably on GPU side).
What is optimal size of RAM on graphis card? (and on PC)
How to get a faster processing times? (use RTX card?, faster CPU, …)
Can you share your processing times with GPU type and memory size?
Hi,
I looking how to install “Edge TPU” on RPI4 with supervised HA OS.
I tried to use commands from: https://coral.ai/docs/accelerator/get-started/#requirements
And after
sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
I got msg- (No such file or directory)
What I should write to get it install? Are it is possible on to do it on the supervised HA OS?
Maybe @snowzach have some idea…?
Hi, thanks for the awesome work @snowzach.
My first post, however been using Home Assistant since the first few days of January this year and I’m impressed (came from Domoticz). I’ve struggled with all kinds of issues and these forums helped me a lot (w/ trial and error) and allowed me to get very far.
My setup is an HA OVA w/ the hass.io addon added through supervisor.
My current issue is that I am trying to see if I can adjust the automation trigger to work with the motion detector AND (this a condition?) if it’s a person defined by doods. I get way too many motion detections (I’ve adjust the sensitvity in Hikvision to try and lower the amount to no avail).
configuration.yaml
image_processing:
- platform: doods
scan_interval: 10000
url: "http://IP:8080"
auth_key: !secret doodskey
detector: tensorflow
file_out:
- "/config/www/pictures/{{ camera_entity.split('.')[1] }}_latest_doods.jpg"
# - "/config/www/pictures/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}_doods.jpg"
source:
- entity_id: camera.ds_2cd2112_i_mainstream
confidence: 40
labels:
- name: person
confidence: 40
- name: car
confidence: 60
- name: truck
confidence: 40
automation.yaml
# DOODs w/ camera (front)
- alias: Camera - Person detected
trigger:
- platform: state
entity_id:
- binary_sensor.videosource_1_motion_alarm
# - binary_sensor.mymotiondetectorrule_cell_motion_detection disabled due to the above working better.
to: "on"
condition: []
action:
- service: image_processing.scan
entity_id: image_processing.doods_ds_2cd2112_i_mainstream
- service: notify.mobile_app_step
data:
title: Motion detected!
message: Someone in front of your house. Prepare for battle!
data:
attachment:
url: https://myIP:8123/local/pictures/ds_2cd2112_i_mainstream_latest_doods.jpg
mode: single
The automation.yaml above spams me with messages (everything seems to works fine). However the following code added silences the message but doesn’t send me a message if I stand in front of the camera:
condition:
- condition: template
value_template: "{{ 'person' in state_attr('image_processing.doods_ds_2cd2112_i_mainstream','summary') }}"
I’m trying to create an “if-then-else” situation where the trigger and the condition are checked (is motion detected and is it a person scanned by doods) before performing the action. However I think I’ll need to start this somewhere before the action-sequence otherwise doods will not have scanned a person:
- service: image_processing.scan
entity_id: image_processing.doods_ds_2cd2112_i_mainstream
On the 30th of January: apparently sometimes it seems to detect someone in front of my house (w/ the extra condition) however there was no person in sight (doods didn’t see a person either):
matches:
car:
- score: 93.24881
box:
- 0.14872093
- 0.061141133
- 0.2969224
- 0.18527538
- score: 89.866615
box:
- 0.21446973
- 0.055247918
- 0.95497346
- 0.6572731
- score: 81.25439
box:
- 0.17098683
- 0.0012004471
- 0.28035635
- 0.056980252
truck:
- score: 40.161026
box:
- 0.04752839
- 0.30466023
- 0.17807434
- 0.49091586
summary:
car: 3
truck: 1
total_matches: 4
process_time: 1.4891642459988361
friendly_name: Doods ds_2cd2112_i_mainstream
Can anyone point me in the right direction?
My second question is:
I know that I can add my back camera in the config.yaml by adding another entity_id to the source. However how can I remove the car label from my backyard (there will never be a car there unless it falls from the sky) while keeping the car label in the front camera.
Third question (while I am at it, this will be my last and least necessary):
Is it possible to rename labels in the txt file or does that mess up the system?
Honestly, I don’t do a ton of work with Nvidia cards. I have an GTX970 that I’ve tested it with. If it’s 0.2-0.3 seconds with the inception model I would say that’s typical if I recall correctly. I couldn’t tell you much about how much memory is required. Sorry, I don’t honestly know much about the CUDA/tensorflow stuff. I just figured out how to make it basically work.
I won’t honestly be much help with HASS automations. I have tried a few times and gotten frustrated with it and switched to using NodeRED and won’t go back at this point. At least for me it’s so much easier.
If you want to create 2 instances of DOODS, just create another one with all the same stuff… ie:
image_processing:
- platform: doods
scan_interval: 10000
url: whatever
blah blah
- platform: doods
url: whatever
scan_interval: 10000
blah blah
You can change the labels if you like. It’s just a file that’s mounted inside the docker image. You’ll need to mount your own labels file over top the one inside the doods image. For the tflite detector it’s /opt/doods/models/coco_labels0.txt for tensorflow it’s /opt/doods/models/coco_labels1.txt the labels are based on the line number. Update it to whatever you want. I’ll leave it to google/stack overflow on how to mount files inside of docker. They explain it better than I can.
1 Like
I spent some time to find how to work with parameters. This code count number of persons in image (it is template sensor). Hope that helps.
- platform: template
sensors:
doods_cam_stodola_person:
friendly_name: Stodola cam person
value_template: >
{% set ns = namespace(ok = 0) %}
{% for item in state_attr('image_processing.doods_cam_stodola', 'matches').person %}
{% set score = (item.score | float) %}
{% set y = (item.box[0] | float) * 100 %}
{% set x = (item.box[1] | float) * 100 %}
{% set h = (item.box[2] | float - item.box[0] | float) * 100 %}
{% set w = (item.box[3] | float - item.box[1] | float) * 100 %}
{% if score > 0 and x > 0 and y > 0 and w > 0 and h > 0 %}
{% set ns.ok = (ns.ok + 1) %}
{% else %}
{% endif %}
{% endfor %}
{{ ns.ok | int}}
Replace camera name (doods_cam_stodola) and object (person)
3 Likes
Thank you for the useful information, I have NodeRED installed and working. Been following tutorials for NodeRED but still need a good use case and example (if you have any, i’ll be much obliged) and I will need to see how this works without HASS automation and still keep the overview.
Also a big thanks to @miki_lbc for showing me an decent example of a person automation!
Honestly this thread is the one that got me started with NodeRED Integrating Blue Iris, Pushover, Node-RED, and Amazon Rekognition - #167 by snowzach
After that I figured out how to use it. Now basically I have and entire alarm system state machine implemented with NodeRED. Any time someone pulls in my driveway or walks to the front door it plays a chime on my Google Homes. If I’m not home and it detects someone it sends notifications to my phone with Pushover. My outside lights are all controlled with it. Those are a handful of my automations.
Thanks a lot for a very cool integration!
In this cropped image, DOODS does not recognise me:
I assume I’m asking too much of DOODS right? It’s a cropped (only looking at the front door part of the image) night image, so I’m not giving DOODS much to work with…
For a human brain it’s blatantly obvious someone is behind the door, but I’m too cheap to pay a human to look at my camera images
Any idea’s for another approach I could take?
I’ll volunteer to monitoring your cameras! 
Nah just kidding, I have similar issues…DOODS detects my dog as 93% human and sometimes 90+% for a chair…it seems to struggle most with night time images, similar to your image…maybe some tweaking is needed here?
1 Like