Hi, thanks for perfect work @snowzach.
Now I test CUDA version (in docker). I use GTX950 with 2G (level 5.2, 750 CUDA), CPU is i3 2nd Gen. Processing times are 0.2-0.3 seconds. I saw a warning about low memory (probably on GPU side).
What is optimal size of RAM on graphis card? (and on PC)
How to get a faster processing times? (use RTX card?, faster CPU, …)
Can you share your processing times with GPU type and memory size?
Hi,
I looking how to install “Edge TPU” on RPI4 with supervised HA OS.
I tried to use commands from: https://coral.ai/docs/accelerator/get-started/#requirements
And after
sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
I got msg- (No such file or directory)
What I should write to get it install? Are it is possible on to do it on the supervised HA OS?
Maybe @snowzach have some idea…?
Hi, thanks for the awesome work @snowzach.
My first post, however been using Home Assistant since the first few days of January this year and I’m impressed (came from Domoticz). I’ve struggled with all kinds of issues and these forums helped me a lot (w/ trial and error) and allowed me to get very far.
My setup is an HA OVA w/ the hass.io addon added through supervisor.
My current issue is that I am trying to see if I can adjust the automation trigger to work with the motion detector AND (this a condition?) if it’s a person defined by doods. I get way too many motion detections (I’ve adjust the sensitvity in Hikvision to try and lower the amount to no avail).
configuration.yaml
image_processing:
- platform: doods
scan_interval: 10000
url: "http://IP:8080"
auth_key: !secret doodskey
detector: tensorflow
file_out:
- "/config/www/pictures/{{ camera_entity.split('.')[1] }}_latest_doods.jpg"
# - "/config/www/pictures/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}_doods.jpg"
source:
- entity_id: camera.ds_2cd2112_i_mainstream
confidence: 40
labels:
- name: person
confidence: 40
- name: car
confidence: 60
- name: truck
confidence: 40
automation.yaml
# DOODs w/ camera (front)
- alias: Camera - Person detected
trigger:
- platform: state
entity_id:
- binary_sensor.videosource_1_motion_alarm
# - binary_sensor.mymotiondetectorrule_cell_motion_detection disabled due to the above working better.
to: "on"
condition: []
action:
- service: image_processing.scan
entity_id: image_processing.doods_ds_2cd2112_i_mainstream
- service: notify.mobile_app_step
data:
title: Motion detected!
message: Someone in front of your house. Prepare for battle!
data:
attachment:
url: https://myIP:8123/local/pictures/ds_2cd2112_i_mainstream_latest_doods.jpg
mode: single
The automation.yaml above spams me with messages (everything seems to works fine). However the following code added silences the message but doesn’t send me a message if I stand in front of the camera:
condition:
- condition: template
value_template: "{{ 'person' in state_attr('image_processing.doods_ds_2cd2112_i_mainstream','summary') }}"
I’m trying to create an “if-then-else” situation where the trigger and the condition are checked (is motion detected and is it a person scanned by doods) before performing the action. However I think I’ll need to start this somewhere before the action-sequence otherwise doods will not have scanned a person:
- service: image_processing.scan
entity_id: image_processing.doods_ds_2cd2112_i_mainstream
On the 30th of January: apparently sometimes it seems to detect someone in front of my house (w/ the extra condition) however there was no person in sight (doods didn’t see a person either):
matches:
car:
- score: 93.24881
box:
- 0.14872093
- 0.061141133
- 0.2969224
- 0.18527538
- score: 89.866615
box:
- 0.21446973
- 0.055247918
- 0.95497346
- 0.6572731
- score: 81.25439
box:
- 0.17098683
- 0.0012004471
- 0.28035635
- 0.056980252
truck:
- score: 40.161026
box:
- 0.04752839
- 0.30466023
- 0.17807434
- 0.49091586
summary:
car: 3
truck: 1
total_matches: 4
process_time: 1.4891642459988361
friendly_name: Doods ds_2cd2112_i_mainstream
Can anyone point me in the right direction?
My second question is:
I know that I can add my back camera in the config.yaml by adding another entity_id to the source. However how can I remove the car label from my backyard (there will never be a car there unless it falls from the sky) while keeping the car label in the front camera.
Third question (while I am at it, this will be my last and least necessary):
Is it possible to rename labels in the txt file or does that mess up the system?
Honestly, I don’t do a ton of work with Nvidia cards. I have an GTX970 that I’ve tested it with. If it’s 0.2-0.3 seconds with the inception model I would say that’s typical if I recall correctly. I couldn’t tell you much about how much memory is required. Sorry, I don’t honestly know much about the CUDA/tensorflow stuff. I just figured out how to make it basically work.
I won’t honestly be much help with HASS automations. I have tried a few times and gotten frustrated with it and switched to using NodeRED and won’t go back at this point. At least for me it’s so much easier.
If you want to create 2 instances of DOODS, just create another one with all the same stuff… ie:
image_processing:
- platform: doods
scan_interval: 10000
url: whatever
blah blah
- platform: doods
url: whatever
scan_interval: 10000
blah blah
You can change the labels if you like. It’s just a file that’s mounted inside the docker image. You’ll need to mount your own labels file over top the one inside the doods image. For the tflite detector it’s /opt/doods/models/coco_labels0.txt for tensorflow it’s /opt/doods/models/coco_labels1.txt the labels are based on the line number. Update it to whatever you want. I’ll leave it to google/stack overflow on how to mount files inside of docker. They explain it better than I can.
1 Like
I spent some time to find how to work with parameters. This code count number of persons in image (it is template sensor). Hope that helps.
- platform: template
sensors:
doods_cam_stodola_person:
friendly_name: Stodola cam person
value_template: >
{% set ns = namespace(ok = 0) %}
{% for item in state_attr('image_processing.doods_cam_stodola', 'matches').person %}
{% set score = (item.score | float) %}
{% set y = (item.box[0] | float) * 100 %}
{% set x = (item.box[1] | float) * 100 %}
{% set h = (item.box[2] | float - item.box[0] | float) * 100 %}
{% set w = (item.box[3] | float - item.box[1] | float) * 100 %}
{% if score > 0 and x > 0 and y > 0 and w > 0 and h > 0 %}
{% set ns.ok = (ns.ok + 1) %}
{% else %}
{% endif %}
{% endfor %}
{{ ns.ok | int}}
Replace camera name (doods_cam_stodola) and object (person)
3 Likes
Thank you for the useful information, I have NodeRED installed and working. Been following tutorials for NodeRED but still need a good use case and example (if you have any, i’ll be much obliged) and I will need to see how this works without HASS automation and still keep the overview.
Also a big thanks to @miki_lbc for showing me an decent example of a person automation!
Honestly this thread is the one that got me started with NodeRED Integrating Blue Iris, Pushover, Node-RED, and Amazon Rekognition - #167 by snowzach
After that I figured out how to use it. Now basically I have and entire alarm system state machine implemented with NodeRED. Any time someone pulls in my driveway or walks to the front door it plays a chime on my Google Homes. If I’m not home and it detects someone it sends notifications to my phone with Pushover. My outside lights are all controlled with it. Those are a handful of my automations.
Thanks a lot for a very cool integration!
In this cropped image, DOODS does not recognise me:
I assume I’m asking too much of DOODS right? It’s a cropped (only looking at the front door part of the image) night image, so I’m not giving DOODS much to work with…
For a human brain it’s blatantly obvious someone is behind the door, but I’m too cheap to pay a human to look at my camera images
Any idea’s for another approach I could take?
I’ll volunteer to monitoring your cameras! 
Nah just kidding, I have similar issues…DOODS detects my dog as 93% human and sometimes 90+% for a chair…it seems to struggle most with night time images, similar to your image…maybe some tweaking is needed here?
1 Like
Yeah, I am not sure what you can do there. I suspect the door frame is confusing the Tensorflow model. The only other thing might be to train a custom model here. I have never done it myself but in theory you could train the model to understand person with a door in the way there. I hope to get time someday to actually learn how to do this myself.
1 Like
As to your suggestion I moved to Node-RED to see if I can make something awesome happen in it.
So far I’ve created a sequence and it seems to me the idea works but I’m trying to see if an attribute contains a value (attribute summary contains person) to continue with the sequence:
This is what I got from the debug log:
So far I’ve tried to fetch the attribute person with this line:
msg.data.attributes.summary person
Though it keeps giving me this error:
“Error: Invalid property expression: unexpected ’ ’ at position 23”
Now before I try to debug something I’ve never done before, maybe anyone can point out what kind of node or parameters/properties I am missing or defined wrong?
`
My tests during the day are also failing, which fits well with your theory that the door frame is confusing the model.
I’m already proud that I managed to integrate DOODS and get it working (even if you seem to have managed to hide most of the complexity, thanks again!), I guess training the model will be out of my league 
Perhaps in a year or two it will become easy enough that even I can do it.
Hi All , Love the project
2 question
1 is it possible to put a Coral Mini PCIe M.2 into a USB converter
2 has anyone done it
I would use the debug option a bunch to see what’s coming out of the previous block. Sometimes I’ve found it easier to use the code node to just process each message by hand. Sometimes the home assistant nodes are goofy. What is coming out of msg. might be a sub object of that message. It takes a little fiddling to get used to node red but once you do it’s magic.
I know that people have used the mini PCIe EdgeTPU with DOODS but I have not heard of anyone doing it with a USB converter. I am pretty skeptical that the EdgeTPU would work in one of those adapters. I think they are mostly designed for network cards that support some sort of USB mode. I could be wrong though.
Thanks for the reply
@snowzach, forgive me if I’m asking in the wrong place, as I’m not using the HA component right now, but rather another instance of DOODS in a Docker container on a different rPi. I’m using motion detection in motionEye to detect and save image to disk, and Node-Red to watch the folder and send found images to DOODS with the following JSON:
"detector_name": "default",
"data": msg.payload.toString('base64'),
"detect": { "car": 60,
"truck": 60
}
}
msg.headers = {
"Content-Type": "application/json"
}
return msg;
That’s working great, and if there are detections returned the file is saved off for review, sensors are triggered, automations run, etc.
What I’m missing is seeing DOODS ‘work’: the bounding boxes and labels and confidence.
In the HA component, there is the file_out: "/config/www/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg" directive that gives me what I want.
From snooping at the msg that DOODS is returning in Node-Red, it looks like there is an image in there. Is it the equivalent of file_out:, and if so, how can I extract it?
Thanks for amazing work with DOODS… it’s my first foray into ML and AI stuff, and for an old guy, it’s truly incredible stuff.
I’m really starting the get the hang of it, the debug block tells me the information on the second picture from my previous post.
I do not know the appropriate syntax or current state block to see if “person” exists in msg.data.attributes.summary.
According to DOODs and the debug log it should tell me whether “car” or “person” or “truck” exists:
msg.data.attributes.summary.person gives me a wrong example because if person = 0, the attribute does not exist.
DOODS doesn’t return any image data. It just returns the coordinates and labels detected. Inside of the home assistant component is where draws the boxes and annotations. This is outside of the scope of what DOODS does (at least for now)