New Custom Compontent - Image Processing - Object Detection - DOODS

DeadEnd · January 17, 2020, 2:21am

Thanks!
I have done so digging and chatting with others. After checking the docker log it appear to take between 0.5 and 0.7 seconds to detect. It looks like the 2-3 second delay may actually be something in the Home Assistant Image Processing component. It seems to be the bottleneck.

Thanks!
DeadEnd

danbutter · January 17, 2020, 2:26am

I didn’t think you needed to do the whitelist unless you were trying to send the images.
I know I didn’t do it until I was trying to send the latest.jpg via telegram notifications.
Even then I did it wrong, but it still worked.

danbutter · January 17, 2020, 2:29am

Any way to do nvidia gpu with this addon?

snowzach · January 17, 2020, 2:40am

It’s on my todo list. I just got a 970 but I haven’t gotten it to work inside of VMware yet. Haven’t had time to mess with it.

DeadEnd · January 17, 2020, 3:27pm

There is nvidia-docker but I think you either have to install the required packages in the container, or use their docker image as a base:

DeadEnd

MrUkleja · January 17, 2020, 4:47pm

I did something bad … But it works.
No I didn’t change my dogs
I changed file with labels - changed labels that were shown (after more testing my dogs were recognized also as elephant, sheep and handbag) to ‘dog’ …
Maybe not the best solution, but I got what I wanted

firstof9 · January 18, 2020, 3:38pm

2020-01-18 02:09:05.869115: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3 SSE4.1

Any idea @snowzach how I could utilize SSE3/SSE4.1?
Thanks.

jeremywaller · January 18, 2020, 3:55pm

Thanks for this awesome component @snowzach! It’s working great with my cameras.

I’m struggling to get motion detection working the way I want though, so I’ve set up a process to save the images where motion was detected and I want to send those over to doods.

Is there any way to specific an image on disk instead of a camera entity?

Ideally my flow would be ( 1 - camera sends image where motion was detected) -> ( 2 - automation is triggered by image showing up in folder) -> ( 3 - automation sends image to doods) -> ( 4- doods does it’s magic)

#3 is the only piece of the puzzle I can’t seem to figure out.

MrUkleja · January 18, 2020, 4:25pm

You can use Generic IP Camera to make local image a camera.

jeremywaller · January 18, 2020, 4:53pm

That looks like exactly what I’m looking for. Didn’t know that was a thing. Thanks!

snowzach · January 19, 2020, 4:04am

Hey @firstof9 I just added a new version to the hassio doods repo. There’s an AMD64 version that supports SSE 4.2. Check it out and let me know.

MrUkleja · January 19, 2020, 10:06am

I also see this in my log:

2020-01-19 10:36:44.036865: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3 SSE4.1 SSE4.2

I installed AMD64 version, but it’s not starting. Noting in add-on log - it’s empty

firstof9 · January 19, 2020, 2:56pm

Sorry I’m using dockerhub to pull images I don’t use HassIO.

snowzach · January 20, 2020, 2:43am

@firstof9 even easier, use the amd64 tag instead of the latest tag.

@MrUkleja if it’s not starting then that means it’s compiled with some other flag that your CPU doesn’t have. It kinda stinks because there’s about 5-6 different flags that you can use and I can’t really create one with every flag. If you want to show me what CPU flags you support, that might help me figure out what to support.

firstof9 · January 20, 2020, 2:52am

@snowzach I tried that version yesterday and it just caused the container not to launch.

MrUkleja · January 20, 2020, 8:46am

Ok, here is what I got after using command:

cat /proc/cpuinfo | grep -i flags

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep
mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2
 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon p
ebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf p
ni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr
 pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave lahf_
lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dthe
rm arat pln pts

chrisb · January 20, 2020, 3:10pm

I have been using this component for about a week and it is great. I would love to see it take advantage of my GPU. Basing the DOODs container on one of the Nvidia-cuda containers or the tensorflow-gpu container seems like it might be a decent starting place. The downside is that some of these containers are huge.

chris_avfc · January 22, 2020, 9:01am

Didn’t ask the original question, but tried this as a test and it appears to be working well.

I have DOODS working on four live camera streams (Porch, Drive, Side and Garden), and now the one static image for the Porch as a test.

Upon motion detection on the porch, a camera snapshot is taken, which is passed to DOODS.
Just need to play with timing a little to ensure the snapshot isn’t taken too early/too early in the motion.

The snapshot method appears to have a higher chance of capturing the subject in the image due to the speed at which I’m seeing DOODS process the image.

iRanduMi · January 22, 2020, 2:31pm

I’m experiencing the same error. Were you able to resolve? If so, what did you do?

Ryan_Grandy · January 22, 2020, 11:21pm

I am getting 2 odd things in the log after trying to switch to a new model, which I can’t figure out and am curious if anyone knows. The default model worked but I wanted to try to get something more accurate. The new model appears to load but I haven’t seen it actually defect anything yet.

When the addon starts up it says “width”: -1, “height”: -1, this doesn’t seem right as the default model is 300x300. Why would this be and what am I doing wrong?
When a scan is initiated it gives me a “WARN” “Missing label”. What is causing this? I have a label file.

2020-01-22 16:51:31.263706: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3 SSE4.1 SSE4.2 AVX AVX2
2020-01-22T16:51:31.294-0600	INFO	detector/detector.go:78	Configured Detector	{"package": "detector", "name": "inception", "type": "tensorflow", "model": "/share/doods/faster_rcnn_inception_v2_coco_2018_01_28.pb", "labels": 165, "width": -1, "height": -1}
2020-01-22T16:51:31.301-0600	INFO	server/server.go:274	API Listening	{"package": "server", "address": ":8080", "tls": false, "version": "v0.1.5-0-g99f5768"}
2020-01-22T16:52:03.726-0600	WARN	tensorflow/tensorflow.go:214	Missing label	{"package": "detector.tensorflow", "name": "inception", "index": 15}
2020-01-22T16:52:03.726-0600	INFO	tensorflow/tensorflow.go:259	Detection Complete	{"package": "detector.tensorflow", "name": "inception", "id": "", "duration": 9.255106498, "detections": 0}
2020-01-22T16:52:03.727-0600	INFO	server/server.go:138	HTTP Request	{"status": 200, "took": 9.470519011, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "03492332-doods.local.hass.io/d2HBZQ4Dfd-000001", "remote": "192.168.1.199:56690"}

Below is my config:

  "server": {
    "port": "8080"
  },
  "auth_key": "",
  "doods.detectors": [
    {
      "name": "default",
      "type": "tflite",
      "modelFile": "/opt/doods/models/coco_ssd_mobilenet_v1_1.0_quant.tflite",
      "labelFile": "/opt/doods/models/coco_labels0.txt",
      "numThreads": 1,
      "numConcurrent": 1,
      "hwAccel": false
    },
    {
      "name": "inception",
      "type": "tensorflow",
      "modelFile": "/share/doods/faster_rcnn_inception_v2_coco_2018_01_28.pb",
      "labelFile": "/share/doods/coco_labels1.txt",
      "numThreads": 1,
      "numConcurrent": 1,
      "hwAccel": false
    }
  ]
}