New Custom Compontent - Image Processing - Object Detection - DOODS

[snowzach (ProZach)]

@ros I added a timeout handling function in the tflite detector. If it hangs for 2 minutes, it will hopefully recover. I really don’t know what’s causing it. I made a couple other changes. Maybe try pulling again.

@nic0dk you can try pulling the latest as well and see if it helps your situation. I am really curious if this is a power issue. Would it be possible to try using it with a powered USB 3.0 hub just to test? There is a low power and a high power library, if it still fails, maybe I’ll try another image with the low power library.

@champ26 try pulling it now. it should handle any number of image channels.

[ros]

@snowzach

@ros I added a timeout handling function in the tflite detector. If it hangs for 2 minutes, it will hopefully recover. I really don’t know what’s causing it. I made a couple other changes. Maybe try pulling again.

I see the Hassio addon has not been updated (I’m using the add-on).

[nic0dk (Martin Nicolaisen)]

need to get an usb 3.0 hub with external power - before im able to test, but will try it out

[snowzach (ProZach)]

@nic0dk actually, I am looking at the specs and it doesn’t draw that much power. It does however get very hot. Are you running continuous detections? I am going to set mine up here to run continuous to see if I can force the crash. Because it seems like it’s not consistent, it makes me feel like it might be a hardware issue.

@ros I actually just updated the docker image. All you should need to do in hass.io is select rebuild from the doods component.

[nic0dk (Martin Nicolaisen)]

@snowzach
yeah i found a well documented article about the coral in use and also with load(https://www.hackster.io/news/say-hello-to-google-coral-cdbb49183864)
when it peaks, it use up to 880ma and on USB type 3.0 - 3.1 you have max 900ma.
on the frigate container, i have no problem with progressing 1 FPS on each of my 5 cameras.

ill try to move the coral to the “yellow” port of the NUC, it seems to be better powered.

else i should use my desktop computer and try out the stick and container for se any different

[ros]

I’m not sure if the latest build helps to recover from 408. Here is the log:

2019-11-13T18:52:02.867+0200	INFO	server/server.go:137	HTTP Request	{"status": 200, "took": 0.217841812, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000668", "remote": "172.30.32.1:41022"}
2019-11-13T18:52:07.847+0200	INFO	tflite/detector.go:317	Detection Complete	{"package": "detector.tflite", "name": "edgetpuCOCO", "id": "", "duration": 0.014095944, "detections": 0}
2019-11-13T18:52:07.849+0200	INFO	server/server.go:137	HTTP Request	{"status": 200, "took": 0.082810301, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000669", "remote": "172.30.32.1:41028"}
2019-11-13T18:52:09.842+0200	INFO	tflite/detector.go:317	Detection Complete	{"package": "detector.tflite", "name": "edgetpuCOCO", "id": "", "duration": 0.013957559, "detections": 0}
2019-11-13T18:52:09.844+0200	INFO	server/server.go:137	HTTP Request	{"status": 200, "took": 0.218564425, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000670", "remote": "172.30.32.1:41040"}
2019-11-13T18:53:45.713+0200	INFO	server/server.go:137	HTTP Request	{"status": 408, "took": 90.090905139, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000671", "remote": "172.30.32.1:41052"}
2019-11-13T18:53:51.000+0200	INFO	server/server.go:137	HTTP Request	{"status": 408, "took": 90.090052005, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000672", "remote": "172.30.32.1:41060"}
2019-11-13T18:55:23.678+0200	INFO	server/server.go:137	HTTP Request	{"status": 408, "took": 90.090411073, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000673", "remote": "172.30.32.1:41130"}
2019-11-13T18:55:28.797+0200	INFO	server/server.go:137	HTTP Request	{"status": 408, "took": 90.040728713, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000674", "remote": "172.30.32.1:41148"}
2019-11-13T18:57:01.652+0200	INFO	server/server.go:137	HTTP Request	{"status": 408, "took": 90.09090961, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000675", "remote": "172.30.32.1:41220"}
2019-11-13T18:57:06.871+0200	INFO	server/server.go:137	HTTP Request	{"status": 408, "took": 90.076446719, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods.local.hass.io/6ptsb96j3s-000676", "remote": "172.30.32.1:41226"}

[champ26 (Champ)]

just pulled the latest amd64 version from dockerhub and updated to use the tensorflow detector, and processing is working like a charm.

thank you very much for the update.

as mentioned, the tensorflow detector does indeed take a bit more processing and time, but it does so on my machine within a few seconds.

very nice! my self as a person and my Ram truck are detected accurately every time with a 90%+ confidence!

[champ26 (Champ)]

very impressed. now i need to work on what i will use to trigger the detections and/or automation notifications, but the proof of concept is solid now.

great work and thank you for your effort!

[champ26 (Champ)]

forgive me for my ignorance, but is there an event trigger in Hass to act on an image processing detection?

the component example shows leveraging a motion to trigger a processing, but i want to do something different. i want to build an automation that does the following:

• trigger: when image processing detects a label (person, truck, etc) with a confidence of X, then trigger my action.

anyone might be able to assist me in the trigger part?

[snowzach (ProZach)]

@ros and @nic0dk try pulling latest again. I replaced it with the lower power/speed/temp version of the edgetpu library. I am curious if that helps…

[snowzach (ProZach)]

You can trigger based on the state of the doods component changing. Basically the count of detected things will be what changes in the state. Eventually it should fire an event.

[champ26 (Champ)]

i’ll play around with it. thx

[Bogdan_Mazurenko]

@snowzach

Hey @Bogdan_Mazurenko I am not really sure there. I wonder if your kernel is too old for the EdgeTPU to work. It actually looks like it does detect the device because it sees it under /sys/bu/usb/devices/2-2… Maybe try just --device /sys/bus/usb and see if that works? The other thing you could try is just using the --privileged option. That will basically allow doods root access.

Thanks for response. But none is working. DOODS doesn’t even try to connect to EdgeTPU. It seems that i need to use external linux machine like raspberry to run with EdgeTPU.

NAS linux kernel version is 4.14.x
Architecture x86_64

[nic0dk (Martin Nicolaisen)]

sorry for first returning back now - my hassio crashed and burned… need to start from scratch again now with backup each night!!

anyway no big change, if im lucky its running for 30 seconds, sometimes a bit more or else, depends on the tweaks i made on:

      numThreads: 4
      numConcurrent: 4

the only thing i havent tried, is a external powered usb 3.0 hub (even moved the stick to the yellow port, no different)

btw @snowzach did you have an edge tpu yourself?

[snowzach (ProZach)]

@nic0dk yeah I have an edgetpu but I don’t use it a ton. I just set it running on a continuous detection loop to see if I can get it to crash. I’ll let you know.

If you or anyone else tries this, make sure to pull the latest docker container if you try it. I did make some changes that should effect how much power it draws.

[nic0dk (Martin Nicolaisen)]

i did test the new version - same behavior ill try to buy an external powered hub asap.

what port are you running your edge tpu on? usb 3.1 ?

[snowzach (ProZach)]

I was running it on an ODroidC2 for a while. I believe it was a USB 2.0 port. I was able to reproduce the hangs now. Actually, it just died completely and I was never able to recover the edgetpu. I think it caused a hardware USB problem of some sort. (Possibly power?) I moved it to a Raspberry Pi4. I also made another change to the code that monitors the detection. You can now specify a timeout: 10s option on the detector. If it stops responding for over 10 seconds, it will error and exit Doods. (The idea is that you have restart: always in a docker config so it will restart doods and the device.) I have it running continuously now so we’ll see what happens over the next day. I still do wonder if this is somehow USB/power related. It’s not easily reproducible in a consistent manor which can sometimes mean hardware. My only concern is that the Python library doesn’t appear to have that issue from what I hear so software definitely isn’t ruled out. Unfortunately there’s not a lot I can do on the edgetpu side as I am calling the C library. I may try older versions of the EdgeTPU library if we still keep having issues. Thanks for the support in testing this.

[runningman84]

This problems sounds similar to this issue:

Doods stops working after some time

[nic0dk (Martin Nicolaisen)]

hmm getting confused here… am i not using the noavx version when i pull from docker the latest version?

and how do i know if i run noavx on docker image @snowzach

[snowzach (ProZach)]

It doesn’t matter which image you pull, I put the change in all of them. Sorry pulling the latest can have multiple meanings.