New Custom Compontent - Image Processing - Object Detection - DOODS

Hey @BlazeYaSmartHome try this:

    file_out:
      - "/tmp/{% raw %}{{ camera_entity.split('.')[1] }}{% endraw %}_latest.jpg"
      - "/tmp/{% raw %}{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}{% endraw %}.jpg"
1 Like

Sorry for the delayed response. Was traveling.
I tried to set the confidence to 10 in the config as you can see below. I also looked at the detection with the debug and it is showing multiple objects detected.

  - platform: doods
    scan_interval: 10000
    url: "http://192.168.1.15:8080"
    detector: default
    file_out:
      - "/home/homeassistant/.homeassistant/www/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
    source:
      - entity_id: camera.driveway
      - entity_id: camera.frontdoor
    confidence: 10
    labels:
      - name: person
        confidence: 10
      - car

image

Log:

2019-08-31T23:59:34.760Z        DEBUG   detector/detector.go:58 Configuring detector    {"package": "detector", "config": {"name":"default","type":"tflite","model_file":"models/coco_ssd_mobilenet_v1_1.0_quant.tflite","label_file":"models/coco_labels0.txt","num_threads":4,"num_concurrent":4,"hw_accel":false}}
INFO: Initialized TensorFlow Lite runtime.
2019-08-31T23:59:34.898Z        INFO    detector/detector.go:76 Configured Detector     {"package": "detector", "name": "default", "type": "tflite", "model": "models/coco_ssd_mobilenet_v1_1.0_quant.tflite", "labels": 80, "width": 300, "height": 300}
2019-08-31T23:59:34.908Z        INFO    server/server.go:222    API Listening   {"package": "server", "address": ":8080", "tls": false}
2019-09-01T00:02:42.919Z        DEBUG   tflite/detector.go:165  Got Image       {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 1066, "height": 600}
2019-09-01T00:02:42.920Z        DEBUG   tflite/detector.go:167  Resizing Image  {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 300, "height": 300}
2019-09-01T00:02:44.207Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 66.796875, "location": "%!d(float32=0.055098265),%!d(float32=0.43829048),%!d(float32=0.48801824),%!d(float32=0.7392647)"}
2019-09-01T00:02:44.207Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 53.90625, "location": "%!d(float32=0),%!d(float32=0.3829388),%!d(float32=0.14547582),%!d(float32=0.48769468)"}
2019-09-01T00:02:44.207Z        INFO    tflite/detector.go:266  Detection Complete      {"package": "detector.tflite", "id": "", "duration": 1.255325322, "detections": 2}
2019-09-01T00:02:44.223Z        INFO    server/server.go:88     API Request     {"status": 200, "took": 1.363177218, "remote": "192.168.1.183:40404", "request": "/detect", "method": "POST", "package": "server.request", "request-id": "f749d7dafdd4/TmowE9Ti4r-000001"}
2019-09-01T00:05:02.430Z        INFO    server/server.go:88     API Request     {"status": 200, "took": 0.016561892, "remote": "192.168.1.183:40872", "request": "/detectors", "method": "GET", "package": "server.request", "request-id": "f749d7dafdd4/TmowE9Ti4r-000002"}
2019-09-01T00:05:25.202Z        DEBUG   tflite/detector.go:165  Got Image       {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 1066, "height": 600}
2019-09-01T00:05:25.202Z        DEBUG   tflite/detector.go:167  Resizing Image  {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 300, "height": 300}
2019-09-01T00:05:26.424Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 65.625, "location": "%!d(float32=0.055098265),%!d(float32=0.44297314),%!d(float32=0.48801824),%!d(float32=0.73458207)"}
2019-09-01T00:05:26.426Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 53.90625, "location": "%!d(float32=0),%!d(float32=0.38767767),%!d(float32=0.14507787),%!d(float32=0.48601562)"}
2019-09-01T00:05:26.429Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 50, "location": "%!d(float32=0),%!d(float32=0.22003245),%!d(float32=0.15798804),%!d(float32=0.43706542)"}
2019-09-01T00:05:26.431Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 39.0625, "location": "%!d(float32=0.052513093),%!d(float32=0.097026706),%!d(float32=0.98623765),%!d(float32=1)"}
2019-09-01T00:05:26.431Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 37.890625, "location": "%!d(float32=0.007793609),%!d(float32=0.47803453),%!d(float32=0.06524968),%!d(float32=0.5402155)"}
2019-09-01T00:05:26.431Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 34.375, "location": "%!d(float32=0.0070268214),%!d(float32=0.37261128),%!d(float32=0.18374865),%!d(float32=0.4977162)"}
2019-09-01T00:05:26.431Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 34.375, "location": "%!d(float32=0.048252463),%!d(float32=0.43728563),%!d(float32=0.58133113),%!d(float32=0.8578448)"}
2019-09-01T00:05:26.431Z        DEBUG   tflite/detector.go:263  Detection       {"package": "detector.tflite", "id": "", "label": "car", "confidence": 33.203125, "location": "%!d(float32=0.013728801),%!d(float32=0.37236094),%!d(float32=0.12890534),%!d(float32=0.41491246)"}
2019-09-01T00:05:26.431Z        INFO    tflite/detector.go:266  Detection Complete      {"package": "detector.tflite", "id": "", "duration": 1.201710235, "detections": 8}
2019-09-01T00:05:26.433Z        INFO    server/server.go:88     API Request     {"status": 200, "took": 1.260348486, "remote": "192.168.1.183:40944", "request": "/detect", "method": "POST", "package": "server.request", "request-id": "f749d7dafdd4/TmowE9Ti4r-000003"}

Please let me know. Thanks!!!

Hi @andreasfelder just pushed a bunch of changes and fixes to the repo. Trying pulling down the latest and see if it fixes your issues. They are reviewing the code now to include it into 0.99

Awesome will pull down the new version now and report back. Thanks!

After updating to the new version. It still didn’t work however this time i started looking into it a bit more and realized i do have an error in the home assistant log. It showed a failure due to incorrect path. I fixed my file_out path and everything is working now.

Sorry to waste your time. Awesome component!!!

Hey @snowzach, in the documentation for Dockers I noticed this image…

x86avx - This is an x64_64 with AVX and SSE4.2 features enabled. Use this for newer, more powerful x86

Is this for Nvidia gpus or just a newer i7?

I also just noticed you added the coral…that’s awesome man that little thing is amazing. Thanks again!

1 Like

Does this component also recognize different persons?

Hey Darbos, there is no docker image that supports GPUs (yet). It’s the avx image supports more modern CPUs. Probably most Core chips support AVX extensions. It’s the really old and simple chips that do not support AVX. I use an Atomic Pi which has a Celeron J processor I think that does not support AVX.

1 Like

@runningman84 if you trained a model to detect different people, it could. It’s really just a mechanism for running tensorflow models. You can train the models to recognize all kinds of stuff. Maybe someday I can add the training function to it.

3 Likes

Oh cool, thanks for the details man I appreciate it.

It would be great if you would provide some hints how to train a model to recognize persons and how to integrate that here.

if you have a coral the instructions are pretty straight forward

Can you post the link to your push to 0.99?

I have created an appdaemon app which triggers the image scanner based on sensors (like motion sensors or something like that):


Maybe this is also useful for other users.

1 Like

Looks very promising, great work getting it as a component.

I have set up robs version as a more accurate alarm system by listening to the save file event and passing back bounding box information. In my case tracking if the x_min value dropped below a certain amount an automation will trigger.

Would it be possible integrate something similar into DOODS to set up some events to pass back information such as object coordinates and the camera which triggered.

see:

1 Like

Hey @robmarkcole if I can get them to accept my merge, I’ll start adding features like events. I don’t want to interrupt the merge and lose my progress.

1 Like

I might have missed this reading through the thread, but since DOODS is already a docker container, any plans to make it available as a HassIO add-on?

1 Like

Yeah, I am working on the build process right now. Building tensorflow is kind of a pain, and building it for arm and aarch64 is even more of a pain. Once I have that working I will work on the hass.io thing next.

1 Like

I am running the laltest container on 0.99.b1 and I can see in the doods logs that it is detecting an object

Sep 16 14:49:53 ubuntu.local docker[23084]: 2019-09-16T18:49:53.132Z        DEBUG        tflite/detector.go:165        Got Image        {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 1920, "height": 1080}
Sep 16 14:49:53 ubuntu.local docker[23084]: 2019-09-16T18:49:53.132Z        DEBUG        tflite/detector.go:167        Resizing Image        {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 300, "height": 300}
Sep 16 14:49:53 ubuntu.local docker[23084]: 2019-09-16T18:49:53.527Z        DEBUG        tflite/detector.go:263        Detection        {"package": "detector.tflite", "id": "", "label": "car", "confidence": 50, "location": "%!d(float32=0.07881215),%!d(float32=0.25541472),%!d(float32=0.99492276),%!d(float32=0.96682394)"}
Sep 16 14:49:53 ubuntu.local docker[23084]: 2019-09-16T18:49:53.527Z        INFO        tflite/detector.go:266        Detection Complete        {"package": "detector.tflite", "id": "", "duration": 0.362563593, "detections": 1}
Sep 16 14:49:53 ubuntu.local docker[23084]: 2019-09-16T18:49:53.528Z        INFO        server/server.go:88        API Request        {"status": 200, "took": 0.473126473, "remote": "192.168.100.13:54048", "request": "/detect", "method": "POST", "package": "server.request", "request-id": "a5c7effbe9bb/FkKqxbcDyU-000003"}
Sep 16 14:51:54 ubuntu.local docker[23084]: 2019-09-16T18:51:54.138Z        DEBUG        tflite/detector.go:165        Got Image        {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 1920, "height": 1080}
Sep 16 14:51:54 ubuntu.local docker[23084]: 2019-09-16T18:51:54.138Z        DEBUG        tflite/detector.go:167        Resizing Image        {"package": "detector.tflite", "id": "", "format": "jpeg", "width": 300, "height": 300}
Sep 16 14:51:54 ubuntu.local docker[23084]: 2019-09-16T18:51:54.564Z        DEBUG        tflite/detector.go:263        Detection        {"package": "detector.tflite", "id": "", "label": "car", "confidence": 51.171875, "location": "%!d(float32=0.09170675),%!d(float32=0.25541472),%!d(float32=0.9934515),%!d(float32=0.96682394)"}
Sep 16 14:51:54 ubuntu.local docker[23084]: 2019-09-16T18:51:54.564Z        INFO        tflite/detector.go:266        Detection Complete        {"package": "detector.tflite", "id": "", "duration": 0.393365321, "detections": 1}
Sep 16 14:51:54 ubuntu.local docker[23084]: 2019-09-16T18:51:54.565Z        INFO        server/server.go:88        API Request        {"status": 200, "took": 0.500263875, "remote": "192.168.100.13:55516", "request": "/detect", "method": "POST", "package": "server.request", "request-id": "a5c7effbe9bb/FkKqxbcDyU-000004"}

but there is nothing in entry changing

you can see I created this template

root@ubuntu:/usr/share/hassio# cat cat.tmp
{{states.image_processing.doods_unifi_garage }}

which I then eval and its empty

root@ubuntu:/usr/share/hassio# hass-cli template cat.tmp


<template state image_processing.doods_unifi_garage=0; matches=, summary=, total_matches=0, friendly_name=Doods unifi_garage @ 2019-09-16T14:05:31.700746-04:00>
root@ubuntu:/usr/share/hassio#

This seems to be the same behavior @andreasfelder reported 16 days ago