New Custom Compontent - Image Processing - Object Detection - DOODS

I have now tried to run docker exec -it -e LOGGER_LEVEL='debug' addon_d5f40609_doods bash, but that doesn’t really change anything. Still not getting debug entries in the log. Can I set the environment variable any other way?

I opened the DOODS addon container in Portainer and added an environment variable LOGGER_LEVEL set to debug. Let Portainer replace the existing container with the new one, and then checked the container log and am seeing the increased logging. Same log should be what you see if you check the log for the DOODS addon in HA.

Thanks! Took a bit to figure out what you meant, but I think I did it.

The full log can be found here: https://pastebin.com/J3D3rjwf. I have included a snippet below, since this the log looks like a repeating pattern.

I haven’t changed my HA config from from the last one. Something I noticed is the WARN entry that talks about a missing label. I didn’t see that one when I checked my logs yesterday.

2021-03-24T05:24:25.399Z	DEBUG	tensorflow/tensorflow.go:207	Detection	{"package": "detector.tensorflow", "name": "tensorflow", "scores": [0.985551, 0.8421051, 0.8397212, 0.55919105, 0.48267362, 0.43471223, 0.42716938, 0.3822009, 0.35555902, 0.34410286, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "classes": [62, 62, 63, 62, 44, 67, 44, 67, 84, 63, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], "locations": [[0.25625303,0.4646785,0.5884545,0.6170678],[0.36208946,0.75828826,0.63467395,0.8442949],[0.36922437,0.010421276,0.9797347,0.31248572],[0.36866656,0.8316601,0.6364439,0.9090653],[0.5025996,0.29972798,0.59729755,0.31670398],[0.42949525,0.24584639,0.8206326,0.4941054],[0.5069038,0.30609703,0.597768,0.32229543],[0.33750704,0.76770705,0.61444306,0.9121719],[0.1173751,0.39867806,0.3617446,0.51284426],[0.31654668,0.0407618,0.6584834,0.38955843],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0],[0,0,0,0]], "count": 10}

2021-03-24T05:24:25.399Z	WARN	tensorflow/tensorflow.go:214	Missing label	{"package": "detector.tensorflow", "name": "tensorflow", "index": 84}

2021-03-24T05:24:25.399Z	INFO	tensorflow/tensorflow.go:228	Detection Complete	{"package": "detector.tensorflow", "name": "tensorflow", "id": "", "duration": 0.350108723, "detections": 10}

2021-03-24T05:24:25.400Z	INFO	server/server.go:139	HTTP Request	{"status": 200, "took": 0.35902036, "request": "/detect", "method": "POST", "package": "server.request", "request-id": "d5f40609-doods/MrL9FpIU2M-000003", "remote": "192.168.1.252:58140"}

The documentation on the model labels/classes is pretty bad so that’s what I’ve pieced together…

This is your scores: "scores": [0.985551, 0.8421051, 0.8397212, 0.55919105..... from 0-1 with 1 being 100% match
This is your classes: "classes": [62, 62, 63, 62, 44, 67, 44, 67, .... which cooresponds to a class/label in the file. I don’t even see a person in that match. Mostly just tv and toilet I think lol. Does that seem right?
Here are the labels and the classes coorespond to the line numbers: https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt

Well, there is no toilet in the picture, and you barely see the TV. It looks at the TV from the top left, so you only see a part of the bezel, none of the screen. Really weird!

Stupid question, i have tried an aqara camera connected to homekit controller integration, dont think camera is supported and its super laggy. But anyways its just for testing.

Got DOODS up and running and made a basic automation

binary sensor -> on -> call image processing (dont know if this is needed) send a image to phone.

However how can i make this smarter, so i can set a title cat detected, and human detected, instead of just motion detected?

hI guys,
can someone please help.
I’ve set up a camera and I’m trying to track just my driveway. as you can see it see my wifes car and reports 1
Screenshot 2021-03-26 112716
yet on the next scan it say’s there are zero cars , then later shows 1 car again

image_processing:
  - platform: doods
    scan_interval: 5
    url: "http://10.0.0.9:8080"
    detector: default
    source:
      - entity_id: camera.embedded_net_dvr_profilename009
    file_out:
      - "/config/www/sensor_scripts/{{ camera_entity.split('.')[1] }}_latest.jpg"
#      - "/config/www/sensor_scripts/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
      - "/config/www/sensor_scripts/{{ camera_entity.split('.')[1] }}_.jpg"
    confidence: 60
    area:
      top: 0.09
      left: 0.17
      # The entire detection must be inside this box
#      covers: true
    labels:
      - name: person
        confidence: 40
#        area:
      - name: car
        confidence: 50
      - truck

what am I doing wrong?
I would like to use the detection as a automation trigger

Does anyone know how to take multiple snapshots (say, 5 within 5 seconds), run them through DOODS and then email them? Thanks!

Hi, I’m quite late to this but just can not get the home assistant image processing to connect to doods addon. As I understand the problem is that I already have some addons that depend on port 8080. Sadly if I change the port on this addon configuration as well as the home assistant configuration I get the error “connection refused”
Sadly changing the port on other addons also not work without errors. Is there something I’m missing here?
The log with changed port. Already tried a lot of others but only on 8080 I get ip:8080/detectors webpage still with error HTTP Status 404 – Not Found but that’s way better than nothing at all on all other ports.

Logger: homeassistant.components.image_processing
Source: components/doods/image_processing.py:83
Integration: Image processing (documentation, issues)
First occurred: April 3, 2021, 11:52:23 PM (1 occurrences)
Last logged: April 3, 2021, 11:52:23 PM

Error while setting up doods platform for image_processing
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.8/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.8/http/client.py", line 1010, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.8/http/client.py", line 950, in send
    self.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f626044a760>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 573, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='192.168.1.20', port=8765): Max retries exceeded with url: /detectors (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f626044a760>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 200, in _async_setup_platform
    await asyncio.shield(task)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/doods/image_processing.py", line 83, in setup_platform
    doods = PyDOODS(url, auth_key, timeout)
  File "/usr/local/lib/python3.8/site-packages/pydoods/__init__.py", line 13, in __init__
    self.get_detectors()
  File "/usr/local/lib/python3.8/site-packages/pydoods/__init__.py", line 20, in get_detectors
    response = requests.get(self.url + "/detectors", **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.1.20', port=8765): Max retries exceeded with url: /detectors (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f626044a760>: Failed to establish a new connection: [Errno 111] Connection refused'))

Is there anything I can try to fix this?

Do anybody know if there’s a way to make a non-horizontal top area line? so it is one percentage in the left and another in the right side?

Anyone using DOODS for bird identification? I can’t seem to find a model that DOODS can use.

Ok, I have this crazy idea and I think I need a little help.
Home assistant/doods/node red; a motion sensor triggers a flow
in NR and when there is a person a photos is stored.
Using a state-node (msg.data.attributes.summary) and a switch-node it was pretty easy to do.

Now the next step; I would like to inspect the parameters of the
bounding box because, if the person is almost out of frame. I would
like to adjust the camera accordingly (PTZ). So that it ‘follows’
How to get these bounding box parameters?

Hope someone smarter than me can get me on track.

Thanks in advance.

Regards,

NongSung

EDIT. Disregard.

Hello,
Im new here and trying to setup integration on hassio, on RPI4. I’ve installed DOODS using Hass.IO Repository for snowzach.
I have problem there is no any entity created for DOODS.:


Logs for DOODS looks like this:

2021-05-22T06:53:07.990+0200	INFO	detector/detector.go:79	Configured Detector	{"package": "detector", "name": "default", "type": "tflite", "model": "/opt/doods/models/coco_ssd_mobilenet_v1_1.0_quant.tflite", "labels": 80, "width": 300, "height": 300}
2021-05-22T06:53:09.323+0200	INFO	detector/detector.go:79	Configured Detector	{"package": "detector", "name": "tensorflow", "type": "tensorflow", "model": "/opt/doods/models/faster_rcnn_inception_v2_coco_2018_01_28.pb", "labels": 65, "width": -1, "height": -1}
2021-05-22T06:53:09.325+0200	INFO	server/server.go:284	API Listening	{"package": "server", "address": ":8080", "tls": false, "version": "v0.2.6-0-gb2a1c53-dirty"}

and my config is pretty simple:

image_processing:
  - platform: doods
    scan_interval: 10800
    url: "http://192.168.8.108/:8080"
    detector: default
    source:
      - entity_id: camera.drive_way
    file_out:
      - "/media/{{ camera_entity.split('.')[1] }}_latest.jpg"
      - "/media/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
    confidence: 50
    labels:
      - name: person
        confidence: 40
      - car
      - truck

Any idea what I’m do ing wrong?
Thanks

Hello all,

I have just started with DOODS and am now running into a problem that I cannot solve. My server has multiple docker containers and without DOODS it ran like a charm, now I have implemented DOODS in HA and now after an hour the server crashes, I can’t even access the server with SSH anymore. To test I have moved DOODS to another server and set the scan_interval from 2 to 6 and then to 10. nothing works, anyone have an idea ??

I don’t think docker container should impact host greatly unless you a lot ridiculous resources to it and I would expect container crash first

Maybe it unrelated to docker and just show when add doods like out of storage.

You have many old container images and need remove unused?

Post docker run command or compose

Without investigating much… I’m betting you’re running out of memory and it’s starting to swap and destroys the machine. What kind of hardware are you running on?

Does anyone know if you can get the additional models when using the add-on or does this only work when you have manually installed in docker?

Also does that need to be ran inside the add-ons container?

Issue solved. There was a typo in line:

> url: "http://192.168.8.108/:8080"

…not needed “/”

I know I’m late, but heres all the things the default model can detect.

  • person
  • bicycle
  • car
  • motorcycle
  • airplane
  • bus
  • train
  • truck
  • boat
  • traffic light
  • fire hydrant
  • stop sign
  • parking meter
  • bench
  • bird
  • cat
  • dog
  • horse
  • sheep
  • cow
  • elephant
  • bear
  • zebra
  • giraffe
  • backpack
  • umbrella
  • handbag
  • tie
  • suitcase
  • frisbee
  • skis
  • snowboard
  • sports ball
  • kite
  • baseball bat
  • baseball glove
  • skateboard
  • surfboard
  • tennis racket
  • bottle
  • wine glass
  • cup
  • fork
  • knife
  • spoon
  • bowl
  • banana
  • apple
  • sandwich
  • orange
  • broccoli
  • carrot
  • hot dog
  • pizza
  • donut
  • cake
  • chair
  • couch
  • potted plant
  • bed
  • dining table
  • toilet
  • tv
  • laptop
  • mouse
  • remote
  • keyboard
  • cell phone
  • microwave
  • oven
  • toaster
  • sink
  • refrigerator
  • book
  • clock
  • vase
  • scissors
  • teddy bear
  • hair drier
  • toothbrush

source: https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt