DOODS integration. Is detecting the same label in two different areas of the same image possible?

I have a working DOODS server and have successfully integrated it into my HA configuration. It works great for most of my cameras, but I have an issue with working out the correct configuration for one specific situation.

I’m trying to detect people on my front yard and driveway. Unfortunately, my front yard has a fire hydrant in the middle of it that DOODS insists is a person. I’ve seen it detect it with as high as 82% confidence. Below is my current configuration for the image_processing and the resultant picture showing the hydrant marked as a person (green box).

- platform: doods
  source:
    entity_id: camera.camera2_last_eventid
  url: http://mydoodsserver:8080
  scan_interval: 10840
  timeout: 10
  detector: pytorch
  confidence: 50
  area:
    top: 0.27
    left: 0
    bottom: 1
    right: 1
    covers: false

What I am currently doing is to move the detection area to below where the hydrant is, such that it isn’t detected anymore. The problem with that solution is that I now miss the foot of my driveway in the detection area. My preference would be to set the detection to look at two or three areas that can avoid the hydrant but cover all the important locations. Indeed I am able to do just that manually on the doods server but I can’t seem to figure out how to translate that into a properly working configuration for the HA integration. In order to set more than a single detection area in the integration, I need to specify a ‘label’ (i.e. person) and if I set the same label more than once, it only appears to look at the last defined area and misses detections in the other areas.

Here’s the working JSON that the doods server accepts to detect ‘person’ in three separate detection areas of the image.

{
  "id": "manual",
  "detector_name": "pytorch",
  "preprocess": [],
  "detect": {
    "*": 100
  },
  "regions": [
    {
      "top": 0.328,
      "left": 0,
      "bottom": 1,
      "right": 1,
      "detect": {
        "person": 50
      },
      "covers": false
    },
    {
      "top": 0.26,
      "left": 0.53,
      "bottom": 0.328,
      "right": 1,
      "detect": {
        "person": 50
      },
      "covers": false
    },
    {
      "top": 0.27,
      "left": 0.26,
      "bottom": 0.328,
      "right": 0.49,
      "detect": {
        "person": 50
      },
      "covers": false
    }
  ]
}

Note the three distinct detection areas (purple boxes) in the image and the fact that it misses the hydrant altogether.

Here’s an example of the (non-working) code I thought would give me the same result.

- platform: doods
  source:
    entity_id: camera.camera2_last_eventid
    name: doods_test
  url: http://mydoodsserver:8080
  scan_interval: 10840
  timeout: 10
  detector: pytorch
  confidence: 50
  area:
    top: 0.328
    left: 0
    bottom: 1
    right: 1
    covers: false
  labels:
    - name: person
      area:
        top: 0.26
        left: 0.53
        bottom: 0.328
        right: 1
        covers: false
    - name: person
      area:
        top: 0.27
        left: 0.26
        bottom: 0.328
        right: 0.49
        covers: false

Does anyone know what the integration configuration would look like to mimic the results of the doods JSON shown above? Or any other pointers as to how to solve my problem would be welcome as well.

Thanks in advance.

Apologies for necro’ing this thread, but I’m facing a similar issue.
Did you ever resolve having multiple areas for a single label?

Nope. I never did figure it out. I pivoted to using node-red to send the entire image to doods and then test the returned results against an arbitrary polygon to see if any points lay within the polygon. It does what I was looking for but it took a fair amount of fussing about.

Here is the node-red flow I use. It watches my three exterior cameras for a doods image scan result. It checks the result and if a person is indicated, it then tests if any of the four x/y points for the person with the highest detection confidence exist within the defined polygon for the camera. If it does, it sets an appropriate sensor to true for five seconds and then resets it.

[
	{
		"id": "8199ce8bd50946ba",
		"type": "tab",
		"label": "Camera Person Detections",
		"disabled": false,
		"info": "",
		"env": []
	},
	{
		"id": "042aad9bc2f6fdb9",
		"type": "ha-binary-sensor",
		"z": "8199ce8bd50946ba",
		"name": "Inform Home Assistant of Camera1 detection state",
		"entityConfig": "ede7114924032712",
		"version": 0,
		"state": "payload",
		"stateType": "msg",
		"attributes": [],
		"inputOverride": "allow",
		"outputProperties": [],
		"x": 1590,
		"y": 360,
		"wires": [
			[]
		]
	},
	{
		"id": "1f3ea010195f0d7f",
		"type": "server-state-changed",
		"z": "8199ce8bd50946ba",
		"name": "Camera2 Detections",
		"server": "6585493a.4a98f8",
		"version": 4,
		"exposeToHomeAssistant": false,
		"haConfig": [
			{
				"property": "name",
				"value": ""
			},
			{
				"property": "icon",
				"value": ""
			}
		],
		"entityidfilter": "image_processing.doods_camera2_last_eventid",
		"entityidfiltertype": "exact",
		"outputinitially": false,
		"state_type": "num",
		"haltifstate": "0",
		"halt_if_type": "num",
		"halt_if_compare": "gt",
		"outputs": 2,
		"output_only_on_state_change": false,
		"for": "0",
		"forType": "num",
		"forUnits": "minutes",
		"ignorePrevStateNull": false,
		"ignorePrevStateUnknown": false,
		"ignorePrevStateUnavailable": false,
		"ignoreCurrentStateUnknown": true,
		"ignoreCurrentStateUnavailable": true,
		"outputProperties": [
			{
				"property": "payload",
				"propertyType": "msg",
				"value": "",
				"valueType": "entityState"
			},
			{
				"property": "data",
				"propertyType": "msg",
				"value": "",
				"valueType": "eventData"
			},
			{
				"property": "topic",
				"propertyType": "msg",
				"value": "",
				"valueType": "triggerId"
			}
		],
		"x": 90,
		"y": 420,
		"wires": [
			[
				"721556fed3b2e251"
			],
			[]
		]
	},
	{
		"id": "b180b4c6aae10e24",
		"type": "server-state-changed",
		"z": "8199ce8bd50946ba",
		"name": "Camera3 Detections",
		"server": "6585493a.4a98f8",
		"version": 4,
		"exposeToHomeAssistant": false,
		"haConfig": [
			{
				"property": "name",
				"value": ""
			},
			{
				"property": "icon",
				"value": ""
			}
		],
		"entityidfilter": "image_processing.doods_camera3_last_eventid",
		"entityidfiltertype": "exact",
		"outputinitially": false,
		"state_type": "num",
		"haltifstate": "0",
		"halt_if_type": "num",
		"halt_if_compare": "gt",
		"outputs": 2,
		"output_only_on_state_change": false,
		"for": "0",
		"forType": "num",
		"forUnits": "minutes",
		"ignorePrevStateNull": false,
		"ignorePrevStateUnknown": false,
		"ignorePrevStateUnavailable": false,
		"ignoreCurrentStateUnknown": true,
		"ignoreCurrentStateUnavailable": true,
		"outputProperties": [
			{
				"property": "payload",
				"propertyType": "msg",
				"value": "",
				"valueType": "entityState"
			},
			{
				"property": "data",
				"propertyType": "msg",
				"value": "",
				"valueType": "eventData"
			},
			{
				"property": "topic",
				"propertyType": "msg",
				"value": "",
				"valueType": "triggerId"
			}
		],
		"x": 90,
		"y": 480,
		"wires": [
			[
				"721556fed3b2e251"
			],
			[]
		]
	},
	{
		"id": "721556fed3b2e251",
		"type": "function",
		"z": "8199ce8bd50946ba",
		"name": "Remove extraneous data",
		"func": "const entity_id = msg.data.entity_id;\nconst matches = msg.data.new_state.attributes.matches;\nvar index = null;\nswitch (entity_id) {\n    case \"image_processing.doods_camera1_last_eventid\":\n        index = 1;\n        break;\n    case \"image_processing.doods_camera2_last_eventid\":\n        index = 2;\n        break;\n    case \"image_processing.doods_camera3_last_eventid\":\n        index = 3;\n        break;\n};\n//node.warn({ \"entity_id\": entity_id, \"index\": index });\nreturn { payload: { entity_id: entity_id, index: index, matches: matches } };",
		"outputs": 1,
		"noerr": 0,
		"initialize": "",
		"finalize": "",
		"libs": [],
		"x": 330,
		"y": 420,
		"wires": [
			[
				"63bf39d22c670fcd"
			]
		]
	},
	{
		"id": "b696b8b77c063e40",
		"type": "server-state-changed",
		"z": "8199ce8bd50946ba",
		"name": "Camera1 Detections",
		"server": "6585493a.4a98f8",
		"version": 4,
		"exposeToHomeAssistant": false,
		"haConfig": [
			{
				"property": "name",
				"value": ""
			},
			{
				"property": "icon",
				"value": ""
			}
		],
		"entityidfilter": "image_processing.doods_camera1_last_eventid",
		"entityidfiltertype": "exact",
		"outputinitially": false,
		"state_type": "num",
		"haltifstate": "0",
		"halt_if_type": "num",
		"halt_if_compare": "gt",
		"outputs": 2,
		"output_only_on_state_change": false,
		"for": "0",
		"forType": "num",
		"forUnits": "minutes",
		"ignorePrevStateNull": false,
		"ignorePrevStateUnknown": false,
		"ignorePrevStateUnavailable": false,
		"ignoreCurrentStateUnknown": true,
		"ignoreCurrentStateUnavailable": true,
		"outputProperties": [
			{
				"property": "payload",
				"propertyType": "msg",
				"value": "",
				"valueType": "entityState"
			},
			{
				"property": "data",
				"propertyType": "msg",
				"value": "",
				"valueType": "eventData"
			},
			{
				"property": "topic",
				"propertyType": "msg",
				"value": "",
				"valueType": "triggerId"
			}
		],
		"x": 90,
		"y": 360,
		"wires": [
			[
				"721556fed3b2e251"
			],
			[]
		]
	},
	{
		"id": "63bf39d22c670fcd",
		"type": "switch",
		"z": "8199ce8bd50946ba",
		"name": "Were people detected?",
		"property": "payload.matches.person",
		"propertyType": "msg",
		"rules": [
			{
				"t": "nnull"
			},
			{
				"t": "else"
			}
		],
		"checkall": "true",
		"repair": false,
		"outputs": 2,
		"x": 590,
		"y": 420,
		"wires": [
			[
				"68eab4ac5c458725",
				"2289292948443639"
			],
			[
				"d42fe572aa92dab7"
			]
		]
	},
	{
		"id": "68eab4ac5c458725",
		"type": "debug",
		"z": "8199ce8bd50946ba",
		"name": "debug 6",
		"active": false,
		"tosidebar": true,
		"console": false,
		"tostatus": false,
		"complete": "true",
		"targetType": "full",
		"statusVal": "",
		"statusType": "auto",
		"x": 800,
		"y": 300,
		"wires": []
	},
	{
		"id": "1c01861e5e65fca0",
		"type": "debug",
		"z": "8199ce8bd50946ba",
		"name": "debug 7",
		"active": false,
		"tosidebar": true,
		"console": false,
		"tostatus": false,
		"complete": "true",
		"targetType": "full",
		"statusVal": "",
		"statusType": "auto",
		"x": 1060,
		"y": 540,
		"wires": []
	},
	{
		"id": "d42fe572aa92dab7",
		"type": "function",
		"z": "8199ce8bd50946ba",
		"name": "No people detected",
		"func": "const state = false;\nconst index = msg.payload.index;\nreturn { payload: { index: index, state: state } };",
		"outputs": 1,
		"noerr": 0,
		"initialize": "",
		"finalize": "",
		"libs": [],
		"x": 870,
		"y": 480,
		"wires": [
			[
				"1c01861e5e65fca0",
				"57697b957810e845"
			]
		]
	},
	{
		"id": "57697b957810e845",
		"type": "switch",
		"z": "8199ce8bd50946ba",
		"name": "Route to correct sensor",
		"property": "payload.index",
		"propertyType": "msg",
		"rules": [
			{
				"t": "eq",
				"v": "1",
				"vt": "num"
			},
			{
				"t": "eq",
				"v": "2",
				"vt": "num"
			},
			{
				"t": "eq",
				"v": "3",
				"vt": "num"
			}
		],
		"checkall": "false",
		"repair": false,
		"outputs": 3,
		"x": 1250,
		"y": 420,
		"wires": [
			[
				"31315a2c0ce3f89b",
				"042aad9bc2f6fdb9"
			],
			[
				"3a886e3e99b0111f",
				"8a5e3a6d4a8eefd8"
			],
			[
				"7860f5b35db9965b",
				"702ae24513cea079"
			]
		]
	},
	{
		"id": "31315a2c0ce3f89b",
		"type": "debug",
		"z": "8199ce8bd50946ba",
		"name": "debug 8",
		"active": false,
		"tosidebar": true,
		"console": false,
		"tostatus": false,
		"complete": "true",
		"targetType": "full",
		"statusVal": "",
		"statusType": "auto",
		"x": 1460,
		"y": 240,
		"wires": []
	},
	{
		"id": "3a886e3e99b0111f",
		"type": "debug",
		"z": "8199ce8bd50946ba",
		"name": "debug 9",
		"active": false,
		"tosidebar": true,
		"console": false,
		"tostatus": false,
		"complete": "true",
		"targetType": "full",
		"statusVal": "",
		"statusType": "auto",
		"x": 1460,
		"y": 280,
		"wires": []
	},
	{
		"id": "7860f5b35db9965b",
		"type": "debug",
		"z": "8199ce8bd50946ba",
		"name": "debug 10",
		"active": false,
		"tosidebar": true,
		"console": false,
		"tostatus": false,
		"complete": "true",
		"targetType": "full",
		"statusVal": "",
		"statusType": "auto",
		"x": 1460,
		"y": 320,
		"wires": []
	},
	{
		"id": "2289292948443639",
		"type": "function",
		"z": "8199ce8bd50946ba",
		"name": "Is person in zone?",
		"func": "/*\n * Test each vertex point in box array for each person in person array.\n * If any point exists inside zone, consider person to be in zone and\n * set state to TRUE\n */\n\nvar state = false;\nconst index = msg.payload.index;\nconst people = msg.payload.matches.person;\n\nvar zone = [[],[],[],[]];\nzone[1] = [\n    [0.000, 0.648],\n    [0.460, 0.336],\n    [0.589, 0.311],\n    [0.589, 0.658],\n    [0.723, 0.658],\n    [0.723, 0.498],\n    [0.590, 0.498],\n    [0.590, 0.311],\n    [1.000, 0.311],\n    [1.000, 0.437],\n    [0.941, 0.437],\n    [0.941, 0.552],\n    [1.000, 0.552],\n    [1.000, 0.800],\n    [0.895, 0.800],\n    [0.895, 1.000],\n    [0.000, 1.000]];\nzone[2] = [\n    [0.000, 0.330],\n    [0.417, 0.146],\n    [0.537, 0.198],\n    [0.537, 0.318],\n    [0.574, 0.318],\n    [0.574, 0.224],\n    [0.700, 0.304],\n    [1.000, 0.510],\n    [1.000, 1.000],\n    [0.000, 1.000]];\nzone[3] = [\n    [0.365, 0.000],\n    [1.000, 0.000],\n    [1.000, 1.000],\n    [0.150, 1.000]];\n\n/*\n * Performs the even-odd-rule Algorithm (a raycasting algorithm) to find out whether a point is in a given polygon.\n * This runs in O(n) where n is the number of edges of the polygon.\n *\n * @param {Array} polygon an array representation of the polygon where polygon[i][0] is the x Value of the i-th point and polygon[i][1] is the y Value.\n * @param {Array} point   an array representation of the point where point[0] is its x Value and point[1] is its y Value\n * @return {boolean} whether the point is in the polygon (not on the edge, just turn < into <= and > into >= for that)\n */\nconst pointInPolygon = function (polygon, point) {\n    //A point is in a polygon if a line from the point to infinity crosses the polygon an odd number of times\n    let odd = false;\n    //For each edge (In this case for each point of the polygon and the previous one)\n    for (let i = 0, j = polygon.length - 1; i < polygon.length; i++) {\n        //If a line from the point into infinity crosses this edge\n        if (((polygon[i][1] >= point[1]) !== (polygon[j][1] >= point[1])) // One point needs to be above, one below our y coordinate\n            // ...and the edge doesn't cross our Y coordinate before our x coordinate (but between our x coordinate and infinity)\n            && (point[0] <= ((polygon[j][0] - polygon[i][0]) * (point[1] - polygon[i][1]) / (polygon[j][1] - polygon[i][1]) + polygon[i][0]))) {\n            // Invert odd\n            odd = !odd;\n        }\n        j = i;\n\n    }\n    //If the number of crossings was odd, the point is in the polygon\n    return odd;\n};\n\npeople.forEach((person) => {\n    //node.warn({\"person\": person});\n    const top = person.box[0];\n    const left = person.box[1];\n    const bottom = person.box[2];\n    const right = person.box[3];\n    const box = [[left, top], [right, top], [right, bottom], [left, bottom]];\n\n    box.forEach((test_point) => {\n        var inside = pointInPolygon(zone[index], test_point);\n        if (inside) { state = true };\n        //node.warn({ \"inside\": inside, \"x\": test_point[0], \"y\": test_point[1] });\n    });\n});\n\nreturn { payload: { index: index, state: state } };\n",
		"outputs": 1,
		"noerr": 0,
		"initialize": "",
		"finalize": "",
		"libs": [],
		"x": 870,
		"y": 360,
		"wires": [
			[
				"c527930f9c02eb23",
				"8c74125c8ccdb5af",
				"57697b957810e845"
			]
		]
	},
	{
		"id": "c527930f9c02eb23",
		"type": "debug",
		"z": "8199ce8bd50946ba",
		"name": "debug 11",
		"active": false,
		"tosidebar": true,
		"console": false,
		"tostatus": false,
		"complete": "true",
		"targetType": "full",
		"statusVal": "",
		"statusType": "auto",
		"x": 1060,
		"y": 300,
		"wires": []
	},
	{
		"id": "8c74125c8ccdb5af",
		"type": "delay",
		"z": "8199ce8bd50946ba",
		"name": "",
		"pauseType": "delay",
		"timeout": "5",
		"timeoutUnits": "seconds",
		"rate": "1",
		"nbRateUnits": "1",
		"rateUnits": "second",
		"randomFirst": "1",
		"randomLast": "5",
		"randomUnits": "seconds",
		"drop": false,
		"allowrate": false,
		"outputs": 1,
		"x": 900,
		"y": 420,
		"wires": [
			[
				"d42fe572aa92dab7"
			]
		]
	},
	{
		"id": "8a5e3a6d4a8eefd8",
		"type": "ha-binary-sensor",
		"z": "8199ce8bd50946ba",
		"name": "Inform Home Assistant of Camera2 detection state",
		"entityConfig": "37b5641b58e0566e",
		"version": 0,
		"state": "payload",
		"stateType": "msg",
		"attributes": [],
		"inputOverride": "allow",
		"outputProperties": [],
		"x": 1590,
		"y": 420,
		"wires": [
			[]
		]
	},
	{
		"id": "702ae24513cea079",
		"type": "ha-binary-sensor",
		"z": "8199ce8bd50946ba",
		"name": "Inform Home Assistant of Camera3 detection state",
		"entityConfig": "a41e18550cfa0e1e",
		"version": 0,
		"state": "payload",
		"stateType": "msg",
		"attributes": [],
		"inputOverride": "allow",
		"outputProperties": [],
		"x": 1590,
		"y": 480,
		"wires": [
			[]
		]
	},
	{
		"id": "ede7114924032712",
		"type": "ha-entity-config",
		"server": "6585493a.4a98f8",
		"deviceConfig": "",
		"name": "camera1_last_eventid_person_detected",
		"version": "6",
		"entityType": "binary_sensor",
		"haConfig": [
			{
				"property": "name",
				"value": "Camera1 Person Detected"
			},
			{
				"property": "icon",
				"value": ""
			},
			{
				"property": "entity_category",
				"value": ""
			},
			{
				"property": "device_class",
				"value": ""
			}
		],
		"resend": false,
		"debugEnabled": false
	},
	{
		"id": "6585493a.4a98f8",
		"type": "server",
		"name": "Home Assistant",
		"addon": true
	},
	{
		"id": "37b5641b58e0566e",
		"type": "ha-entity-config",
		"server": "6585493a.4a98f8",
		"deviceConfig": "",
		"name": "camera2_last_eventid_person_detected",
		"version": "6",
		"entityType": "binary_sensor",
		"haConfig": [
			{
				"property": "name",
				"value": "Camera2 Person Detected"
			},
			{
				"property": "icon",
				"value": ""
			},
			{
				"property": "entity_category",
				"value": ""
			},
			{
				"property": "device_class",
				"value": ""
			}
		],
		"resend": false,
		"debugEnabled": false
	},
	{
		"id": "a41e18550cfa0e1e",
		"type": "ha-entity-config",
		"server": "6585493a.4a98f8",
		"deviceConfig": "",
		"name": "camera3_last_eventid_person_detected",
		"version": "6",
		"entityType": "binary_sensor",
		"haConfig": [
			{
				"property": "name",
				"value": "Camera3 Person Detected"
			},
			{
				"property": "icon",
				"value": ""
			},
			{
				"property": "entity_category",
				"value": ""
			},
			{
				"property": "device_class",
				"value": ""
			}
		],
		"resend": false,
		"debugEnabled": false
	}
]
1 Like

Thanks for responding.

I’ve never got round to digging in node-red (and the thought of migrating around 4 years of finely tuned yaml automations fills me with horror :grinning_face_with_smiling_eyes:).

I guess I’ll look to reinstate some functionality I had in my deepstack automations before I switched to Doods, which does similar work to your node red flow from the sounds of it.

That’s the beauty of the node-red integration; you don’t have to migrate everything over. Just use it where it makes sense. This flow is literally the only one I have currently running in node-red, compared to the 100’s of other sensors, templates and automations I have configured. I use YAML for the simple stuff and if any sensor or automation is too complex for the native stuff, I look at using node-red.