Object detection with Amazon Rekognition

Hi all
I’ve written a custom component for performing object detection with the Amazon Rekognition service. You can optionally configure a target object (the default is Person), and the state of the sensor is the number of those objects found in the image. All object classes with greater than 50% probability are listed in the sensor attributes.

This is a cloud service but damn fast in my testing, just over a second to find objects in images, although the exact time does seem to vary. The free tier is quite generous, but you will still want to limit computation and not just use the default 10 seconds scan_interval. Looking for feedback to get this out as an official integration.


I tried something similar too, trying to reduce the number of false alarms on my CCTV. Did you also try Google’s API to do the same? I got much better recognition results with that, neither however managed to recognise a cat, although Google did classify it as a “pet”.

Amazon on the other hand saw suitcases and all sorts of stuff that wasn’t there :smiley:

I’ll try and give your component a go over Christmas.

Yes none of these models get it right all the time, classic approach is to use several models in an ‘ensemble’

Thanks for all your work. If understand correctly I can use this with Hass.io? I am using the dafang camera with custom FW. Can one use this only when a snapshot is taken by the dafang? Is there anyway to see the number of api calls used within HA?


Yes any HA platform, its just posting the image bytes to a rest API. I don’t know if you can see the number of API calls but will look into it

1 Like

Is this the best solution when running hass.io (on pi) for getting image processing or what do you say? Tensorflow not being available for hass.io.

I would say so yes

I’ll give this a go I’m currently using machine box am interested in the ensemble approach you mentioned. I imagine this is going to be like presence detection and require some kind of bayesian to bring them together.

The only official machinebox integration is for Facebox (Faces), although I also published one for Classification box (classes). Amazon recognition is for objects, including a face or person, but amazon have a separate service for recognising specific faces. I’m interested to know which models you would want to bring together?

So, seems to work just fine!

Target is what will be shown regarding result (number of hits of target)? If I put car there it will show number of cars seen? It will always look for everything ie persons and whatnot which I can see in attributes?

Will the component in the future show bounding boxes? Ie return image with bounding boxes?

Anyways, so far really good. Got an error message when picking eu-north1 one but works with eu-west-1 other than that it is all good.


Great! Yes state is the number of target objects identified.
Bounding boxes needs to be addressed at the platform level.
I also noticed some errors with regions, might have it default to USA if error.

Ok, thanks!

So this works really well so far. So some questions.

Is 20000 the highest one can set to save on api calls?

If one is to add another camera. What to do? Add a new section with different source ? Need to give it a name: then? Or add entity_id to current?

How can one detect if image recognition has returned new values? I want to send a notification but if state is 1 ie it has detected a person and it runs again and detects 1 persone again I cant trigger on state right?

Whats the best way to pull all of the attributes to be able to send it in a notification?

Thanks, really excited about this!

Sorry I didn’t explain myself properly, I indeed use machinebox for facial recognition, I am however interested in the ensemble approach to object recognition for my fridge I currently have a pi zero with camera in the fridge trying to identify used items to generate an automated shopping list, whilst it works for items I place in the same spot this might help to work out what if my wife or kids aren’t anal retentive like myself and put the milk in the same spot in the fridge door every time.


Congratulations Rob,

The component is very useful and it’s working well! :slight_smile:

I have the same doubts of M203 about how to setup the component on HA.
If I want to add another camera, what I need to do on Configuration.yaml file? Add a new section with different source? Need to give it a name?

Is it possible detects more that 1 target? How configure more than 1 target on Congifuration.yaml?

How can create an automation using the return of detection? For example, if a person was detected, I would like turn on the light.

Thanks a lot!

1 Like

You can just list the cameras. Only 1 target, but you can create multiple entities each with a different target.
For person detection just use a regular automation on the state, no?

Great thanks!

But state wont work right? If it detects one person and then next time it detects one person again then state is same or?

Oh you mean you want to detect different people? That’s not possible yet. You would need to detect the face of the person using one of the face recognition components.

No not different. But if state is 1 and new recognition is 1 then state us same. How can I trigger notification on state then?

I’d use a binary sensor in that case

So been running this now for little over a week and really like it. Regarding bounding boxes, is this something you will try to implement? Amazon does provide it?

Thanks for a great component for us hassio users not able to use tensorflow!