Hi all
just published image classification with Clarifai. You can configure which model you want to use, then have an entity which displays the most likely concept within a camera image. Optionally you can configure particular concepts of interest, and the component will fire an event when those concepts are identified in an image.
Please note this is a work in progress. The ultimate goal is a completely locally hosted classifier.
Cheers
6 Likes
Great work , thanks
To do - setup classification as a service so it can be called by automations when needed
Are you going to use this for your machine learning project(s) too?
You seem to be doing some cool things in these regards. I’d like to get more involved in this but unsure where to start? Just recently got influxdb set up so dont have much data to use yet but maybe this will somehow be used in some bayesian sensors?
Testing out this component would be helpful, and suggesting improvements and new use cases?
I’ll give it a shot for sure.
Not particularly creative but one idea would be to use the ‘logo’ model to monitor and keep a bar stocked. Say a bottle gets removed, the component will update to remove that brand from the list and alert you to buy another bottle or automatically add it to a shopping list.
Could also work with a pantry.
1 Like
EDIT I can get it to work fine with the General Model ID (and the photo from your Github) but not the Logo Model ID
Tried setting it up. Getting an error:
2018-03-15 23:39:54 ERROR (MainThread) [homeassistant.helpers.entity] Update for image_processing.clarifai_pantry fails
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 204, in async_update_ha_state
yield from self.async_device_update()
File "/usr/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 325, in async_device_update
yield from self.async_update()
File "/usr/lib/python3.6/site-packages/homeassistant/components/image_processing/__init__.py", line 136, in async_update
yield from self.async_process_image(image)
File "/usr/lib/python3.6/asyncio/futures.py", line 332, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup
future.result()
File "/usr/lib/python3.6/asyncio/futures.py", line 245, in result
raise self._exception
File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/config/custom_components/image_processing/clarifai.py", line 85, in process_image
data = response['outputs'][0]['data']['concepts']
KeyError: 'concepts'It looks like that model returns data in a different format or with a different key (probably ‘labels’). Thanks for the heads up. 
Hmm that’s weird, hope it’s an easy fix.
Is there a good way to ‘turn off’ the scanning since the limit is 5000/month? I have the interval at 1000 seconds but i was thinking maybe i can create an input select or something to turn it off when not in use and then get the scan interval more reasonable, like 60 seconds
You can set scan_interval to some other value. Need to figure out the best way to use clarification - will probably rewrite as a service
Super cool component - I think a service would be a lot more useful.
Ie on a motion trigger, take snapshot from camera and process that.
1 Like
This would be great in a security setting, for example an automation trigger that is fired with a person appears in the picture, but not a cat or dog.
1 Like
Exactly, my dog is pretty big he sets off all the motion sensors 
Another possible use case, if my vacuum gets stuck and gives error, see if it can identify it on the security cameras and alert me to the location w/ photo.
1 Like