I am trying to install tensorflow on NUC(nuc7pjyh) running on Ubuntu 18.04, but got stuck on “The TensorFlow library was compiled to use AVX instructions, but these aren’t available on your machine.”
I used @cooloo’s method and installed tensorflow==1.5.0 and it worked. Thank you
Actually when looking further this is the reason (from the tensorflow page)
I have been having an issue where it will be running fine, then just say it cant get image… The disk is not full, the cpu is super powerful, and there is plenty of memory. I get no other errors…
Anyone else?
Is there a limit to the number of times you can call tensorflow?
There is no artificial limit, must be an error somewhere
I’m running in a pythong venv on an older I3 with 4gb ram on an SSD. I followed the guide and feel like everything installed fine. No major issues. I now have an entity image_processing.tensorflow_front_door which is pointing to my front door camera. I dont get any errors, but when I manually initiate a scan using the services menu, nothing happens. The entity timestamp does not change, no image, nothing. Any thoughts on where to start troubleshooting?
my config
image_processing:
- platform: tensorflow
scan_interval: 20000
source:
- entity_id: camera.front_door
model:
graph: /home/homeassistant/.homeassistant/tensorflow/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb
OK, might just be there was not much in my camera picture at the moment. I added another camera and did get back objects. But no image for me to see what it thinks it sees. How do you get that?
Edit:
OK, so I added two cameras. They are different model cameras. but both show up in HA just fine. One of them just does not get processed by tensorflow. No errors, nothing happens.
I updated my config
image_processing:
- platform: tensorflow
scan_interval: 20000
source:
- entity_id: camera.front_door
- entity_id: camera.back_gate
file_out:
- "/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "/tmp/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
model:
graph: /home/homeassistant/.homeassistant/tensorflow/faster_rcnn_inception_v2_coco_2018_01_28/frozen_inference_graph.pb
When I process a scan for the front door, I get nothing in /tmp, no processing. The other camera does process and puts an image there.
What am I missing? Is there an image/camera requirement?
Here is my camera
- platform: generic
still_image_url: http://192.168.1.xxx/ISAPI/Streaming/channels/101/picture
name: Front Door
username: admin
password: !secret camera_doorbell_password
authentication: basic
Ok, I’m sure I have a weird typo or am missing something very simple. Just to rule out my front door camera, I created a local file camera using a snapshot of my front door. It also fails to process, nothing happens.
I guess I have a question about the order of processing. If the image is analyzed and nothing is there, will it still output a picutre, but with no boxes?
Edit: Nevermind, It was that there really was nothing to process. It’s cold out so I did not want to go out and stand in front of the camera, but I did, and it did process a person.
So… If there is nothing to process, and snapshot will not be taken
Edit2: more questions
Couple of quick questions about tensorflow and configuration.
So, if I have multiple cameras but I want to exclude an area from one of the cameras and not the other how can I do that. Here is my configuration
image_processing:
- platform: tensorflow
scan_interval: 20000
source:
- entity_id: camera.front_door
- entity_id: camera.back_gate
file_out:
- "/home/homeassistant/.homeassistant/www/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "/home/homeassistant/.homeassistant/www/tmp/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
model:
graph: /home/homeassistant/.homeassistant/tensorflow/faster_rcnn_inception_v2_coco_2018_01_28/frozen_inference_graph.pb
area:
# Exclude top 13% of image
top: 0.13
# Exclude left 15% of image
left: 0.30
# Exclude right 15% of image
right: 0.10
I really just want to exclude the area’s from one camera.
Also I was curious about the categories, is there a list somewhere. Other than people, car, truck
edit3:
I’ve been using this for a couple weeks now and have really enjoyed it. For me the most useful for better alerts for my camera. I used to get a good amount of “false” alerts with motion, or “alerts” for things I did not care about. Now, I get alerts for things that I actually want alerts for, and it’s been nearly flawless. I trigger the scan based on motion, but I trigger my alerts based on the results. here are a few sensors I’ve created with some help
objects_in_driveway:
friendly_name: Objects in driveway
value_template: "{{ states('image_processing.tensorflow_front_door') | float >= 1 }}"
entity_id: image_processing.tensorflow_front_door
person_in_driveway:
friendly_name: Person in Drivway
value_template: >
{% set m = state_attr('image_processing.tensorflow_front_door', 'matches') %}
{{ m.person is defined and (m.person[0].score) | float >= 80 }}
entity_id: image_processing.tensorflow_front_door
vehicle_in_driveway:
friendly_name: Vehicle in Driveway
value_template: >
{% set m = state_attr('image_processing.tensorflow_front_door', 'matches') %}
{{ m.truck is defined and (m.truck[0].score) | float >= 80 or m.car is defined and (m.car[0].score) | float >= 80 or m.bike is defined and (m.bike[0].score) | float >= 80 }}
entity_id: image_processing.tensorflow_front_door
OK, I got this going but it keeps saying it sees a person (when there is none)
it shows…
{'person': [{'box': [0.3856241703033447, 0.47053608298301697, 0.9265235662460327, 0.9915212392807007], 'score': 85.14859676361084}]}
any way to know where in the picture it is seeing this, so I know where to limit it? From what I understand this is saying it is 85% sure there is a person in the picture.
Never mind I got the file_out: working and it shows the pic.
Hi Jason - what was your final code for this - look really good !
eg the sensors and automations etc
I dont have much set up… Follow the guide to get a complete setup
image_processing:
- platform: tensorflow
scan_interval: 1
source:
- entity_id: camera.FrontDoorCam
model:
graph: /home/homeassistant/c9workspace/homeassistant/tensorflow/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb
categories:
- category: person
area:
right: 0.85
file_out:
- "/home/homeassistant/.homeassistant/tensorflow/{{ camera_entity.split('.')[1] }}_latest.jpg"
- "/home/homeassistant/.homeassistant/tensorflow/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
and automation to send pic when it see someone.
- alias: "SendDoorBellCameraPic"
initial_state: True
trigger:
platform: state
entity_id: image_processing.tensorflow_frontdoorcam
from: '0'
action:
- service: notify.xxxxxxxxha_bot
data:
title: Send an images
message: "Someone is at the door!"
data:
photo:
- file: '/home/homeassistant/.homeassistant/tensorflow/frontdoorcam_latest.jpg'
So far it works great.
Anyone have any suggestions with which model you are using? I feel like it’s often missing people using ssd_mobilenet_v2_coco. I’ve lowered the confidence to 10% but it often misses people but my motion snapshot is triggering so it is definitely triggering the image processing. Also do I need to reboot after changing the frozen_inference_graph.pb file? Or will it use the new one on the next image scan? Any suggestions would be appreciated
I think a restart of HA but not necessarily a reboot is required
So I will be migrating my hassio setup to a NUC. Will there be some way of running on an NUC? I will have the processing power but seems no good way of running tensorflow with hassio.
Just me, everyone has their own thoughts on the best install. I have an older computer equivalent to a NUC. I’m just running straight up Ubuntu server, no UI, and HA in a Python VENV. Tensorflow works great following the instructions in this thread. Honestly everything works great, you just lose out on the plugin capabilities.
Currently running the latest (0.91.0) version of HA and think I have successfully integrated the TensorFlow component. I have 2 cameras, each with built-in motion that I have mapped to a HA binary_sensor. What I would like to happen is the following:
I must be doing something wrong… would appreciate another set of eyes:
- platform: tensorflow
scan_interval: 10000
source:
- entity_id: camera.driveway
- entity_id: camera.frontdoor
file_out:
- "/opt/homeassistant/config/www/tmp/{{ camera_entity.split('.')[1] }}_latest.jpg"
model:
graph: /opt/homeassistant/config/tensorflow/frozen_inference_graph.pb
categories:
- category: person
- alias: "camera motion on driveway"
trigger:
platform: state
entity_id: binary_sensor.dahua_driveway
to: 'on'
action:
service: image_processing.scan
entity_id: image_processing.tensorflow_driveway
- alias: "camera motion on frontdoor"
trigger:
platform: state
entity_id: binary_sensor.dahua_frontdoor
to: 'on'
action:
service: image_processing.scan
entity_id: image_processing.tensorflow_frontdoor
- alias: "tensorflow driveway"
initial_state: True
trigger:
platform: state
entity_id: image_processing.tensorflow_driveway
from: '0'
action:
service: notify.ios_iPhone
data:
title: "Tensorflow"
message: "driveway"
data:
attachment:
content-type: jpeg
url: "https://MYURL.ui.nabu.casa/local/tmp/driveway_latest.jpg"
- alias: "tensorflow frontdoor"
initial_state: True
trigger:
platform: state
entity_id: image_processing.tensorflow_frontdoor
from: '0'
action:
service: notify.ios_iPhone
data:
title: "Tensorflow"
message: "frontdoor"
data:
attachment:
content-type: jpeg
url: "https://MYURL.ui.nabu.casa/local/tmp/frontdoor_latest.jpg"
You don’t say what problem you have with your configuration.
But in any case, change service: notify.ios_iPhone to service: notify.ios_iphone.
Is there a way to set the threshold for detection? I’d like to set it somewhere between 90-95%. Right now I regularly get notified that a stump or a plant is a person and it’s in the ~85% range.
I use templates to create sensors for the things I want to be notified about and set the threshold there.
Unfortunately, I don’t think that will work for me. I have Tensorflow querying my four cameras directly every five seconds and if something is detected, a new image file is created. Node-RED detects the new image and sends it to my family via Pushbullet.
The problem I’ve run into going the route you’re proposing is keeping the images in sync with the detection. As calls to Tensorflow from Node-RED are asynchronous and Node-RED runs flows in parallel, there’s no way to tie sensor results to specific images.
Trying to leverage motion detection is not a viable option from my cameras as during storms or heavy winds they detect motion continually. FFMPEG motion detection has the same issue and is very expensive from a CPU perspective.
I found it’s actually cheaper from a CPU perspective to just run Tensorflow every 5 seconds and let it make the determination irrespective of motion.
Keeping in sync might be problematic. I was not necessarily saying to use the whole solution, but recommending using a sensor “person detected” and only send a notification if that is true. And you can set the person detection threshold to what you want it to be. I did find tensorflow to be a bit CPU intensive, which is why I ended up only using motion to trigger it to run. But it really depends on your needs.