In my testing dlib was OK. Alternatively checkout facebox or deepstack face
I think dLib would work ok but I can see some significant delay with maybe a high resolution stream and some occasional false positive when the stream gets dropped and restarted. I am looking for something faster but obviously with additional hardware support. For Deepstack, I guess I will need to install it in a docker with and add an NCS2. Will pursue this for now. Thanks!
turns out it wasn’t Dlib which was slow… it was me…
I think the default homassistant 10s image processing scan interval was causing delays and I just had to turn it up a bit to 2s. I would still like to be able to offload this task from the cpu though which may not be possible with Dlib alone so I may still check deepstack.
1 Like
very useful guide. btw. I struggle to install mine on a raspberry pi docker container. so I spent a week tried to build tensorflow 1.13.2 from source. you can download the pre-built wheel file to install on home assistant on a raspberry pi here https://github.com/chaisaeng/Tensorflow-1.13.2-alpine-armv7
1 Like
@chaisaeng Nice one! Can you advise if tflite can be built also? I am maintaining a project which I hope to deply as a hassio addon here
I think you can built tflite for alpine on raspberry pi as well as google provide the source code and also build tool chain (bazel). all you need to do is clone the repository then configure the tool chain so that necessary dependencies are met and start build with bazel build //
1 Like
@ptdalen, I am experiencing the exact same problems you were. I have 3 cameras going, and they all show up in HA. However, when I used a photo of a box on my doormat, it doesn’t feel like that (a) tensorflow is processing; (b) when I do an image_processing.scan, it feels like it doesn’t scan. How did you resolve your problems?
Sorry missed this and forgot to come back. Have to resolved your issue. My issue was actually that there was no issue. When I walked out in front of my camera, and kicked off a scan, it reported me as a person. IN my case my testing camera, had nothing to report.
Hi everyone
I have a hikvision coax cameras hook to NVR and then to home assistant.
I currently have line detection set up, yet I would like to make them smarter.
Could I set up tensorflow and have it linked to my cameras on my RPI 4 (which runs HA).
Hey guys,
since a couple of days i’m receiving this error:
Platform error: image_processing - Requirements for tensorflow not found: [‘tensorflow==1.13.2’].
Tensorflow is not working now, i did not have the entity.
What’s wrong?
Version 0.113 updated the python version which was not compatible with tensorflow. This should be fixed as of 0.114. You will have to manually update your tensorflow config.
I’ve upgraded to tensorflow 2 and .114, but unfortunately now it’s not working anymore for me. I end up with this in my logs
2020-08-14 16:54:16 ERROR (MainThread) [homeassistant] Error doing job: Future exception was never retrieved
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/tensorflow/image_processing.py", line 156, in tensorflow_hass_start
model_config=pipeline_configs["model"], is_training=False
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/builders/model_builder.py", line 1062, in build
add_summaries)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/builders/model_builder.py", line 375, in _build_ssd_model
is_training=is_training)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/builders/model_builder.py", line 350, in _build_ssd_feature_extractor
return feature_extractor_class(**kwargs)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 312, in __init__
name=name)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 163, in __init__
overrides={'rescale_input': False})
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 494, in from_name
model = cls(config=config, overrides=overrides)
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 452, in __init__
output = efficientnet(image_input, self.config)
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 388, in efficientnet
x = mb_conv_block(x, block, config, block_prefix)
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 263, in mb_conv_block
name=prefix + 'depthwise')
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 200, in conv2d_block
name=name + '_bn')(x)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 960, in __call__
self._set_inputs(cast_inputs, outputs)
File "/usr/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 418, in inner_cm
for fn in self._scope_exit_callbacks:
TypeError: 'NoneType' object is not iterable
I’ve wiped and tried a couple of times. I was looking through the PR and did see a mention of venv, which is what I am running. Anyone have this working, running in a python VENV? Any tips, suggestions? I tried installing as root, as my HA user, in the venv,not in the venv? No luck.
Thanks in advance
From the 114 release notes: TensorFlow 1 object detection models are not compatible with TensorFlow 2, therefore you’ll will need to walk through the setup documentation again and choose a new model from the Model Zoo.
I did download new models. I used the link and the setup instructions from the documentation. I even deleted all files and folders in the tensorflow folder just to be safe
I just redid this morning just to be safe. Here are my steps running in a VENV
Change to my HA user, and then activate the VENV
Change to my home assistant folder
rm -r tensorflow (start fresh)
git clone https://github.com/hunterjm/hass-tensorflow.git tensorflow
cd into the models directory
curl -OL http://download.tensorflow.org/models/object_detection/tf2/20200
711/efficientdet_d2_coco17_tpu-32.tar.gz
tar -xzvf efficientdet_d2_coco17_tpu-32.tar.gz
Then updated my impage processing lines to use the new model
- platform: tensorflow
scan_interval: 20000
source:
- entity_id: camera.front_door
name: front door
file_out:
- "/home/homeassistant/.homeassistant/www/tmp/front_door_latest.jpg"
model:
graph: /home/homeassistant/.homeassistant/tensorflow/models/efficientdet_d2_coco17_tpu-32/
I updated to 114.1 as well. This is what I see in the logs
2020-08-15 09:32:57 ERROR (MainThread) [homeassistant] Error doing job: Future exception was never retrieved
Traceback (most recent call last):
File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/tensorflow/image_processing.py", line 156, in tensorflow_hass_start
model_config=pipeline_configs["model"], is_training=False
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/builders/model_builder.py", line 1016, in build
add_summaries)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/builders/model_builder.py", line 375, in _build_ssd_model
is_training=is_training)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/builders/model_builder.py", line 350, in _build_ssd_feature_extractor
return feature_extractor_class(**kwargs)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 485, in __init__
name=name)
File "/home/homeassistant/.homeassistant/tensorflow/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 163, in __init__
overrides={'rescale_input': False})
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 494, in from_name
model = cls(config=config, overrides=overrides)
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 452, in __init__
output = efficientnet(image_input, self.config)
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 388, in efficientnet
x = mb_conv_block(x, block, config, block_prefix)
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 263, in mb_conv_block
name=prefix + 'depthwise')
File "/srv/homeassistant/lib/python3.7/site-packages/official/vision/image_classification/efficientnet/efficientnet_model.py", line 200, in conv2d_block
name=name + '_bn')(x)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 922, in __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/layers/normalization.py", line 741, in call
outputs = self._fused_batch_norm(inputs, training=training)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/layers/normalization.py", line 632, in _fused_batch_norm
self.add_update(mean_update)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1459, in add_update
updates = [process_update(x) for x in updates]
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1459, in <listcomp>
updates = [process_update(x) for x in updates]
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1455, in process_update
reachable = tf_utils.get_reachable_from_inputs(relevant_inputs, [update])
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 138, in get_reachable_from_inputs
outputs = x.consumers()
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 676, in consumers
for name in consumer_names
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 676, in <listcomp>
for name in consumer_names
File "/srv/homeassistant/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3663, in _get_operation_by_name_unsafe
return self._nodes_by_name[name]
KeyError: 'stack_0/block_0/se_reshape/Shape'
EDIT/Update:
It might actually be a camera issue. I have two cameras set up for image processing. One is OVNIF, and the other is a rstp stream. I was able to get successful processing on the rstp stream. I’ll need to to more testing to see what may be going on with the OVNIF camera. So… I’ll say it’s working for me now.
EDIT2: Both camera’s work. I suspect that the OVNIF camera was not ready yet when tensorflow was starting up, which caused the errors. I will say that I was not getting errors in my log on startup previously, but with that said, I can live with the errors since they only appear for me once after restarting HA.
Now my question is… What are the improvments with tensorflow 2? Anyone know? Any new things I can do, faster processsing?
Getting these errors as well with new tensorflow. No clue as to how to fix it.
I have not seen many posts on this. Wonder if its because no one else if having issues, or if this component is not used too much. EIther way, I’ve not a full reinstall of HA on a test machine, and just installed tensorflow for the first time, and am still having the same issues. No detection on camera with obvious items, cars, people, etc.
Are others working just fine? Anyone ?
Possibly related to fact that tensorflow 1 models are not compatible with tensorflow 2.
This integration requires quite deep appreciation for the details
Are you suggesting that maybe the tensorflow 2 models are not as good yet at object detection? I could see that as a possibility.
I find myself (and others, see this thread) that tensorflow v2 is not as good as the V1.
In fact it’s so bad that I’ve stayed on 0.112 on the machine that does primarily TF and video analysis…
It would be interesting to see if others are having issues with the TF V2 model or if it works for them…