Tensorflow and official docker image - CPU instruction issue

Configuration
1

I’ve followed the instructions on setting up the new tensorflow component but have run into an issue. I compiled the models exactly as the instructions (on the host machine) and copied all the files into the mounted config folder. but unfortunately keep getting the following error which stops HA starting.

The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.

Could this be to do with the models being compiled outside the docker image, even though its the same architecture?

2 Likes
2

Wouldn’t it mean the cpu/board doesn’t support AVX?

3

Yes it does.

I’ve realised it’s the tensorflow build not the models.

I’d need to compile tensorflow from source to make it work, Not easy within the ha docker build I don’t think?

4

I’m getting the same sort of error:
The TensorFlow library was compiled to use SSE4.1 instructions, but these aren’t available on your machine.
Except for SSE4.1
I’m using a Proxmox Hypervisor with a Intel Core i3 processor(with HT).

5

So i’ve build the Docker image on my host machine, and I still have the same error.
I even switched to the Dev branch and rebuild the image, but still the same error.
If I do a docker exec, and install tensorflow i don’t get the error. But Hass won’t start.

6

I’ve created a Github issue: https://github.com/home-assistant/home-assistant/issues/18435

7

Building the docker image wont help. You need to build tensorflow itself (can be done with a docker image).

Then substitute the home assistant dockers pip package with your own build ( I’m not sure how to do this yet)

Thanks

8

So I got a little further I think: Using one of the non AVX community wheels here https://github.com/yaroslavvb/tensorflow-community-wheels/issues dropped into the config folder I bashed into the ha docker image and ran

pip install /config/{wheelname.whl}

And restarted home assistant (not docker) with the tensorflow config in, home assistant starts now. But in the log I get the following error :

No TensorFlow Object Detection library found! Install or compile for your system following instructions here: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md

pip worked, uninstalled the old tensorflow and added the new one from the file so I’m not sure where to go from here.

9

I messed with this yesterday and it appears to be working, if a bit slow. I use a Synology NAS, which also fails on the default install due to the lack of AVX instruction sets.

Generate a custom wheel using the CPU build.sh method here - https://github.com/hadim/docker-tensorflow-builder

Install the wheel manually within the docker container

pip install tensorflow-1.8.0-cp27-cp27mu-linux_x86_64.whl

Add to HA config

image_processing:
- platform: tensorflow
source:
  - entity_id: camera.motion_front
  - entity_id: camera.motion_frontdoor
  - entity_id: camera.motion_laundry
  - entity_id: camera.motion_laundryback
  - entity_id: camera.motion_backyard
  - entity_id: camera.motion_backdeck
model:
  graph: /config/tensorflow/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb
  categories:
    - person
file_out:
  - "/storagemotion/tensorflow/{{ camera_entity.split('.')[1] }}_latest.jpg"
  - "/storagemotion/tensorflow/{{ camera_entity.split('.')[1] }}_{{ now().strftime('%Y%m%d_%H%M%S') }}.jpg"
scan_interval: 1000000

Then call the service -

image_processing.tensorflow_motion_frontdoor (for example)

It took about 45 seconds to generate though;

2018-11-13 23:16:38 WARNING (MainThread) [homeassistant.helpers.entity] Update of image_processing.tensorflow_motion_frontdoor is taking over 10 seconds
2018-11-13 23:17:13 INFO (SyncWorker_14) [homeassistant.components.image_processing.tensorflow] Saving results image to /storagemotion/tensorflow/motion_frontdoor_latest.jpg
2018-11-13 23:17:13 INFO (SyncWorker_14) [homeassistant.components.image_processing.tensorflow] Saving results image to /storagemotion/tensorflow/motion_frontdoor_20181114_091713.jpg

Here is the result (red box is from Motion, yellow box plus person label is from tensorflow)

motion_frontdoor_20181114_091713

Logs show stuff taking awhile at startup;

2018-11-13 23:15:04 WARNING (SyncWorker_18) [homeassistant.components.image_processing.tensorflow] No OpenCV library found. TensorFlow will process image with PIL at reduced resolution.
2018-11-13 23:15:11 WARNING (MainThread) [homeassistant.components.image_processing] Setup of platform tensorflow is taking over 10 seconds.

So i’m pleased with the result but need a big performance boost before it would be usable

2 Likes
10

Wondering if the latest issue is more the moels than tenosr actually.

@steve1 did you follow this gist?

https://gist.github.com/hunterjm/6f9332f92b60c3d5e448ad936d7353c3

Thanks

Carl

11

45 seconds?! Gulp, I was wondering what the frame rate would be, but that’s surprising. If the idea is to use a pi, then I would suggest offloading to a compute stick and running the models through the VPU

12

@cwjackson yep, i followed that gist and then chose a model that seemed faster on low powered hardware (ssd_mobilenet_v2_coco)

I’d be very interested to see what sort of speeds other people are getting. My NAS is pretty busy though, might try today with CPU hungry tasks stopped and see if speed improves.

13

I’ve installed OpenCV within the container as well, to see if that improves things. Speed was about the same but the error referencing opencv no longer occurs.

With my Motion container stop (which uses pretty high CPU), tensorflow catagorises People only correctly in 3-5 seconds - much better!

With the catagories section commented out, i’m averaging 7-9 seconds.

If anyone wants a tensorflow wheel compiled for a synology 918+ let me know and i can save you the compile time.

14

Really cool. What cameras are you using? Just to get the quality of input for tensorflow.

15

These examples are from oldish hikvisions, with tensorflow analysing the low quality version of the images (640x480 i think). I’d imagine the accuracy would be increased if the source was better quality but it does seem to work well with low quality images.

I’ve just tried this config change and we picked up a car nicely with very high accuracy

    model:
  graph: /config/tensorflow/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb
  categories:
    - person
    - car
    - cat
    - dog
    - bird

motion_front_20181115_105538

Pretty pleased, some clever people out there building these integrations!

@M203 Send me a photo if you like and i can see what tensorflow would pick up from it, see if your source would be good enough?

16

I’ve not been able to get this working reliably though and its definitely not stable.

@M203 i tried processing those test images you sent me and HA crashes immediately. I then switched back to my previous images and the docker container crashes immediately after i try to process an image.

I put HA into debug mode but cannot generate a crash log or anything like that, this is the last line in the logs.

2018-11-16 09:16:44 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=image_processing, service=scan, service_data=entity_id=image_processing.tensorflow_motion_front, service_call_id=e0b992c7222c4c19a25b9e7e0adc6c6b>
2018-11-16 09:16:54 WARNING (MainThread) [homeassistant.helpers.entity] Update of image_processing.tensorflow_motion_front is taking over 10 seconds
1 Like
17

Thanks for trying! Hopefully it will get more robust with coming releases. It is a really useful component!

18

How did you get the bounding box to show? Can you post the configuration please ?

19

the config i used is above

20

Thank you got everything working now, What program are you using for detecting motion from camera?
I am using Zoneminder but it is a CPU hog.