Getting Coral USB Accelerator working

[mannebk]

Hi there,

i got some TPU problems too…

thats my frigate log

2024-11-22 13:14:29.505598971  [2024-11-22 14:14:25] frigate.app                    INFO    : Camera processor started for BK-Flur: 560
2024-11-22 13:14:29.509265473  [2024-11-22 14:14:25] frigate.app                    INFO    : Camera processor started for BK-WoZi: 563
2024-11-22 13:14:29.509272822  [2024-11-22 14:14:25] frigate.app                    INFO    : Camera processor started for BK-Terasse: 569
2024-11-22 13:14:29.509280623  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for SM-Hof-Nord: 574
2024-11-22 13:14:29.512568407  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for SM-Hof-West-Zelt: 596
2024-11-22 13:14:29.518151584  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for SM-Buero-OG: 605
2024-11-22 13:14:29.519907975  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for BK-PhyLink: 608
2024-11-22 13:14:29.524688321  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-Flur: 617
2024-11-22 13:14:29.524695113  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-WoZi: 626
2024-11-22 13:14:29.524697793  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-Terasse: 628
2024-11-22 13:14:29.524700517  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for SM-Hof-Nord: 634
2024-11-22 13:14:29.524708034  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for SM-Hof-West-Zelt: 641
2024-11-22 13:14:29.524756813  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for SM-Buero-OG: 647
2024-11-22 13:14:29.524759990  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-PhyLink: 652
2024-11-22 13:14:29.554739261  Traceback (most recent call last):
2024-11-22 13:14:29.554746158    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2024-11-22 13:14:29.554748324      delegate = Delegate(library, options)
2024-11-22 13:14:29.554750690    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2024-11-22 13:14:29.554752831      raise ValueError(capture.message)
2024-11-22 13:14:29.554754618  ValueError
2024-11-22 14:14:37.882538  2024-11-22 13:14:29.554756399
2024-11-22 13:14:29.554758608  During handling of the above exception, another exception occurred:
2024-11-22 14:14:37.882601  2024-11-22 13:14:29.554760173
2024-11-22 13:14:29.554761951  Traceback (most recent call last):
2024-11-22 13:14:29.554763915    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-11-22 13:14:29.554765602      self.run()
2024-11-22 13:14:29.554767847    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-11-22 13:14:29.554769845      self._target(*self._args, **self._kwargs)
2024-11-22 13:14:29.554771980    File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-11-22 13:14:29.554777947      object_detector = LocalObjectDetector(detector_config=detector_config)
2024-11-22 13:14:29.554779973    File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-11-22 13:14:29.554862853      self.detect_api = create_detector(detector_config)
2024-11-22 13:14:29.554865483    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-11-22 13:14:29.554867681      return api(detector_config)
2024-11-22 13:14:29.554869814    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 41, in __init__
2024-11-22 13:14:29.554871814      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2024-11-22 13:14:29.554910002    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2024-11-22 13:14:29.554912465      raise ValueError('Failed to load delegate from {}\n{}'.format(
2024-11-22 13:14:29.554914417  ValueError: Failed to load delegate from libedgetpu.so.1.0
2024-11-22 14:14:37.882622  2024-11-22 13:14:29.554916252

HA VM CLI output of lsusb (with terminal addon)

and output of lsusb from host

root@ProLiant-Gen8:~# lsusb
Bus 005 Device 003: ID 0424:2660 Standard Microsystems Corp. Hub
Bus 005 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 002: ID 1a6e:089a Global Unichip Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
root@ProLiant-Gen8:~#

and output of VBoxManage list -l usbhost for user vbox

$ VBoxManage list -l usbhost
Host USB Devices:

UUID:               fe40be31-3a6e-4676-b6ba-7a02fdd099b2
VendorId:           0x1a6e (1A6E)
ProductId:          0x089a (089A)
Revision:           1.0 (0100)
Port:               0
USB version/speed:  3/Super
Manufacturer:       Global Unichip Corp.
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1c.6/0000:04:00.0/usb2/2-1//device:/dev/vboxusb/002/002
Current State:      Captured

$

and frigate config:

mqtt:
  enabled: true
  host: 10.101.111.36
  port: 1883
  user: mqtt-user
  password: 9hdFfNWtd9akyss5e8dV334zXdIscsg1t

detectors: # <-------- Add this section to specify the detectors you want to use
  coral: # <---------- Name your detector (you can choose any name)
    type: edgetpu # <-- This specifies that you want to use the Coral USB Accelerator
    device: usb # <-- This tells Frigate to use the USB connected Coral device

timestamp_style:
  # Optional: Position of the timestamp (default: shown below)
  #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
  position: tl
  # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
  #           Additional Examples:
  #             german: "%d.%m.%Y %H:%M:%S"
  #             usa: "%m/%d/%Y %H:%M:%S"
  format: '%d.%m.%Y %H:%M:%S'
  # Optional: Color of font
  #color:
  # All Required when color is specified (default: shown below)
  #  red: 255
  #  green: 255
  #  blue: 255
  # Optional: Line thickness of font (default: shown below)
  #thickness: 2
  # Optional: Effect of lettering (default: shown below)
  #           None (No effect),
  #           "solid" (solid background in inverse color of font)
  #           "shadow" (shadow for font)
  #effect: None

record: # make frigate check directory for deleted files and update database accordingly
  sync_recordings: true

snapshots: #global activation and settings of snapshots
#  required_zones:
#  - motion_area
  enabled: true
  timestamp: true
  bounding_box: true
  retain:
    default: 7

from here on its just stream configs, and the all work fine

idears?

maybe as a headsup: im still on stretch with the host, too much data to just move around and no decent replacement option for the hp micro gen8, and i was lazy

[Tiernan23]

When you See Global Unichip means that its not showing the correct device ID. This is default when you plug in a Coral. Frigate has a driver for this and should automatically work if it has the ability to do a usb reset on the coral device.
If you have gcc/make installed you could try the following process to see if the coral id changes when its flashed.

1. git clone --recurse-submodules GitHub - google-coral/webcoral
2. cd webcoral
3. make reset

This should reflash the coral to hopefully show Google.
Bus 001 Device 003: ID 18d1:9302 Google Inc

If it doesn’t change the usb id then you may have an issue with the way the usb is shared. I had a similar issue with vmware esxi where i had to buy another usb pci card for it to work.

An old thread where we talked about this issue with vmware is here. It may or may not be helpful.

Google Coral and ESXI 7

Guys, also started with Frigate, i’m runnin an HP Microserver Gen 8, with Xeox 1265cpu (no quicksync) As you can see in sceenshot, i have already an USB Host controller in passtrough to HassOS, its used for an USB Canable, this controller has 2 ports, so 1 extra free port for maybe a coral Only with 1 camera added to Frigate, and my cpu was acting 10-20% more WIll an google coral help me? is it sufficiant to buy the USB version? I’m new to coral, will it work on my ESX , it still runs 6.7 t…

[mannebk]

thank you kindly, ill lokk into this.

i had some serious issues with adding the coral to vbox, turned out vbox user was not part of vboxuser group… got that fixed. then it showed up in the vm…

off topic for history reasons

back when i started with this server i tried proxmox but couldn’t get the host to forward the hwraid controller to the NAS VM due to it using the same interrupt as HPs iLo does. no way to change that in iLo or the hwraid controller, hence i used OMV as the host instead as vm since it brought its on kvm capabilities, but with vbox as kvm environment. and that’s caused a lot if issues over the years, even preventing me from upgrading after stretch/omv-4 as 10/5 did not have kvm support, only 11/6 readded this back due to a boatload of requests and now on 7/12 the gui for kvm is back in omv-extas, but i haven’t spent the time to move data, and start from scratch, then move 18 tb back.

[mannebk]

@Tiernan23 awesome, your comment sent me down the right track.

In the end I had to install the EDGE TPU Runtime from here

Get started with the USB Accelerator | Coral

on the host. That made it inicialice on the host. and then show up in the VM

For average statistics. before / after coral:

I run 4 streams from reolink 5mp cams and 2 android wall panels on fire10 gen11 and a noname spy cam from china.

Before coral i had constant notification in frigate of delay in the area of 150-300ms

After getin the coral to work, it only reports back system is healty and the detector runs at around 32ms.

In HA system hardware metrics, before Coral is saw 60-80% of CPU usage and 2,5 our of 3 gb of RAM used. Now with Coral i see 25-50%CPU and 1,9/3GB RAM

On the host, the load was hovering between 7 and 8 on my 8 core Intel Xeon E3-1280v2 @ 3,6GHz HP-ProLiant Micro Gen8.
On peaks with all streams detecting and analizing at the same time it could lead to other stuff being seroiusly slowed down. That host runs my baikal, pi-hole, 2 Windows VMs and on top of that 2 HA instances, inside the (new and now) main HA i have the frigate and many more dockers running… It gets crowded on that little machine, once in a while, and memory is alway too less.

With Coral i see in top a load of somewhere around 4,5/4,5/3,8

please consider its snowing here currently, thats why that detector cpu is jumping a bit. all outside streams do have IR highlighted snowflakes in fron of the cameras. With ZoneMinder that would be continuues recording. Just dumped 12TB of Data from 8 Months of ZM recordings, mostly recording cobwebs in front of cameras at night in IR moving by wind.

All in, Im pretty happy with the money spent on that Coral EDGE TPU!

The setup was a PITA, du to my systems desing, but thats an other storry.

All considerd, frigate and Coral, altho it was at steps a real PITA, in the end it was a breeze and well worth it.

Thanks @crzynik and all who make frigate possible and so incredible.