Getting Coral USB Accelerator working

Installation
5

I am having the same issue. Which is:
-Installed Frigate (full access)
-Configured using the Frigate website
-Success in saying “TPU Found”
-Check the hardware performance and the CPU usage is high. The Coral USB has no usage.
-I configure additional cameras to overload the CPU. Whatever the CPU can’t handle, it is then off loaded to the Coral USB.

Now I am running on an 2008 iMac which is not meant to handle more than 1 stream. I’ve tried 2 cameras or more on raw CPU and it crashes. It shows signs that the Coral USB is working but it not the first choice of Frigate to use. Is there a way to force Frigate to use Coral USB as the primary?

6

that’s not how frigate works. If you configure it to use the coral then it will only use the coral for object detection

I think you are confusing something, the coral does not support usage stats as it only supports one inference at a time.

7

Thanks for the response, it got me thinking to look at it a different angle.

Do you mean that coral only runs when there are objects in the frame?

After looking at the Coral datasheet, the Coral has a light: if it is a solid light it is to let you know that it is powered/initialized, if the light is “pulsing” then it means the TPU is running/activity on it. My light only stays solid and only pulses when there is a subject on the frame. The only thing I can reason here is that the CPU is constantly analyzing the frames and hands off the object recognition process to the Coral USB. Would this be an accurate statement?

8

the CPU is used to run motion detection and regions (portions of the camera frame) are sent to object detection.

you should provide a screenshot of the system page to see which process is using high amounts of CPU

1 Like
9

Thank you for the insight. I think this explains it well and hopefully it serves as a reference point for other users that are wanting to use Coral with Frigate (or the like).

11

Hi. I’m trying frigate for the first time and I ended with trying to know more about coral, because my CPU get’s to +90% usage.
It seems you get that working. Did the CPU usage dropped?

12

Even though I don’t see any recent activity (if 18 days isn’t recent) I wanted to jump in here as I have similar circumstances. I am running Home Assistant on a Windows 10 machine under a VMWare Workstation (17). I got the Coral working by modifying the .vmx file and the logs show basically what Lockie (OP) states. My problem is that occasionally the Coral “gets stuck” and restarts leaving about 30 seconds where it is offline. I moved the coral to a powered USB hub in case that was the problem but it didn’t seem to help. I noticed my inference times were around 300 ms which I thought was okay until I saw that the norm was 10-30 ms. Can anyone confirm that there is no need for additional setup of the Coral with HA? - sorry Nick. Seems like I saw something in a HA forum about different models (or libraries?). Also, there are many forums about using Coral with HA so it isn’t even close to being “plug and play”. Example: 7 months ago on Reddit the OP was directed to the Google Coral getting started page, which begins with the code Lockie mentioned - so we’re back to the original question of how/where to enter that code. I also tried Advanced SSH and web terminal add on. Steps 1 and 2 are to “Add our Debian package repository to your system:” and “Install the Edge TPU runtime:” I am Linux illiterate (obviously) but I can’t be the only one. I’m surprised there isn’t something in HACS to help, but maybe it’s too easy???

13

everything needed is included in Frigate, but there can be setup on the host side when not running docker natively (like the docs recommend). VMs and proxmox can especially cause extra issues and require configuration.

Installing pycoral and other libs is never required as the frigate docker container contains these, users saying this are misinformed

1 Like
14

That makes sense. I might just have a faulty Coral - I was wondering how it was available when I got it back in August. I got it from Allied Electronics for about 80 bucks. I noticed it was down to 65ms earlier and now it’s back to 295ms. Funny because it “got stuck” 3 times within 20 minutes earlier (at 65ms) and I restarted an hour ago with zero “stucks” so far.
Another thing I noticed is it says under "Storage/ recordings and snapshots that I have 31 gig of data - for the life of me I don’t know where. Not in “media/frigate”. You are 100% correct about VMs - they are a pain in the ass. If I didn’t have Blue Iris I would try a different setup. Of course, if I could get Frigate to work with the Coral dependably I could get rid of BI. Still having trouble viewing clips from Frigate via the HA app though. Thought I had it working albeit slowly earlier. I am trying out Nabu Casa in case Cloudflare affected it. Anyway Thanks for the help - I see you answering questions all over the net

15

I’ve been searching forums for more info and I see you come up a lot (example: [Detector Support]: Google Coral USB TPU get stuck · Issue #7427 · blakeblackshear/frigate · GitHub). I also see you and other people recommending trying different USB cables. I am currently on the short one provided with the coral. I have tried other ones, but how do I know if I have a quality cable other than spending more and hoping “you get what you pay for?” The powered hub I have has it’s own (proprietary?) cord between itself and the USB outlet but the problem existed before I bought it.
Current situation is inference at 140ms and Coral was “stuck” only once in the past 15 hours - I could live with that if it were consistent. I restarted HA once about an hour ago so will see. I haven’t moved anything around physically since I started asking questions yesterday but I did restart the VM, so it may be pointing to a problem there still

1 Like
16

Forgot to update, but I switched to a mini PC with HA installed via Ubuntu as suggested in Installation Option for x86-64. Everything is so much easier now - inference is under 10ms and don’t seem to get any instances where the Coral is “stuck” anymore. Thanks as always for your help and input Nick.

1 Like
17

Ah. That’s not the best news. I was hoping I could run one camera on my Pi4, Frigate as an addon with Coral.

18

I don’t think there is a problem with running frigate on a pi4. What makes you think otherwise?

1 Like
19

Thanks for the reply. This.

20

@DaveW1 was not running on a Pi.

21

I was going to say the same thing as Nick. Plenty of people ( I see online) run Frigate with Coral on a pi even with more than 1 camera. In fact, I’m wondering if the OP got his working; maybe they’re like me and forgot to come back to confirm whether they did or if they just gave up. You’re definitely in the right place for help. If/when you get it working on the pi, be sure to come back here and let anybody searching for a similar answer know - I kinda detoured with my experience. At the time I probably thought it was the closest to my experience I could find
Hopefully I’ll remember to bookmark the forum homepage this time LOL

22

Hi there,

i got some TPU problems too…

thats my frigate log

2024-11-22 13:14:29.505598971  [2024-11-22 14:14:25] frigate.app                    INFO    : Camera processor started for BK-Flur: 560
2024-11-22 13:14:29.509265473  [2024-11-22 14:14:25] frigate.app                    INFO    : Camera processor started for BK-WoZi: 563
2024-11-22 13:14:29.509272822  [2024-11-22 14:14:25] frigate.app                    INFO    : Camera processor started for BK-Terasse: 569
2024-11-22 13:14:29.509280623  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for SM-Hof-Nord: 574
2024-11-22 13:14:29.512568407  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for SM-Hof-West-Zelt: 596
2024-11-22 13:14:29.518151584  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for SM-Buero-OG: 605
2024-11-22 13:14:29.519907975  [2024-11-22 14:14:26] frigate.app                    INFO    : Camera processor started for BK-PhyLink: 608
2024-11-22 13:14:29.524688321  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-Flur: 617
2024-11-22 13:14:29.524695113  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-WoZi: 626
2024-11-22 13:14:29.524697793  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-Terasse: 628
2024-11-22 13:14:29.524700517  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for SM-Hof-Nord: 634
2024-11-22 13:14:29.524708034  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for SM-Hof-West-Zelt: 641
2024-11-22 13:14:29.524756813  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for SM-Buero-OG: 647
2024-11-22 13:14:29.524759990  [2024-11-22 14:14:26] frigate.app                    INFO    : Capture process started for BK-PhyLink: 652
2024-11-22 13:14:29.554739261  Traceback (most recent call last):
2024-11-22 13:14:29.554746158    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2024-11-22 13:14:29.554748324      delegate = Delegate(library, options)
2024-11-22 13:14:29.554750690    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2024-11-22 13:14:29.554752831      raise ValueError(capture.message)
2024-11-22 13:14:29.554754618  ValueError
2024-11-22 14:14:37.882538  2024-11-22 13:14:29.554756399
2024-11-22 13:14:29.554758608  During handling of the above exception, another exception occurred:
2024-11-22 14:14:37.882601  2024-11-22 13:14:29.554760173
2024-11-22 13:14:29.554761951  Traceback (most recent call last):
2024-11-22 13:14:29.554763915    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-11-22 13:14:29.554765602      self.run()
2024-11-22 13:14:29.554767847    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-11-22 13:14:29.554769845      self._target(*self._args, **self._kwargs)
2024-11-22 13:14:29.554771980    File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-11-22 13:14:29.554777947      object_detector = LocalObjectDetector(detector_config=detector_config)
2024-11-22 13:14:29.554779973    File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-11-22 13:14:29.554862853      self.detect_api = create_detector(detector_config)
2024-11-22 13:14:29.554865483    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-11-22 13:14:29.554867681      return api(detector_config)
2024-11-22 13:14:29.554869814    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 41, in __init__
2024-11-22 13:14:29.554871814      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2024-11-22 13:14:29.554910002    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2024-11-22 13:14:29.554912465      raise ValueError('Failed to load delegate from {}\n{}'.format(
2024-11-22 13:14:29.554914417  ValueError: Failed to load delegate from libedgetpu.so.1.0
2024-11-22 14:14:37.882622  2024-11-22 13:14:29.554916252

HA VM CLI output of lsusb (with terminal addon)
grafik

and output of lsusb from host

root@ProLiant-Gen8:~# lsusb
Bus 005 Device 003: ID 0424:2660 Standard Microsystems Corp. Hub
Bus 005 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 002: ID 1a6e:089a Global Unichip Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
root@ProLiant-Gen8:~#

and output of VBoxManage list -l usbhost for user vbox

$ VBoxManage list -l usbhost
Host USB Devices:

UUID:               fe40be31-3a6e-4676-b6ba-7a02fdd099b2
VendorId:           0x1a6e (1A6E)
ProductId:          0x089a (089A)
Revision:           1.0 (0100)
Port:               0
USB version/speed:  3/Super
Manufacturer:       Global Unichip Corp.
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1c.6/0000:04:00.0/usb2/2-1//device:/dev/vboxusb/002/002
Current State:      Captured

$

and frigate config:

mqtt:
  enabled: true
  host: 10.101.111.36
  port: 1883
  user: mqtt-user
  password: 9hdFfNWtd9akyss5e8dV334zXdIscsg1t

detectors: # <-------- Add this section to specify the detectors you want to use
  coral: # <---------- Name your detector (you can choose any name)
    type: edgetpu # <-- This specifies that you want to use the Coral USB Accelerator
    device: usb # <-- This tells Frigate to use the USB connected Coral device

timestamp_style:
  # Optional: Position of the timestamp (default: shown below)
  #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
  position: tl
  # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
  #           Additional Examples:
  #             german: "%d.%m.%Y %H:%M:%S"
  #             usa: "%m/%d/%Y %H:%M:%S"
  format: '%d.%m.%Y %H:%M:%S'
  # Optional: Color of font
  #color:
  # All Required when color is specified (default: shown below)
  #  red: 255
  #  green: 255
  #  blue: 255
  # Optional: Line thickness of font (default: shown below)
  #thickness: 2
  # Optional: Effect of lettering (default: shown below)
  #           None (No effect),
  #           "solid" (solid background in inverse color of font)
  #           "shadow" (shadow for font)
  #effect: None

record: # make frigate check directory for deleted files and update database accordingly
  sync_recordings: true

snapshots: #global activation and settings of snapshots
#  required_zones:
#  - motion_area
  enabled: true
  timestamp: true
  bounding_box: true
  retain:
    default: 7

from here on its just stream configs, and the all work fine

idears?

maybe as a headsup: im still on stretch with the host, too much data to just move around and no decent replacement option for the hp micro gen8, and i was lazy :slight_smile:

23

When you See Global Unichip means that its not showing the correct device ID. This is default when you plug in a Coral. Frigate has a driver for this and should automatically work if it has the ability to do a usb reset on the coral device.
If you have gcc/make installed you could try the following process to see if the coral id changes when its flashed.

  1. git clone --recurse-submodules GitHub - google-coral/webcoral
  2. cd webcoral
  3. make reset

This should reflash the coral to hopefully show Google.
Bus 001 Device 003: ID 18d1:9302 Google Inc

If it doesn’t change the usb id then you may have an issue with the way the usb is shared. I had a similar issue with vmware esxi where i had to buy another usb pci card for it to work.

An old thread where we talked about this issue with vmware is here. It may or may not be helpful.

1 Like
24

thank you kindly, ill lokk into this.

i had some serious issues with adding the coral to vbox, turned out vbox user was not part of vboxuser group… got that fixed. then it showed up in the vm…

off topic for history reasons

back when i started with this server i tried proxmox but couldn’t get the host to forward the hwraid controller to the NAS VM due to it using the same interrupt as HPs iLo does. no way to change that in iLo or the hwraid controller, hence i used OMV as the host instead as vm since it brought its on kvm capabilities, but with vbox as kvm environment. and that’s caused a lot if issues over the years, even preventing me from upgrading after stretch/omv-4 as 10/5 did not have kvm support, only 11/6 readded this back due to a boatload of requests and now on 7/12 the gui for kvm is back in omv-extas, but i haven’t spent the time to move data, and start from scratch, then move 18 tb back.

25

@Tiernan23 awesome, your comment sent me down the right track.

In the end I had to install the EDGE TPU Runtime from here

on the host. That made it inicialice on the host. and then show up in the VM

For average statistics. before / after coral:

I run 4 streams from reolink 5mp cams and 2 android wall panels on fire10 gen11 and a noname spy cam from china.

Before coral i had constant notification in frigate of delay in the area of 150-300ms

After getin the coral to work, it only reports back system is healty :slight_smile: and the detector runs at around 32ms.

In HA system hardware metrics, before Coral is saw 60-80% of CPU usage and 2,5 our of 3 gb of RAM used. Now with Coral i see 25-50%CPU and 1,9/3GB RAM

On the host, the load was hovering between 7 and 8 on my 8 core Intel Xeon E3-1280v2 @ 3,6GHz HP-ProLiant Micro Gen8.
On peaks with all streams detecting and analizing at the same time it could lead to other stuff being seroiusly slowed down. That host runs my baikal, pi-hole, 2 Windows VMs and on top of that 2 HA instances, inside the (new and now) main HA i have the frigate and many more dockers running… It gets crowded on that little machine, once in a while, and memory is alway too less.

With Coral i see in top a load of somewhere around 4,5/4,5/3,8

grafik
grafik1920×1032 66.4 KB

please consider its snowing here currently, thats why that detector cpu is jumping a bit. all outside streams do have IR highlighted snowflakes in fron of the cameras. With ZoneMinder that would be continuues recording. Just dumped 12TB of Data from 8 Months of ZM recordings, mostly recording cobwebs in front of cameras at night in IR moving by wind.

All in, Im pretty happy with the money spent on that Coral EDGE TPU!

The setup was a PITA, du to my systems desing, but thats an other storry.

All considerd, frigate and Coral, altho it was at steps a real PITA, in the end it was a breeze and well worth it.

Thanks @crzynik and all who make frigate possible and so incredible.

2 Likes