Google Coral USB + Frigate + PROXMOX

Third party integrations
22

Here is how I have done this in an Unpriviledged container in ProxMox

Create unpriviledged LXC container (Ubuntu in my case), install docker, frigate, etc

Assumptions:

  1. LXC container uses default user/group mapping
  2. Container ID is 200
  3. Coral USB sits within /dev/bus/usb/003

Create a convenience name for the container’s root group (100000)

proxmox$ groupadd -g 100000 lxc-frigate-root

Add these lines in the LXC config file /etc/pve/lxc/200.conf

usb0: host=1a6e:089a,usb3=1 # coral ID pre-load
usb1: host=18d1:9302,usb3=1 # coral ID post-load
lxc.cgroup2.devices.allow: c 189:* rwm # usb coral
lxc.mount.entry: /dev/bus/usb/003 dev/bus/usb/003 none bind,optional,create=dir

Add this to /etc/udev/rules.d/60-mycoraltpu.rules

SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="9302", GROUP="lxc-frigate-root"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="1a6e", ATTRS{idProduct}=="089a", GROUP="lxc-frigate-root"

And it works!

Explanation: the udev rule recognises and assigns the Coral USB to group 100000 in Proxmox. Group 100000 is mapped to the Root group of the unpriviledged container. Doing this allows the LXC root group to read/write to the Coral USB on the Proxmox host.

In my system, sometimes the Coral is assigned to bus 002 rather than 003. So I added an additional line in the 200.conf file

# usb0: host=1a6e:089a,usb3=1 # coral ID pre-load (this entry not needed)
# usb1: host=18d1:9302,usb3=1 # coral ID post-load (this entry not needed)
lxc.cgroup2.devices.allow: c 189:* rwm # usb coral
lxc.mount.entry: /dev/bus/usb/003 dev/bus/usb/003 none bind,optional,create=dir
lxc.mount.entry: /dev/bus/usb/002 dev/bus/usb/002 none bind,optional,create=dir

UPDATE: perhaps it may be more ‘correct’ to assign Coral USB to plugdev group, and assign the LXC root user (100000) to be a member of plugdev group. On the other hand I think the Coral USB is not going to be ‘shared’ amongst other VMs/Containers so probably doesn’t matter. Maybe not…

11 Likes
23

I cannot get this working for the life of me. Current LXC config:

arch: amd64
cores: 3
features: keyctl=1,nesting=1
hostname: docker
memory: 2125
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=6A:97:28:DE:AF:CE,ip=dhcp,type=veth
onboot: 1
ostype: debian
parent: ha20220814
rootfs: local-lvm:vm-101-disk-0,size=8G
swap: 512
unprivileged: 1
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.apparmor.profile: unconfined
lxc.cgroup2.devices.allow: a
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/bus/usb/001 dev/bus/usb/001 none bind,optional,create=dir 0, 0
lxc.cap.drop:
lxc.mount.auto: cgroup:rw

lsusb on proxmox host

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560 Jefferson Peak (JfP)
Bus 001 Device 002: ID 18d1:9302 Google Inc. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

I’m able to successfully run the test model on the proxmox host, but not in the LXC (docker container running. Still getting the following error - any advice on how I might be able to get this resolved? Have tried all the posts I can find, but still nothing is helping. Any help appreciated : )

ValueError: Failed to load delegate from libedgetpu.so.1.0
24

In my case removing the following line finally made everything work. Must have snuck in from another tutorial:

unprivileged: 1
4 Likes
25

Has anyone found a way to get around the BUS number changing upon reboot?

Or would using docker on a Proxmox VM rather than LXC and passing through the Coral be a more reliable and persistent way to achieve this?

27

How do I get the coral USB detected in an LXC container ?
It shows up as Global Unichip Corp. and frigate says it can’t find any EdgeTPU

28

Did you install the runtime on the host machine?

Once you do that and run the test, I think that’s when it changes to the Google Branded device in the lsusb list. I went through this last night.

I’m passing mine like this:
image

1 Like
29

Here’s something to try.

In the Proxmox Shell run (replace 106 with your LXC ID)
Always remember to use due diligence when sourcing scripts and automation tasks from third-party sites.

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/frigate-support.sh)" -s 106

Reboot the LXC to apply the changes

5 Likes
30

Found the issue ! I had a trailing slash in the usb mount entry. Thx for the help !

31

I just installed frigate successfully in a test machine (fairly low powered thin client running a quad core pentium). No Coral yet but I may order a USB one for delivery in a couple of months.

So my PVE CPU usage for a single camera is about 6-8% with some constant motion in the frame (detection is turned off for this test), which is not bad I think given the CPU.

My primary use will be for recording. Before ordering the Coral I wanted to ask would it reduce CPU usage in this instance?

32

I’m currently looking at doing this to reduce my power consumption and dump my NVR. I have an Optiplex 5070 with i5-9600 and 16GB memory to run it on.

Name of the game here is to get CPU usage and therefore power consumption as low as possible, so I’m going to need to connect both the Coral and likely the iGPU to Frigate.

I’m just wondering, has anyone managed this? I’ve read things elsewhere on the net but haven’t been able to draw a decent conclusion.

I’m currently running Hyper-V with HASSOS and some other stuff virtualised but considering switching to Proxmox to make this happen. One thing I’m not understanding though (I’ve never used Proxmox so may be missing the obvious).

Why not just install Docker on the Proxmox host and run Frigate there?

33

I got it set up on proxmox on my dell usff box, passing thru the intel onboard graphics (nothing great) and the coral, but it took some doing. I “amost” got it working a couple of times, but the final best instructions were at the top of this thread, except that I didn’t use Portainer. Seems to be running pretty well so far with 7 h264 cameras doing detection on 640x480ish feeds (recording is off the full-res feeds). I personally don’t want to run anything on the proxmox host I absolutely don’t have to – I want that to be “pristine” and easy to recreate if necessary, and rely on the internal vm/ct backups, not the configuration of the host. Feels like you’re violating the whole idea behind the hypervisor/management system to just run it on the host… especially if you do end up with another node for failover (for example), although I suspect most home users don’t run a cluster outside of experimentation.

2 Likes
34

Thanks for that - I’ve spun up Proxmox on my machine but not done much else yet.

The rig is for my entire home server needs and I’m struggling with how best to set up shared storage. In this instance, say I have a VM running HASSOS and an LXC container with the Frigate/Coral setup. Do you host your storage somewhere else, like in another VM or within the Frigate container? Or set up something like SMB on the HA server and use that?

EDIT - This is a wider confusion for me than just HA/Frigate, I’m actually struggling with Plex, Radarr, Sonarr etc also (Plex container with all the storage? Downloader container with it all?!). Looking for the least power-hungry, most efficient storage options. It was much simpler with the Windows host doing all the sharing!

EDIT2 - Bind mounts for shared storage. Storage has to be on the host though and haven’t worked out how to include that in backups yet.

Thanks.

35

Hello To all

I’m try to install the Coral TPU in a NIC running proxmox but the device still not appear in the list …

I already try to change the USB ports and the problem persist

Coral_proxmox

How can I solve this ?

Thank you in advance for your help

36

I had issues passing through the usb device id etc. The usb port was better but not perfect.
Best thing to do is to pass through the pci device as a whole.

37

I try also, but the Frigate still not find the device

But the mine is a USB and not a PCI

And in the proxmox the device still appear as unknown …Coral_proxmox1

38

Yes mine is usb too
in proxmox i selected this pci device

Screenshot 2022-09-09 164455
Screenshot 2022-09-09 1644551620×618 172 KB

Which passes thorough the whole usb controller

39

I try with a PCI as you described but i still having a problem once the frigate still not detecting the device

I already try to change the frigate.yml file as PCI and also as USB but the problem persist

Coral_proxmox3
Coral_proxmox3987×81 4.34 KB

Coral_proxmox2

Thank you

40

My Docker LXC (running frigate and portainer) seems to use approx 1.7% of my 4 VCPUs on average.

42

I tried switching back to CPU. My inference speed went from 10 to 50, but my CPU seems to be about the same (1.75% of 4 VCPU).

43

Thanks, my machine has an intel 605 GPU, but with 2 cameras its like 9% CPU on a proxmox LXC container with detect turned off. So far I’ve been unable to get Frigate to use hardware acceleration even though the GPU is being passed through to docker (as far as I can tell). I’ll probably scrub this and try a bare docker install when I get the chance.