I have updated my original post above with what I did to get it going (PROXMOX->LXC->Docker->Frigate).
I will update it again when I get the HASS add-on and integration going, and the custom card that was suggested.
Perhaps HW related, but I also noticed if I pass through any other usb3 device to this or any other vm, and actually try to saturate the connection (i.e. copy few Gb of files to an usb3 ssd enclosure) it will cause the tpu port to keep resetting with xhci errors in dmesg and tpu errors in frigate log same as when I was running it in vm instead of lxc.
I’m using NUC with xeon e2176m so maybe the small factor makes some sacrifices and the usb3 ports actually share the capacity instead of being able to saturate the ports for multiple devices. Maybe worth mentioning as possible troubleshooting step to try to run the tpu alone 1st.
Updated my post with the final frigate NVR proxy step.
anybody know where I can get my hand on Google Coral in the US?
Hello. I have a problem. Everything works ok, but if I do a reset, Promox assigns me a different bus and won’t connect, what should I do?
You may have already seen this, but if you hit BUY on coral.ai it will show you the official distributors… alas, last I checked, they were all out.
Any other reseller is likely to price gouge you…
I just ran into the same issue today.
You need to go back into your LXC config and adjust the bus number.
From the host console:
- type “lsusb” to see which bus is assigned
- update your conf file (eg. /etc/pve/lxc/103.conf) and change “/dev/bus/usb/002 dev/bus/usb/002” to “/dev/bus/usb/003 dev/bus/usb/003” (or whatever the new bus is).
Does anyone know an easier fix for this?
1 Like
Here is how I have done this in an Unpriviledged container in ProxMox
Create unpriviledged LXC container (Ubuntu in my case), install docker, frigate, etc
Assumptions:
- LXC container uses default user/group mapping
- Container ID is 200
- Coral USB sits within
/dev/bus/usb/003
Create a convenience name for the container’s root group (100000)
proxmox$ groupadd -g 100000 lxc-frigate-root
Add these lines in the LXC config file /etc/pve/lxc/200.conf
usb0: host=1a6e:089a,usb3=1 # coral ID pre-load
usb1: host=18d1:9302,usb3=1 # coral ID post-load
lxc.cgroup2.devices.allow: c 189:* rwm # usb coral
lxc.mount.entry: /dev/bus/usb/003 dev/bus/usb/003 none bind,optional,create=dir
Add this to /etc/udev/rules.d/60-mycoraltpu.rules
SUBSYSTEMS=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="9302", GROUP="lxc-frigate-root"
SUBSYSTEMS=="usb", ATTRS{idVendor}=="1a6e", ATTRS{idProduct}=="089a", GROUP="lxc-frigate-root"
And it works!
Explanation: the udev rule recognises and assigns the Coral USB to group 100000 in Proxmox. Group 100000 is mapped to the Root group of the unpriviledged container. Doing this allows the LXC root group to read/write to the Coral USB on the Proxmox host.
In my system, sometimes the Coral is assigned to bus 002 rather than 003. So I added an additional line in the 200.conf file
# usb0: host=1a6e:089a,usb3=1 # coral ID pre-load (this entry not needed)
# usb1: host=18d1:9302,usb3=1 # coral ID post-load (this entry not needed)
lxc.cgroup2.devices.allow: c 189:* rwm # usb coral
lxc.mount.entry: /dev/bus/usb/003 dev/bus/usb/003 none bind,optional,create=dir
lxc.mount.entry: /dev/bus/usb/002 dev/bus/usb/002 none bind,optional,create=dir
UPDATE: perhaps it may be more ‘correct’ to assign Coral USB to Maybe not…plugdev group, and assign the LXC root user (100000) to be a member of plugdev group. On the other hand I think the Coral USB is not going to be ‘shared’ amongst other VMs/Containers so probably doesn’t matter.
11 Likes
I cannot get this working for the life of me. Current LXC config:
arch: amd64
cores: 3
features: keyctl=1,nesting=1
hostname: docker
memory: 2125
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=6A:97:28:DE:AF:CE,ip=dhcp,type=veth
onboot: 1
ostype: debian
parent: ha20220814
rootfs: local-lvm:vm-101-disk-0,size=8G
swap: 512
unprivileged: 1
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 29:0 rwm
lxc.cgroup2.devices.allow: c 189:* rwm
lxc.apparmor.profile: unconfined
lxc.cgroup2.devices.allow: a
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/bus/usb/001 dev/bus/usb/001 none bind,optional,create=dir 0, 0
lxc.cap.drop:
lxc.mount.auto: cgroup:rw
lsusb on proxmox host
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 8087:0aaa Intel Corp. Bluetooth 9460/9560 Jefferson Peak (JfP)
Bus 001 Device 002: ID 18d1:9302 Google Inc.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
I’m able to successfully run the test model on the proxmox host, but not in the LXC (docker container running. Still getting the following error - any advice on how I might be able to get this resolved? Have tried all the posts I can find, but still nothing is helping. Any help appreciated : )
ValueError: Failed to load delegate from libedgetpu.so.1.0
In my case removing the following line finally made everything work. Must have snuck in from another tutorial:
unprivileged: 1
4 Likes
Has anyone found a way to get around the BUS number changing upon reboot?
Or would using docker on a Proxmox VM rather than LXC and passing through the Coral be a more reliable and persistent way to achieve this?
How do I get the coral USB detected in an LXC container ?
It shows up as Global Unichip Corp. and frigate says it can’t find any EdgeTPU
Did you install the runtime on the host machine?
Once you do that and run the test, I think that’s when it changes to the Google Branded device in the lsusb list. I went through this last night.
I’m passing mine like this:
1 Like
Here’s something to try.
In the Proxmox Shell run (replace 106 with your LXC ID)
Always remember to use due diligence when sourcing scripts and automation tasks from third-party sites.
bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/frigate-support.sh)" -s 106
Reboot the LXC to apply the changes
4 Likes
Found the issue ! I had a trailing slash in the usb mount entry. Thx for the help !
I just installed frigate successfully in a test machine (fairly low powered thin client running a quad core pentium). No Coral yet but I may order a USB one for delivery in a couple of months.
So my PVE CPU usage for a single camera is about 6-8% with some constant motion in the frame (detection is turned off for this test), which is not bad I think given the CPU.
My primary use will be for recording. Before ordering the Coral I wanted to ask would it reduce CPU usage in this instance?
I’m currently looking at doing this to reduce my power consumption and dump my NVR. I have an Optiplex 5070 with i5-9600 and 16GB memory to run it on.
Name of the game here is to get CPU usage and therefore power consumption as low as possible, so I’m going to need to connect both the Coral and likely the iGPU to Frigate.
I’m just wondering, has anyone managed this? I’ve read things elsewhere on the net but haven’t been able to draw a decent conclusion.
I’m currently running Hyper-V with HASSOS and some other stuff virtualised but considering switching to Proxmox to make this happen. One thing I’m not understanding though (I’ve never used Proxmox so may be missing the obvious).
Why not just install Docker on the Proxmox host and run Frigate there?
I got it set up on proxmox on my dell usff box, passing thru the intel onboard graphics (nothing great) and the coral, but it took some doing. I “amost” got it working a couple of times, but the final best instructions were at the top of this thread, except that I didn’t use Portainer. Seems to be running pretty well so far with 7 h264 cameras doing detection on 640x480ish feeds (recording is off the full-res feeds). I personally don’t want to run anything on the proxmox host I absolutely don’t have to – I want that to be “pristine” and easy to recreate if necessary, and rely on the internal vm/ct backups, not the configuration of the host. Feels like you’re violating the whole idea behind the hypervisor/management system to just run it on the host… especially if you do end up with another node for failover (for example), although I suspect most home users don’t run a cluster outside of experimentation.
2 Likes
Thanks for that - I’ve spun up Proxmox on my machine but not done much else yet.
The rig is for my entire home server needs and I’m struggling with how best to set up shared storage. In this instance, say I have a VM running HASSOS and an LXC container with the Frigate/Coral setup. Do you host your storage somewhere else, like in another VM or within the Frigate container? Or set up something like SMB on the HA server and use that?
EDIT - This is a wider confusion for me than just HA/Frigate, I’m actually struggling with Plex, Radarr, Sonarr etc also (Plex container with all the storage? Downloader container with it all?!). Looking for the least power-hungry, most efficient storage options. It was much simpler with the Windows host doing all the sharing!
EDIT2 - Bind mounts for shared storage. Storage has to be on the host though and haven’t worked out how to include that in backups yet.
Thanks.