Google Coral and ESXI 7

importfanatik · July 4, 2021, 6:34am

I have been trying to follow the below article to get the google coral recognized with ESXI 7 on a Dell Optiplex 9020 Micro (USFF) but to no avail.
I can confirm frigate run on ESX 7.0 (ubuntu server 20.04) with google coral and ffmpeg hardware acceleration - frigate (gitmemory.com)

Would anyone have any tips for me?

Thanks in advance

Prodigyplace · July 4, 2021, 12:03pm

Ask on an appropriate forum? Very few, if any users here would be experienced in that mix of software.

importfanatik · July 4, 2021, 5:23pm

The end goal here is to get the frigate home assistant add-on running using the coral for image analysis / object recognition. I know a couple of users on this forum have pulled it off in a virtualized environment. I will try message them directly.

importfanatik · July 8, 2021, 5:56pm

Here are some more details on my struggle.

I have purchased both the m.2 coral and the USB coral
My dell optiplex has two 6 USB 3.0 ports and two M.2 ports

Regarding the M.2 Coral:
One of the two M.2 ports might work, but it is already in use by an NVME drive which I am reluctant to give up. The other M.2 port will not recognize the coral in the BIOS, and can likely only be used for a specific dell wifi card from what I understand.

Regarding the USB Coral:
ESX recognizes the coral, but it shows it as “Global Unichip product 0x089a”, and it doesn’t appear to be detected from the HASS hardware page, or from Frigate.
I have tried passing through the below USB controllers (I think?) but with no success there either.

kentwillumsen · July 8, 2021, 6:09pm

You have to use a USB PCIe card - you cannot use the built-in USB ports.
I used a cheap “ELUTENG USB C PCIE Card” bought on Amazon.

kentwillumsen · July 8, 2021, 6:11pm

Once you have the PCIe USB card you have to activate passthrugh in ESXi:

kentwillumsen · July 8, 2021, 6:12pm

Once activated you can add the PCIe card to your VM:

rpress · July 8, 2021, 6:14pm

I use the Coral M.2 with KVM/QEMU, passing through the PCIe device. This works fine; the normal IOMMU group challenges apply here, I use the ACS override kernel patch to overcome my motherboard limitations.

The Coral USB I have had zero success passing through the USB device itself. I think the Coral driver cannot work with the increased latency. You might have success passing through the whole USB controller PCIe device but this is more complicated and has the same IOMMU group limitations. Anyway the PCIe device is more performant and is cleaner in my rackmount server setup so I sold the Coral USB.

I imagine ESXI has the same IOMMU group requirements as KVM/QEMU, so maybe this is something you’re running up against. I think you’re going to have to dig a lot deeper to figure out why it’s not working for you. My guess is that your motherboard does not have the right IOMMU groupings to passthrough the M.2 device or the USB controller.

You might be able to get this working with KVM/QEMU and the ACS override patch but it’s a long road. If you could add a PCIe USB card (I don’t think your Optiplex has any slots, does it?) you might be able to pass it though to ESXI if your groups work out.

kentwillumsen · July 8, 2021, 6:14pm

Depending on your OS you may have to reserve the memory - but for the HASS pre-built VM no need though:

achurak1 · July 8, 2021, 6:31pm

I use proxmox and run my dockers in ubuntu based lxc containers. It’s trivial to allow these containers direct access to any host devices. With the lxc containers having almost no overhead and direct access to the coral (usb), I easily run 5 cameras with inference speed of 8ms, plus these lxc containers have access to the Intel based gpu acceleration to take some load off of the cpu.

importfanatik · July 9, 2021, 6:43pm

Thanks everyone!

So it looks like my options are:

Abandon ESX and start over without virtualization
Replace my m.2 NVME SSD drive with the m.2 coral, and just live with a SATA SSD for storage.
Replace my m.2 NVME SSD drive with an m.2 to PCIE converter, install a PCIE usb controller and plug the coral into that. And just live with a SATA SSD for storage.
Abandon the Opltiplex 9020 micro and start over with something similar that has PCIE slots.

didi767 · October 28, 2021, 3:36am

dual TPU m.2

I tried all possible variations in the last few days.
to summarize:

couldn’t make ESXi to work with the TPU. it’s not being detected by the HA VM. (I suspect it has to do with the fact that even the ESXi doesn’t recognize the TPU, and just shows “unknown VGA”
on Proxmox, it worked flawlessly, but only 1 TPU is working due to the channels seperate issue.

As much as I love ESXi, I have to switch to Proxmox

importfanatik · January 22, 2022, 6:15am

Thanks. I think I will give this a shot. It sounds like it must be done as a proxmox “LXC” right?
…man…not looking forward to recreating my entire host from scratch…

importfanatik · January 23, 2022, 10:37pm

OK,… I hope Im not speaking too soon,. but I think I have it working. Appears to be getting 10ms inference speed, whereas I normally see 60ms when running off the CPU.
(Google Coral USB, Frigate running inside of Home Assistant (running in docker in the usual way), which is running as a VM inside of proxmox.)

didi767 · January 23, 2022, 11:17pm

Running it on proxmox was never the issue this thread is about making it work on ESXi if anyone was able to.
Glad that you figured it out on proxmox though.

importfanatik · January 23, 2022, 11:39pm

Thanks again for pointing me at PROXMOX. I had no idea that was even an option until you mentioned it.
I have created a new thread here to continue the discussion now that were talking PROXMOX.
If you have a chance could you take a look and confirm if I am on the right track?
Thanks again!

Tiernan23 · December 7, 2022, 2:19pm

If anyone is wondering.
It is possible to get the coral AI working with ESXI.
This process worked with ESX 7… I am yet to try ESX 8

disable vmkusb on the esxi host
esxcli system module set -m=vmkusb -e=FALSE
reboot host
mount the USB to the VM.

if the usb still shows up as globalchip then do the following.

git clone --recurse-submodules GitHub - google-coral/webcoral
cd webcoral
make reset
This will re-flash the Coral USB and it should show up
Bus 001 Device 003: ID 18d1:9302 Google Inc

If this process should fail then share USB Host controller from the host and attach it to the VM.
Follow the process above if it shows up as globalchip.

Kingy444 · February 21, 2023, 6:18am

@Tiernan23 thanks for the above - Tried on esxi8 and after rebooting the option to add USB Device is missing - any clues on where to start troubleshooting this one ?
edit: I can set to TRUE which restores the globalchip issue - but looking for passthrough to Frigate VM

Tiernan23 · February 24, 2023, 8:19am

I have tried ESX 8 and so far have failed to get the coral USB to work. It seems the USB disappears completely if you use the process i specified. I had to revert to ESX 7 to get the USB to work again.

Unfortunately i think for now ESX 7 or ESX 6.7 will need to be used for a Coral USB device.

Kingy444 · February 24, 2023, 8:54am

Thanks for getting back to me.

Annoyingly I had been on ESXI 7 for ages and not found this fix for Coral so went to ProxMox 3 days before you posted the above

I also use iGPU pass through which is super easy on ESXI but just could not get it to work on ProxMox and came back to ESXI late January as a result

Another rebuild I guess