Google Coral and ESXI 7

#1

I have been trying to follow the below article to get the google coral recognized with ESXI 7 on a Dell Optiplex 9020 Micro (USFF) but to no avail.
I can confirm frigate run on ESX 7.0 (ubuntu server 20.04) with google coral and ffmpeg hardware acceleration - frigate (gitmemory.com)

Would anyone have any tips for me?

Thanks in advance

Local realtime person detection for RTSP cameras
#2

Ask on an appropriate forum? Very few, if any users here would be experienced in that mix of software.

#3

The end goal here is to get the frigate home assistant add-on running using the coral for image analysis / object recognition. I know a couple of users on this forum have pulled it off in a virtualized environment. I will try message them directly.

#4

Here are some more details on my struggle.

  1. I have purchased both the m.2 coral and the USB coral
  2. My dell optiplex has two 6 USB 3.0 ports and two M.2 ports

Regarding the M.2 Coral:
One of the two M.2 ports might work, but it is already in use by an NVME drive which I am reluctant to give up. The other M.2 port will not recognize the coral in the BIOS, and can likely only be used for a specific dell wifi card from what I understand.

Regarding the USB Coral:
ESX recognizes the coral, but it shows it as “Global Unichip product 0x089a”, and it doesn’t appear to be detected from the HASS hardware page, or from Frigate.
I have tried passing through the below USB controllers (I think?) but with no success there either.

image
image1439Ă—139 12 KB

image
image731Ă—429 20.2 KB

#5

You have to use a USB PCIe card - you cannot use the built-in USB ports.
I used a cheap “ELUTENG USB C PCIE Card” bought on Amazon.

1 Like
#6

Once you have the PCIe USB card you have to activate passthrugh in ESXi:

image
image1440Ă—193 16.6 KB

#7

Once activated you can add the PCIe card to your VM:

image
image778Ă—244 11.9 KB

#8

I use the Coral M.2 with KVM/QEMU, passing through the PCIe device. This works fine; the normal IOMMU group challenges apply here, I use the ACS override kernel patch to overcome my motherboard limitations.

The Coral USB I have had zero success passing through the USB device itself. I think the Coral driver cannot work with the increased latency. You might have success passing through the whole USB controller PCIe device but this is more complicated and has the same IOMMU group limitations. Anyway the PCIe device is more performant and is cleaner in my rackmount server setup so I sold the Coral USB.

I imagine ESXI has the same IOMMU group requirements as KVM/QEMU, so maybe this is something you’re running up against. I think you’re going to have to dig a lot deeper to figure out why it’s not working for you. My guess is that your motherboard does not have the right IOMMU groupings to passthrough the M.2 device or the USB controller.

You might be able to get this working with KVM/QEMU and the ACS override patch but it’s a long road. If you could add a PCIe USB card (I don’t think your Optiplex has any slots, does it?) you might be able to pass it though to ESXI if your groups work out.

#9

Depending on your OS you may have to reserve the memory - but for the HASS pre-built VM no need though:
image

#10

I use proxmox and run my dockers in ubuntu based lxc containers. It’s trivial to allow these containers direct access to any host devices. With the lxc containers having almost no overhead and direct access to the coral (usb), I easily run 5 cameras with inference speed of 8ms, plus these lxc containers have access to the Intel based gpu acceleration to take some load off of the cpu.

image
image1263Ă—585 22 KB

1 Like
#11

Thanks everyone!

So it looks like my options are:

  1. Abandon ESX and start over without virtualization
  2. Replace my m.2 NVME SSD drive with the m.2 coral, and just live with a SATA SSD for storage.
  3. Replace my m.2 NVME SSD drive with an m.2 to PCIE converter, install a PCIE usb controller and plug the coral into that. And just live with a SATA SSD for storage.
  4. Abandon the Opltiplex 9020 micro and start over with something similar that has PCIE slots.
#12

dual TPU m.2

I tried all possible variations in the last few days.
to summarize:

  1. couldn’t make ESXi to work with the TPU. it’s not being detected by the HA VM. (I suspect it has to do with the fact that even the ESXi doesn’t recognize the TPU, and just shows “unknown VGA”
  2. on Proxmox, it worked flawlessly, but only 1 TPU is working due to the channels seperate issue.

As much as I love ESXi, I have to switch to Proxmox :confused:

2 Likes
#13

Thanks. I think I will give this a shot. It sounds like it must be done as a proxmox “LXC” right?
…man…not looking forward to recreating my entire host from scratch… :grimacing:

#14

OK,… I hope Im not speaking too soon,. but I think I have it working. Appears to be getting 10ms inference speed, whereas I normally see 60ms when running off the CPU.
(Google Coral USB, Frigate running inside of Home Assistant (running in docker in the usual way), which is running as a VM inside of proxmox.)

#15

Running it on proxmox was never the issue :slight_smile: this thread is about making it work on ESXi if anyone was able to.
Glad that you figured it out on proxmox though.

Google Coral USB + Frigate + PROXMOX
#16

Thanks again for pointing me at PROXMOX. I had no idea that was even an option until you mentioned it.
I have created a new thread here to continue the discussion now that were talking PROXMOX.
If you have a chance could you take a look and confirm if I am on the right track?
Thanks again!

#17

If anyone is wondering.
It is possible to get the coral AI working with ESXI.
This process worked with ESX 7… I am yet to try ESX 8

  1. disable vmkusb on the esxi host
    esxcli system module set -m=vmkusb -e=FALSE
  2. reboot host
  3. mount the USB to the VM.

if the usb still shows up as globalchip then do the following.

  1. git clone --recurse-submodules GitHub - google-coral/webcoral
  2. cd webcoral
  3. make reset
    This will re-flash the Coral USB and it should show up
    Bus 001 Device 003: ID 18d1:9302 Google Inc

If this process should fail then share USB Host controller from the host and attach it to the VM.
Follow the process above if it shows up as globalchip.