Local realtime person detection for RTSP cameras

@tokyotexture i successfully spun up Authelia as a SSO/2FA auth service in front of my nginx reverse proxy. It protects HA and frigate (along with others) via 2FA.

It also serves frigate via https so you can view it in the sidebar iframe from outside your network. If you go this route and have questions, let me know.

1 Like

So I want to take the plunge and get the dual TPU E-Key card, but I canā€™t seem to find any adapters to PCIE x4 or x16. Do any of you have a recommendation? If not, any recommendations for M-Key adapters?

Ok so I found my problem.
Camera names canā€™t seem to have spaces and successfully save clips.
@blakeblackshear I donā€™t know if thatā€™s documented.
I see the camera in the doc has an underscore but I donā€™t see a warning about this and it worked for everything else.
Regardless thanks for all of your hard work.
This is awesome.

Hi,

I added a Coral TPU (USB) to my HA blue, but itā€™s not detected by HA and Frigate will not start, if I add CPU as a detector everything works fine, so I know my Frigate is working fine but off course uses too many resources so want to move to a Coral as a detector, any idea what Iā€™m missing here?

detectors:
coral:
type: edgetpu
device: usb

Do I need to add separate drivers for this? If I read the release notes of the OS version, I see this, so seems it should work?

Google Coral PCIe AI Accelerator Support

@ryddler enabled the driver for Google Coral PCIe TPU devices. This allows to use Google Coral Mini PCIe Accelerator or M.2 Accelerator on all boards supporting PCIe.

Thatā€™s all Folks! Now go out and update!

Version information

System Health

version: core-2021.8.2
installation_type: Home Assistant OS
dev: false
hassio: true
docker: true
user: root
virtualenv: false
python_version: 3.9.6
os_name: Linux
os_version: 5.10.53
arch: aarch64
timezone: Europe/Brussels

GitHub API: ok
Github API Calls Remaining: 4196
Installed Version: 1.13.2
Stage: running
Available Repositories: 909
Installed Repositories: 40

logged_in: true
subscription_expiration: 26 augustus 2021 02:00
relayer_connected: true
remote_enabled: true
remote_connected: true
alexa_enabled: false
google_enabled: true
can_reach_cert_server: ok
can_reach_cloud_auth: ok
can_reach_cloud: ok

host_os: Home Assistant OS 6.2
update_channel: stable
supervisor_version: supervisor-2021.06.8
docker_version: 20.10.6
disk_total: 113.9 GB
disk_used: 12.4 GB
healthy: true
supported: true
board: odroid-n2
supervisor_api: ok
version_api: ok
installed_addons: Samba share (9.5.1), Visual Studio Code (3.6.1), deCONZ (6.9.0), Samba Backup (4.5.0), Assistant Relay (0.7.4), ADB - Android Debug Bridge (0.8.0), Tautulli (2.3.0), Node-RED (9.2.1), Mosquitto broker (6.0.1), File editor (5.3.3), MariaDB (2.4.0), Eufy Home Assistant MQTT Bridge (1.21.0), Frigate NVR (1.13)

Thanks in advance!

Proper spacing or proper quotes for post :slight_smile:

Is it detected by OS?
I donā€™t use blue or RasPi so not sure how you may confirm but thatā€™d be good start point

Those that are on proxmox I use a coral on a USB 3 port of the host, use pass through from proxmox absolutely fine. Not so happy on a USB 2 port Iā€™d add maybe thatā€™s the issue?

Iā€™m curious what inference speed are you seeing with the USB Coral @versigo, trying to determine if itā€™s worth it or not. Iā€™m currently averaging 78 with 5 cameras. I have a NUC so USB is my only option without changing hardware. Second question is you donā€™t mind, have you seen overall CPU reduce as a result of the Coral?

Did you managed to access it to docker ? Or still the same issue ?

I see a lot of people going to a lot of trouble to use integrated Corals, often wondered if I was missing something - I just went for 2 x USB 3.0 connected devices, on the same USB 3.0 hub/port, seems ok to me :slight_smile:

Iā€™m using an old Gigabyte Brix, i7, NUC-like unit.

There are 2 aspects to CPU load:

  • The FFmpeg slicing and dicing of images, this runs on the CPU with hardware acceleration if set/available
  • Detecting objects within the sliced/diced frames, this is what the Coral helps with quite considerably.

have you found a solution for this jerky problem? if have the same problem here with an similar setup

@blakeblackshear

Thank you for a cool and useful product!
I want to make automation based on frigate for opening the gate by identifying my or my wifeā€™s cars. I plan to train my own model. But I havenā€™t done it yet.
Can you tell me how to properly train my own model? Will this instruction be suitable as a template (Training Custom Object Detector ā€” TensorFlow 2 Object Detection API tutorial documentation)?
Which of the pre-trained models do you recommend using SD Mobile Net v2 320x320, SD Mobile Net V1 FN 640x640, SD MobileNet V2 FPNLite 320x320, SSD MobileNet V2 FPNLite 640x640?
What size should the marked-up images be?

Iā€™m surprised the Inference speed is so good, thatā€™s not good it means I now have another toy on my wish list :slight_smile: Curious, are you running on bare metal or virtualized?

Dedicated box - supervised install.

ā€¦. so that when I break the storage, proxmox nodesā€¦ I can still I control lights etc :slight_smile:

1 Like

Hi @tmjpugh I donā€™t see it in the hardware section in the supervisor hardware panel, but this seems to be normal?

Here a tread with all the steps I already tried: Coral TPU not detected by HA blue for Frigate - #13 by dmertens

@Eoin
Iā€™m getting 21 - 25 ms
On an old i3 (4th gen I think) with default ffmpeg options.

1 Like

This may be interesting to you:

1 Like

Oops, I made a serious rookie mistake when I installed and started using my Coral Accelerator. I didnā€™t read the details closely enough and this morning I realized that thereā€™s a serious performance to be gained with using USB 3.0. I went down and moved it from USB 2.0 to a USB 3.0.

This is data from one Coral USB while detecting on 6/7 cameras. It was the crack of dawn so the sun hadnā€™t come up yet and there were some cobwebs on one of the cameras. It dropped quickly after the transition because I cleaned off the cobwebs from one of the cameras. The gain in bandwidth is huge!

My interface speed had bottomed out at ~30ms, which resulted in a maximum detection FPS of 30fps. I donā€™t claim to know how the system works but it makes me wonder if the conversion is 1/0.030s = 30fps. Anyway, after taking advantage of USB 3.0 the detection FPS was running at 70+fps for a bit before I cleaned off the cobwebs.

Anyway, I thought Iā€™d post this here just in case anyone didnā€™t realize that they should be using USB 3.0 with a USB Coral Accelerator.

1 Like

If Iā€™m not wrong, the default tflite model currently used by frigate is Googleā€™s SSDLite MobileDet detector (you can find it here: Models - Object Detection | Coral).

I followed this tutorial to retrain that object detection model: Google Colaboratory

1 Like

When ā€œedge tpu not detected ā€œ message is received I expect hardware cause. Tpu need good power source so make sure usb 3.0 and make sure PC has stable power as well

I donā€™t use blue so I not know good troubleshooting for this device but would suspect power then verify tpu appear to OS as expected like /dev/apex0

1 Like

thanks for the advice