Local realtime person detection for RTSP cameras

@tokyotexture I could be interested in your extra Coral. We should touch base directly.

Glad to hear it can work in a VM, will have to figure out how to pass it on to the VM.

Should be pretty much the same as passing through a GPU, so I would start there. A quick google yields XCP-ng 7.5.0 PCI pass through | TrueNAS Community . Seems straightforward enough.

Looks pretty straight forward to me. Just need to get a Coral and an adapter board and give it a go. Did you need to install any drivers in the VM?

Which model did you get by mistake? Any recommendations on adapter boards?

Be carefull,i thought it will be easy,vut i had a problem with iommu groups. Coral was bound with other essentional periferies like ssd or network int adapter. So the solution was to add additional pci card for host OS ssd,then the iommu group was free for pci coral (with adapter that nobody was saying about xD that the minipci is not compatible with the small pci on MB xD)

Hi,
I use Frigate to record the clips.
I tried to add this code

output_args: 
  clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

to record the audio as well, but unfortunately I can no longer view the cam, but only a green screen.
What is the right procedure for recording clips with audio?

@tokyotexture I am going through the same process moving Frigate from HAOS addon to a separate docker container. I just found out that the HA frigate integration also pulls the frigate clips into the media browser. Instead of pointing the integration at the addon, i just pointed it at the ip/port of the frigate docker container.

I’m running HAOS in VM on unraid and frigate in a separate docker container, fyi.

EDIT: next step will be setting up Authelia w/ 2FA to access frigate via nginx proxy manager from outside my network.

Right you are! I just realized I was setting things up and only had snapshots so far when I tested the media browser…no wonder everything was empty! d’oh :slight_smile:

Everything working great in a separate container so far, and nice to be able to manage things outside of HAOS.

I’m in the same boat in the sense of having to set up a reverse proxy with some sort of authentication. I was thinking traefik. Authelia looks interesting, but hoping to keep things as simple as possible without being terribly insecure. Maybe Traefik + google authentication + duckdns + let’s encrypt…sometimes I really do wish things were a bit more turn key…

@tokyotexture i successfully spun up Authelia as a SSO/2FA auth service in front of my nginx reverse proxy. It protects HA and frigate (along with others) via 2FA.

It also serves frigate via https so you can view it in the sidebar iframe from outside your network. If you go this route and have questions, let me know.

1 Like

So I want to take the plunge and get the dual TPU E-Key card, but I can’t seem to find any adapters to PCIE x4 or x16. Do any of you have a recommendation? If not, any recommendations for M-Key adapters?

Ok so I found my problem.
Camera names can’t seem to have spaces and successfully save clips.
@blakeblackshear I don’t know if that’s documented.
I see the camera in the doc has an underscore but I don’t see a warning about this and it worked for everything else.
Regardless thanks for all of your hard work.
This is awesome.

Hi,

I added a Coral TPU (USB) to my HA blue, but it’s not detected by HA and Frigate will not start, if I add CPU as a detector everything works fine, so I know my Frigate is working fine but off course uses too many resources so want to move to a Coral as a detector, any idea what I’m missing here?

detectors:
coral:
type: edgetpu
device: usb

Do I need to add separate drivers for this? If I read the release notes of the OS version, I see this, so seems it should work?

Google Coral PCIe AI Accelerator Support

@ryddler enabled the driver for Google Coral PCIe TPU devices. This allows to use Google Coral Mini PCIe Accelerator or M.2 Accelerator on all boards supporting PCIe.

That’s all Folks! Now go out and update!

Version information

System Health

version: core-2021.8.2
installation_type: Home Assistant OS
dev: false
hassio: true
docker: true
user: root
virtualenv: false
python_version: 3.9.6
os_name: Linux
os_version: 5.10.53
arch: aarch64
timezone: Europe/Brussels

GitHub API: ok
Github API Calls Remaining: 4196
Installed Version: 1.13.2
Stage: running
Available Repositories: 909
Installed Repositories: 40

logged_in: true
subscription_expiration: 26 augustus 2021 02:00
relayer_connected: true
remote_enabled: true
remote_connected: true
alexa_enabled: false
google_enabled: true
can_reach_cert_server: ok
can_reach_cloud_auth: ok
can_reach_cloud: ok

host_os: Home Assistant OS 6.2
update_channel: stable
supervisor_version: supervisor-2021.06.8
docker_version: 20.10.6
disk_total: 113.9 GB
disk_used: 12.4 GB
healthy: true
supported: true
board: odroid-n2
supervisor_api: ok
version_api: ok
installed_addons: Samba share (9.5.1), Visual Studio Code (3.6.1), deCONZ (6.9.0), Samba Backup (4.5.0), Assistant Relay (0.7.4), ADB - Android Debug Bridge (0.8.0), Tautulli (2.3.0), Node-RED (9.2.1), Mosquitto broker (6.0.1), File editor (5.3.3), MariaDB (2.4.0), Eufy Home Assistant MQTT Bridge (1.21.0), Frigate NVR (1.13)

Thanks in advance!

Proper spacing or proper quotes for post :slight_smile:

Is it detected by OS?
I don’t use blue or RasPi so not sure how you may confirm but that’d be good start point

Those that are on proxmox I use a coral on a USB 3 port of the host, use pass through from proxmox absolutely fine. Not so happy on a USB 2 port I’d add maybe that’s the issue?

I’m curious what inference speed are you seeing with the USB Coral @versigo, trying to determine if it’s worth it or not. I’m currently averaging 78 with 5 cameras. I have a NUC so USB is my only option without changing hardware. Second question is you don’t mind, have you seen overall CPU reduce as a result of the Coral?

Did you managed to access it to docker ? Or still the same issue ?

I see a lot of people going to a lot of trouble to use integrated Corals, often wondered if I was missing something - I just went for 2 x USB 3.0 connected devices, on the same USB 3.0 hub/port, seems ok to me :slight_smile:

I’m using an old Gigabyte Brix, i7, NUC-like unit.

There are 2 aspects to CPU load:

  • The FFmpeg slicing and dicing of images, this runs on the CPU with hardware acceleration if set/available
  • Detecting objects within the sliced/diced frames, this is what the Coral helps with quite considerably.

have you found a solution for this jerky problem? if have the same problem here with an similar setup

@blakeblackshear

Thank you for a cool and useful product!
I want to make automation based on frigate for opening the gate by identifying my or my wife’s cars. I plan to train my own model. But I haven’t done it yet.
Can you tell me how to properly train my own model? Will this instruction be suitable as a template (Training Custom Object Detector — TensorFlow 2 Object Detection API tutorial documentation)?
Which of the pre-trained models do you recommend using SD Mobile Net v2 320x320, SD Mobile Net V1 FN 640x640, SD MobileNet V2 FPNLite 320x320, SSD MobileNet V2 FPNLite 640x640?
What size should the marked-up images be?

I’m surprised the Inference speed is so good, that’s not good it means I now have another toy on my wish list :slight_smile: Curious, are you running on bare metal or virtualized?

Dedicated box - supervised install.

…. so that when I break the storage, proxmox nodes… I can still I control lights etc :slight_smile:

1 Like