Local realtime person detection for RTSP cameras

[Jooced (Rob)]

Can someone help me fill in the last missing part of my setup?

I have Frigate working on 5 cameras and it’s working great with Hassio integrations, node-red alerting, etc.

The only thing I’m missing is how to move the folder that Frigate uses by default. I want to start keeping 24/7 recordings but I need Frigate to use a different drive than the default Hassio media folder. With docker-compose it would be pretty straight forward by changing the volumes. I just don’t understand where I can do that in the HA space.

[juan11perez (Juan)]

Good day gents, after a few days playing with this component I wanted to share a bit of my experience.

I’ll start with thanking the developer @blakeblackshear for producing and sharing this software.
The object detection, recording and direct integration into home assistant is seamless and just works very well.
I have not come across another piece of software that does all of this, this well and without having to mend different pieces.

The only struggle I’ve had is related to latency and after reading several associated posts I now understand that it is more of a Hass limitation that anything else.

In my case 4 cameras had an average latency of approximately 10secs which is not a big issue for me, but my doorbell had about 35 seconds, which is significant.

I tried some suggestions namely, config, webrtx component and the live555 rtps proxy but I didn’t get significant improvement.

Ultimately what worked for me is zoneminder. Loading the cameras into zoneminder for monitoring only and adding them into HA via the HA integration produced the best results. Nearly imperceptible latency.

So I have frigate doing the whole object detection, recording and notification whilst zoneminder only provides a “window” and I’m happy with the result.

Once again thank you!

[Minglarn]

Curious… Is your HA running on a PI or a NUC?
I’m having near to 0 latency with my 9 cams running on my NUC.

Second question: Camera brand? I’m using ReoLinks with RTMP streams. Runs flawless…

[imellor (Ian Mellor)]

On one of my Lovelace Dashboards I have placed one of the still images of person detection.

type: picture-entity
entity: camera.front_person
tap_action:
  action: more-info

Is it possible to configure so that when it is clicked it goes to the clip of that person trigger.
I see there is a clip API:

https://HA_URL/api/frigate/notifications/<event-id>/<camera>/clip.mp4

However not sure how I grab the “event-id”
Looking in Developer Tools / States my camera.front_person entity does not give an ID:

access_token: d2ab1f018bef12a43fd735815273194c3ab751451bf8986d509432d5a015f0ff
friendly_name: Front Person
entity_picture: /api/camera_proxy/camera.front_person?token=d2ab1f018bef12a43fd735815273194c3ab751451bf8986d509432d5a015f0ff
supported_features: 0

[juan11perez (Juan)]

Unraid server on a ryzen3900x, 64gb ram. Plenty power.

Foscam cameras and that maybe part of the problem. But zoneminder doesn’t seem to care.

[viertausend]

I got it working! And it performes good even without Google Coral.

See this thread:

Adding RTSP-Stream from ESP32-CAM into Frigate

I got it working and I must say I am really impressed. Even without a Google Coral Stick it’s fast and usable. Didn’t expected that. To get it working I flashed the firmware from GitHub - geeksville/TenDollarWebcam: A small example app for ESP32 + Micro-RTSP onto my TTGO T-Camera ESP32 so it is able to output an RTSP stream. The AutoWifi-setup can be a bit annoying but it worked in the end. After that I use the following configuration (frigate.yml): cameras: ESP_TTGO_Cam: ffmpeg: …

[harihoudini]

@blakeblackshear I have setup Frigate on Unraid as docker and installed the companion app within HA and everything seems to be setup right. I don’t get any errors while starting up Frigate Docker and i see all the MQTT notifications etc in HA. I also see clips and recordings in the media browser. I currently have 2 issues which i cannot figure out:

1. My detection for cameras stops working almost immediately (it records maybe a clip or two from one camera) before it just stops registering any person or movement. There are no errors or messages. Frigate is still running, but does nothing.

2. The recorded clip (using higher resolution rtsp stream) is without audio even though I have verified the same stream via VLC is with full audio. Is this as expected?

My cameras are unifi G3 running in standalone mode
I currently do not have a Coral device or anything else. Currently just using CPU to test. My server is 12 core / 24 threads and its not being taxed very much, so its not a cpu or memory bottleneck

I cannot seem to find a way to upload/attach my config.yml here.

[jonkristian]

I’ve got this old Dell Optiplex FX160 just laying around and thought about how nice it would be to run frigate on it. It has an Intel Atom 230. So I guess my question is, would this work? And also anybody know if google coral would work on it?

[jan.lanner (Surfarn_)]

I have now made a clean install (my proxmox server crached), and this time installed Frigate as an Hassio addon. Using Proxmox LXC container.

I am using an Intel I7 4810MQ, but I but the image is distorted.

I do not get any error messages:

The image is distorted both when I do not use any hwaccel and when I use the parameters for Intel Gen 10. If I try to use the parameters for < gen 10 I get an error that vaapi is not found (This was the settings I used on my last install and it did work)

 #ffmpeg:
#  hwaccel_args:
  #  - -hwaccel
  #  - qsv
  #  - -qsv_device
  #  - /dev/dri/renderD128

What have I done wrong?

[Jooced (Rob)]

That’s almost always caused by putting in the wrong video dimensions. If you want to verify your WxH just open the stream in VLC or similar app. Check the stream information to get the dimensions.

[shukkar]

I wanted to thank the author for the excellent software - for object detection as well as for the HASS add-on. I ran it with a CPU detector on my nVidia Jetson Nano for over a month with two cameras - without a hitch - usually you don’t get that type of stability from paid production software these days.

I recently added a Coral Edge TPU (m2 - dual TPU version) to the Jetson Nano. After few trials and errors, I was able to get it to run and recognize one TPU.(not sure how to use the second TPU yet). Here are the steps I followed

1. Install drivers and software from this page: Get started with the M.2 or Mini PCIe Accelerator | Coral
2. Added “pcie_aspm=off gasket.dma_bit_mask=32” to /boot/extlinux/extlinux.conf - following troubleshooting instructions on the bottom of the above page, The line from the file now looks like

      APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 gasket.dma_bit_mask=32 pcie_aspm=off

3. I change my detectors in frigate config.yml to

detectors:
  coral_pci:
    type: edgetpu
    device: pci

4. I run my docker using following command (YMMV)

docker pull blakeblackshear/frigate:stable-aarch64

docker create \
  --name frigate \
  --restart=unless-stopped \
  --device /dev/apex_0:/dev/apex_0 \
  --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \
  -v /dev:/dev \
  -v /mnt/media:/media/frigate \
  -v /mnt/config/config.yml:/config/config.yml:ro \
  -v /etc/localtime:/etc/localtime:ro \
  -p 5000:5000 \
  -p 1935:1935 \
  blakeblackshear/frigate:stable-aarch64

docker start frigate

Hope this helps someone

[sender]

@jan.lanner

Frigate as an Hassio addon. Using Proxmox LXC container

How did you do this!?

[ukro]

@sender

How did you do this!?

The hardest part is not how to do this but how to bound to the HASS interface so i can see through nabucasa

[sender]

Ok, but how did you do the easy part then? I am serious in my question as I would like to do the same.

[ukro]

I never dealt with proxmox containers but i see a lot of tutorials online.
But i would suggest you to learn how to make containers.

Installation | Frigate

Here are the setup from the developer of this addon.
So i think this might be very easy.
I’m now trying again with the HASS frigate addon mount network share for media folder… long night to go ))

[zeliant (Terence Yung)]

hello, do you have hardware acceleration for stream decoding running on your ryzen CPU?

[juan11perez (Juan)]

Good day,
Yes, nvidia TI1050 and it didn’t make much difference.
Thank you

[jan.lanner (Surfarn_)]

It was the video dimensions that where wrong.

Big thanks!

[sender]

I asked you how you did it in LXC and you point me to Docker?

Edit: I know there is a snippet for LXC there. But I cant get that working in LXC, hence I currently have an ubuntu vm with Frigate. But I prefer LXC hence my question.

[ukro]

Well if you cant get it working, then post here where are the problems, maybe somebody know how to solve it