Local realtime person detection for RTSP cameras

I got it working! And it performes good even without Google Coral.

See this thread:

@blakeblackshear I have setup Frigate on Unraid as docker and installed the companion app within HA and everything seems to be setup right. I don’t get any errors while starting up Frigate Docker and i see all the MQTT notifications etc in HA. I also see clips and recordings in the media browser. I currently have 2 issues which i cannot figure out:

  1. My detection for cameras stops working almost immediately (it records maybe a clip or two from one camera) before it just stops registering any person or movement. There are no errors or messages. Frigate is still running, but does nothing.

  2. The recorded clip (using higher resolution rtsp stream) is without audio even though I have verified the same stream via VLC is with full audio. Is this as expected?

My cameras are unifi G3 running in standalone mode
I currently do not have a Coral device or anything else. Currently just using CPU to test. My server is 12 core / 24 threads and its not being taxed very much, so its not a cpu or memory bottleneck

I cannot seem to find a way to upload/attach my config.yml here.

I’ve got this old Dell Optiplex FX160 just laying around and thought about how nice it would be to run frigate on it. It has an Intel Atom 230. So I guess my question is, would this work? And also anybody know if google coral would work on it?

I have now made a clean install (my proxmox server crached), and this time installed Frigate as an Hassio addon. Using Proxmox LXC container.

I am using an Intel I7 4810MQ, but I but the image is distorted.

I do not get any error messages:

The image is distorted both when I do not use any hwaccel and when I use the parameters for Intel Gen 10. If I try to use the parameters for < gen 10 I get an error that vaapi is not found (This was the settings I used on my last install and it did work)

 #ffmpeg:
#  hwaccel_args:
  #  - -hwaccel
  #  - qsv
  #  - -qsv_device
  #  - /dev/dri/renderD128

What have I done wrong?

That’s almost always caused by putting in the wrong video dimensions. If you want to verify your WxH just open the stream in VLC or similar app. Check the stream information to get the dimensions.

2 Likes

I wanted to thank the author for the excellent software - for object detection as well as for the HASS add-on. I ran it with a CPU detector on my nVidia Jetson Nano for over a month with two cameras - without a hitch - usually you don’t get that type of stability from paid production software these days.

I recently added a Coral Edge TPU (m2 - dual TPU version) to the Jetson Nano. After few trials and errors, I was able to get it to run and recognize one TPU.(not sure how to use the second TPU yet). Here are the steps I followed

  1. Install drivers and software from this page: Get started with the M.2 or Mini PCIe Accelerator | Coral
  2. Added “pcie_aspm=off gasket.dma_bit_mask=32” to /boot/extlinux/extlinux.conf - following troubleshooting instructions on the bottom of the above page, The line from the file now looks like
      APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 gasket.dma_bit_mask=32 pcie_aspm=off

  1. I change my detectors in frigate config.yml to
detectors:
  coral_pci:
    type: edgetpu
    device: pci
  1. I run my docker using following command (YMMV)
docker pull blakeblackshear/frigate:stable-aarch64

docker create \
  --name frigate \
  --restart=unless-stopped \
  --device /dev/apex_0:/dev/apex_0 \
  --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \
  -v /dev:/dev \
  -v /mnt/media:/media/frigate \
  -v /mnt/config/config.yml:/config/config.yml:ro \
  -v /etc/localtime:/etc/localtime:ro \
  -p 5000:5000 \
  -p 1935:1935 \
  blakeblackshear/frigate:stable-aarch64

docker start frigate

Hope this helps someone

4 Likes

How did you do this!?

The hardest part is not how to do this but how to bound to the HASS interface so i can see through nabucasa :smiley:

Ok, but how did you do the easy part then? I am serious in my question as I would like to do the same.

I never dealt with proxmox containers but i see a lot of tutorials online.
But i would suggest you to learn how to make containers.

Here are the setup from the developer of this addon.
So i think this might be very easy.
I’m now trying again with the HASS frigate addon mount network share for media folder… long night to go ))

hello, do you have hardware acceleration for stream decoding running on your ryzen CPU?

Good day,
Yes, nvidia TI1050 and it didn’t make much difference.
Thank you

It was the video dimensions that where wrong.

Big thanks!

I asked you how you did it in LXC and you point me to Docker?

Edit: I know there is a snippet for LXC there. But I cant get that working in LXC, hence I currently have an ubuntu vm with Frigate. But I prefer LXC hence my question.

Well if you cant get it working, then post here where are the problems, maybe somebody know how to solve it :slight_smile:

This is leading nowhere… :slight_smile: You are not even answering my question but only sending me back to documentation. I kindly ask for some help on guidance how to setup this on LXC…

1 Like

Ok, concrete question here. I run frigate in an ubuntu VM atm and since coral is not able to deliver I need to use CPU…

my config:

mqtt:
  host: 192.168.1.142
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate200
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate200
  # Optional: user
  user: mqtt
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  password: mqtt
  # Optional: interval in seconds for publishing stats (default: shown below)
  stats_interval: 60
cameras:
  front:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
            - clips
    width: 2560
    height: 1440
    fps: 6
    snapshots:
      enabled: true
      crop: false
    objects:
      track:
        - person
  back:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
            - clips
    width: 2048
    height: 1536
    fps: 4
    snapshots:
      enabled: true
      crop: false
    objects:
      track:
        - person
        - car
  parking:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
    width: 3840
    height: 2160
    fps: 4
    snapshots:
      enabled: true
      crop: false
    objects:
      track:
        - person
        - car
    motion:
      mask: 3840,0,3840,625,3840,786,2769,619,2774,357,2132,230,382,169,377,0
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
clips:
  # Optional: Maximum length of time to retain video during long events. (default: shown below)
  # NOTE: If an object is being tracked for longer than this amount of time, the cache
  #       will begin to expire and the resulting clip will be the last x seconds of the event.
  max_seconds: 30
  # Optional: size of tmpfs mount to create for cache files (default: not set)
  # mount -t tmpfs -o size={tmpfs_cache_size} tmpfs /tmp/cache
  # NOTICE: Addon users must have Protection mode disabled for the addon when using this setting.
  # Also, if you have mounted a tmpfs volume through docker, this value should not be set in your config.
  tmpfs_cache_size: 256m
  # Optional: Retention settings for clips (default: shown below)
  retain:
    # Required: Default retention days (default: shown below)
    default: 100
    # Optional: Per object retention days
    objects:
      person: 150

This does “work” but for camera’s back and parking it stops after starting the server (almost immediately… For camera front it keeps working… I see the time in the UI stopping for those cams and going further for the working one…

what can be the issue?

Edit: what is weird though… the image is not updating and the time in the corner is also stopped and not updating… (for those 2 non working cams) but this is "updating’image

EDIT: there seems to be some confusion around my question(s).

I will clarify below.

a. I am not running frigate in an LXC currently (only asking how this can be done).
b. I am running frigate in a ubuntu VM yes.
c. I am (in another instance) running the frigate addon in home assistant (hassos) which works well with 2 cams. But since it takes away all resources from my homeassistant VM I prefer not to run it there as an addon.

Hence I am now trying 2 routes:

  1. Ubuntu VM (I have that running and that is what this post-question is about)
    2. LXC container (on proxmox) (I don’t have that running)

For 2, I asked another question above.
For 2, I keep asking the same question: how to start a frigate instance on LXC. There is only this in the docs: https://blakeblackshear.github.io/frigate/installation#proxmox but I keep getting pointed to the docs…

First of all my thanks to the developer of this excellent app. I successfully have it running on my Raspberry Pi 4B as an add-on to HA and even without a Coral TPU accelerator it is managing to detect @ 5fps on a 640x480 sub stream and create clips on a 2560x1920 main stream. The detection accuracy is amazing and integration with HA a god-send.

I wanted to highlight something to any users of Reolink’s IP cams and in particular the RLC-520A who are experiencing smearing, glitching, or poor performance. Use the RTMP stream instead of RTSP stream, and include ffmpeg arguments as outlined in the docs (https://blakeblackshear.github.io/frigate/configuration/cameras#reolink-410520-possibly-others). This should fix the issues. Details of RTMP url schema can be found online easily.

Thanks again to the dev.

1 Like

I’m running Home Assistant OS with the Add-on. Running on an old i3 intel Nuc. It’s running great althoug high cpu without a coral at the moment.

Problem is that my HA is fast running out of space due to my frigate clips. Is there a way to store these to a USB drive or NAS?

I am mounting an NFS drive and passing it as an argument to docker with -v /mynfsdir:/media/frigate argument. It works mostly - but a couple of time frigate.db got corrupted when stopping frigate. I had to delete the db and restart (which worked for the most part except lost history of clips). I wish there was a separate directory for database vs media.