Local realtime person detection for RTSP cameras

Edit: Easy done… :slight_smile:
Created a entity:

  - platform: local_file
    file_path: /config/www/img/snapshots/parkering.jpg
    name: cam_parkering_snapshot_latest

And using a Automation when Frigate senses a car … bang… :slight_smile:
Sending that snapshot to “plate recognizer” and a pushover sends me a notify with snapshot and plates. :+1:

Can anyone explain this event, it seems the score is higher in the event:

image
than in the best image:
image

Any tips on eliminating false positives like these? I have max and min area set per camera but this camera covers some distance so object far away appear smaller. Is Zone object filters something added to the feature list?

OK thanks I’ll give this a try. I found the sample frigate config on the site. So the process would be:

  • create the LXC as per config.
  • What templarte or OS do I use for the container? Is any OK or is there a preferred option? (Debian/Ubuntu/Alpine/ Arch?)
  • Install frigate
  • share a directory for media clips and then mount in home assistant also.

Is this correct?

This is how I did it

Hi, I’m not quite familiar with input boolean.
Did i get this right?

/config/input_boolean.yaml

    input_boolean.block_notify_car_porch:
      name: Notify car porch 
      icon: mdi:car
      
    input_boolean.block_notify_luar_dapur:
      name: Notify luar dapur
      icon: mdi:kitchen

Ok, i’ve corrected it

  block_notify_car_porch:
    name: Car Porch
    icon: mdi:car

  block_notify_luar_dapur:
    name: Outside Kitchen
    icon: mdi:food-turkey

I’m getting this error with invalid configuration

Unexpected error calling config validator: unhashable type: 'NodeListClass'

I searched around apparently there’s too many condition?

    alias: Notify on Frigate events
    trigger:
      - platform: mqtt
        topic: frigate/events
    condition:
      - condition: template
        value_template: >-
          {{ trigger.payload_json["after"]["camera"] in ['front_door', 'rightside',
          'leftside', 'backyard', 'driveway'] }}
      - condition: template
        value_template: >-
          {{ is_state(
          "input_boolean.block_notify_"~(trigger.payload_json["after"]["camera"]),
          "off" ) }}

Following up on my previous post and tried the PoE Pi4 with the USB Google Coral and a USB NVMe drive. At least with the combo I had, it was drawing too much to boot. Will try again when the PoE+ hat comes in. Without the NVMe plugged in the switch reports it drawing about 5w-8.5w at 54v, with the NVMe was capping it around 9.3w, this is also a class 0 PoE hat so guessing it was stopping it at that end. It is a PoE+ compliant switch which is barely at 1/2 capacity as well. The official Pi PoE hat reports as Class 3, however I’ve had 3 out of 6 fail, so was using a different one which looks a bit lower power output capability.

@konna, raspi4 can manage h.264 decode till to 1080p, have a look to the previous post Local realtime person detection for RTSP cameras - #4774 by bremby

Hi, i am getting the same problem. Can you please explain the extra steps from the google coral tutorial that you did? Thank you

Sure, in my case I am using the M.2 version of the Coral and I followed these instructions to install (I’m using Debian - Proxmox):

Yes, i have also followed this tutorial but i am having the same HIB errors that you were having when i try to run the example.

Thanks for this. I managed to make a lot of progress.

Remaining issues:

  • VAAPI acceleration is not working for me with the Radeon APU. I’m using kernel 5.11 and passed through /dev/dri/renderD128 to the LXC and then in the compose.
frigate.mqtt                   INFO    : MQTT connected
frigate.app                    INFO    : Camera processor started for livingroom: 38
frigate.app                    INFO    : Capture process started for livingroom: 39
frigate.video                  INFO    : livingroom: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : livingroom: ffmpeg process is not running. exiting capture thread...
ffmpeg.livingroom.detect       ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
ffmpeg.livingroom.detect       ERROR   : [AVHWDeviceContext @ 0x55f119548cc0] Failed to initialise VAAPI connection: -1 (unknown libva error).
ffmpeg.livingroom.detect       ERROR   : Device creation failed: -5.
ffmpeg.livingroom.detect       ERROR   : [h264 @ 0x55f119540080] No device available for decoder: device type vaapi needed for codec h264.
ffmpeg.livingroom.detect       ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
  • What did you do or install to get the Coral USB detected? My lsusb on both Proxmox host and within the LXC shows “Bus 003 Device 004: ID 1a6e:089a Global Unichip Corp.” only.

From memory my issue was the gasket-dkms and libedgetpu1-std packages weren’t being installed (I wasn’t paying attention to the output of the ‘sudo apt-get install gasket-dkms libedgetpu1-std’ as there were errors due to not have the kernel headers installed).
Installing the kernel headers package first cleared this up and it installed successfully.

Had this a few times… reboot host, and do pveupdate and pveupgrade

Did you install the kernel headers in the Proxmox host or the LXC container or both?

Just on the Proxmox host. I actually use Docker on my Proxmox host to run Frigate but I imagine it would be the same for running it in a LXC container.

Thanks but still no luck, I did the following:

  • Install kernel headers.
  • Install gasket-dkms and libedgetpu1-std.
  • Plugged in and out and still getting the Global Unichip Corp identification rather than Google.
  • Added gasket to /etc/modules and rebooted. Same result. I can see the module is running by checking with lsmod.
  • Hot plugged a few more times with same result.
  • There seems to be a related udev rules file but it’s not being picked up:
cat /usr/lib/udev/rules.d/60-libedgetpu1-std.rules 
SUBSYSTEM=="usb",ATTRS{idVendor}=="1a6e",GROUP="plugdev"
SUBSYSTEM=="usb",ATTRS{idVendor}=="18d1",GROUP="plugdev"

UDEV when running in debug gives this:

May 31 14:52:48 littleman kernel: usb 3-2.2: New USB device found, idVendor=1a6e, idProduct=089a, bcdDevice= 1.00
May 31 14:52:48 littleman kernel: usb 3-2.2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
May 31 14:52:48 littleman systemd-udevd[432]: Validate module index
May 31 14:52:48 littleman systemd-udevd[432]: Check if link configuration needs reloading.
May 31 14:52:48 littleman systemd-udevd[432]: Successfully forked off 'n/a' as PID 3992.
May 31 14:52:48 littleman systemd-udevd[432]: 3-2.2: Worker [3992] is forked for processing SEQNUM=3717.
May 31 14:52:48 littleman systemd-udevd[432]: 3-2.2:1.0: Device (SEQNUM=3718, ACTION=add) is queued
May 31 14:52:48 littleman systemd-udevd[432]: 3-2.2: Device (SEQNUM=3719, ACTION=bind) is queued
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Processing device (SEQNUM=3717, ACTION=add)
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: IMPORT builtin 'usb_id' /usr/lib/udev/rules.d/50-udev-default.rules:13
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: IMPORT builtin 'hwdb' /usr/lib/udev/rules.d/50-udev-default.rules:13
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: MODE 0664 /usr/lib/udev/rules.d/50-udev-default.rules:45
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: GROUP 46 /usr/lib/udev/rules.d/60-libedgetpu1-std.rules:1
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Handling device node '/dev/bus/usb/003/009', devnum=c189:264, mode=0664, uid=0, gid=46
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Setting permissions /dev/bus/usb/003/009, 020664, uid=0, gid=46
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Creating symlink '/dev/char/189:264' to '../bus/usb/003/009'
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: sd-device: Created db file '/run/udev/data/c189:264' for '/devices/pci0000:00/0000:00:08.1/0000:03:00.4/usb3/3-2/3-2.2'
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Device (SEQNUM=3717, ACTION=add) processed
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: sd-device-monitor: Passed 517 byte to netlink monitor
May 31 14:52:48 littleman systemd-udevd[432]: 3-2.2:1.0: sd-device-monitor: Passed 315 byte to netlink monitor
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2:1.0: Processing device (SEQNUM=3718, ACTION=add)
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2:1.0: IMPORT builtin 'hwdb' /usr/lib/udev/rules.d/50-udev-default.rules:14
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2:1.0: RUN 'kmod load $env{MODALIAS}' /usr/lib/udev/rules.d/80-drivers.rules:5
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2:1.0: sd-device: Created db file '/run/udev/data/+usb:3-2.2:1.0' for '/devices/pci0000:00/0000:00:08.1/0000:03:00.4/usb3/3-2/3-2.2/3-2.2:1.0'
May 31 14:52:48 littleman systemd-udevd[3992]: Loading module: usb:v1A6Ep089Ad0100dc00dsc00dp00icFEisc01ip02in00
May 31 14:52:48 littleman systemd-udevd[3992]: Failed to find module 'usb:v1A6Ep089Ad0100dc00dsc00dp00icFEisc01ip02in00'
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2:1.0: Device (SEQNUM=3718, ACTION=add) processed
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2:1.0: sd-device-monitor: Passed 360 byte to netlink monitor
May 31 14:52:48 littleman systemd-udevd[432]: 3-2.2: sd-device-monitor: Passed 308 byte to netlink monitor
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Processing device (SEQNUM=3719, ACTION=bind)
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: IMPORT builtin 'usb_id' /usr/lib/udev/rules.d/50-udev-default.rules:13
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: IMPORT builtin 'hwdb' /usr/lib/udev/rules.d/50-udev-default.rules:13
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: GROUP 46 /usr/lib/udev/rules.d/60-libedgetpu1-std.rules:1
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Handling device node '/dev/bus/usb/003/009', devnum=c189:264, mode=0660, uid=0, gid=46
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Setting permissions /dev/bus/usb/003/009, 020660, uid=0, gid=46
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Preserve already existing symlink '/dev/char/189:264' to '../bus/usb/003/009'
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: sd-device: Created db file '/run/udev/data/c189:264' for '/devices/pci0000:00/0000:00:08.1/0000:03:00.4/usb3/3-2/3-2.2'
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: Device (SEQNUM=3719, ACTION=bind) processed
May 31 14:52:48 littleman systemd-udevd[3992]: 3-2.2: sd-device-monitor: Passed 529 byte to netlink monitor

It seems to identify the Coral USB and process the rules but not actually do anything either. And the rest is past my understanding of udev.

Finally, do you have any issues running docker directly on the promox host as I’m tempted to go down this path rather than messing with LXC if I ever get the coral detected properly.

Did you have to do anything special to get it running on Synology Docker with Coral? I’m stuck at Frigate not detecting the usb-tpu. Any tips would be greatly appreciated!

Thanks @sender and @asve99.

Managed to get it working by running something that needs the TPU. This triggered the udev rule and switched the device ID over to Google. Also changes the USB bus numbers but as long as it stays in the same port, it remains the same from then on (at least after a few reboots).

Now I’m on my final issue, getting the graphics to work. I’ve upgraded my kernel to 5.11 which is supposed to suppot AMD Renior graphics but I’m getting:

ffmpeg.livingroom.detect       ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
ffmpeg.livingroom.detect       ERROR   : [AVHWDeviceContext @ 0x562a566e7e00] Failed to initialise VAAPI connection: -1 (unknown libva error).
ffmpeg.livingroom.detect       ERROR   : Device creation failed: -5.
ffmpeg.livingroom.detect       ERROR   : [h264 @ 0x562a56660ac0] No device available for decoder: device type vaapi needed for codec h264.
ffmpeg.livingroom.detect       ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate.video                  INFO    : livingroom: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : livingroom: ffmpeg process is not running. exiting capture thread...

@blakeblackshear any ideas? I have seen mentions of needing a recent mesa driver (20.3+) for decoding but not sure if that helps here.

From my understanding it means that 1080p is the maximum resolution. I’m having 720p so I suppose it should work without any errors. Thanks anyways!

Is there a good tutorial how to install the coral (USB) on a raspi 4 running Home Assistant OS?

Didnt found something in this thread.

€: It seems to be easier then I thought. Just plugged it in and added the coral as detector and it seems to be working.
The integration shows me a inference speed from the coral and the video is running. Waiting for some movement in front of the camera.
grafik