Local realtime person detection for RTSP cameras

That’s unfortunate, I have been following your updates as I wanted to do the same on Unraid.
I only have USB2 on my system, done some googling but I cannot see if it will have a performance impact on the Coral USB card? That’s another reason I wanted to get the USB3 PCI card working.

If you do ever get it working, please put an update on this thread. Thank you.

1 Like

I certainly will update when/if I get it working. I am going to post in the unraid forums shortly for some help. I definitely can get Unraid to recognize the PCI card, just don’t know how to get the container to recognize it.

thinking about running a low-resource usage linux vm just for frigate. That may help with figuring out the correct way to pass it through

also having some weird issues where it takes a few minutes of the server being on to be able to use the USB device. Not sure what’s going on there.

1 Like

Thanks :slight_smile:
I just got the USB version working, went from 540 to 160 inference speed, now trying to get it to detect motion. Me jumping about is making the detection frames go up but no motion alert…
I took out the mask setting from the config as I want all the camera image to be scanned, not sure yet if thats the problem.
Bl00dy addictive this is, i need to go to bed, its 4am here, DOH!

nice! 160 inference speed still seems high. I get 10ms when the usb is not detected, jumps to hundreds when motion detected. 7-8ms constant when usb is working.

Here’s my config - only thing I have filtering is the area and score.

web_port: 5000

detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: 192.168.1.X
  port: 1883
  topic_prefix: f_porch
  client_id: f_porch

save_clips:
  max_seconds: 60
  clips_dir: /clips
  cache_dir: /cache
  
objects:
  track:
    - person
  filters:
    person:
      min_area: 15000
      min_score: 0.80
    
cameras:
  porch:
    ffmpeg:
      input: rtsp://
      height: 1200
      width: 1600
      fps: 30

    save_clips:
      enabled: True
      pre_capture: 5
      objects:
        - person      

    snapshots:
     show_timestamp: False
     draw_zones: True
     draw_bounding_boxes: false

    objects:
      track:
        - person

Thanks, that helped.

Got motion detected on 2 cameras but not on 3rd ( 7 cameras in total but its -1c so too cold and dark to do any more testing) so probably some typo somewhere but too tired now, off to bed :slight_smile:
Happy it’s starting to come together. Will update later
Cheers :slight_smile:

1 Like

Did you follow the getting started guide? https://coral.ai/docs/m2/get-started/#requirements

I tried and never got the card to be recognized as lspci -x | grep 089a

trying to use a coral device on my synology 718+

inserting it give:
lsusb

|__usb1          1d6b:0002:0404 09  2.00  480MBit/s 0mA 1IF  (Linux 4.4.59+ xhci-hcd xHCI Host Controller 0000:00:15.0) hub
  |__1-4         f400:f400:0100 00  2.00  480MBit/s 200mA 1IF  (Synology DiskStation 65007CFF19DA5A70)
|__usb2          1d6b:0003:0404 09  3.00 5000MBit/s 0mA 1IF  (Linux 4.4.59+ xhci-hcd xHCI Host Controller 0000:00:15.0) hub
|__usb3          1d6b:0002:0404 09  2.00  480MBit/s 0mA 1IF  (Linux 4.4.59+ etxhci_hcd-170202 Etron xHCI Host Controller 0000:02:00.0) hub
|__usb4          1d6b:0003:0404 09  3.00 5000MBit/s 0mA 1IF  (Linux 4.4.59+ etxhci_hcd-170202 Etron xHCI Host Controller 0000:02:00.0) hub
  |__4-1         1a6e:089a:0100 00  3.10 5000MBit/s 896mA 1IF  ( ffffffd1ffffffb2ffffffdbffffffad)
```
starting Frigate gives: No EdgeTPU detected. Falling back to CPU.

i'm starting Frigate in docker:
```
docker run --rm --privileged --shm-size=1g -v /dev/bus/usb:/dev/bus/usb -v /volume1/docker/frigate/config:/config:ro -v /volume1/docker/frigate/cache:/cache -v /volume1/docker/frigate/config:/clips -v /etc/localtime:/etc/localtime:ro -v /volume1/docker/frigate/storage:/storage -p 5555:5555 -e FRIGATE_RTSP_PASSWORD='xxx' blakeblackshear/frigate:0.7.3-amd64 &

not sure how to solve this please?

This could be because the USB port you are using is a sub-hub as described here: https://coral.ai/docs/edgetpu/multiple-edgetpu/#performance-considerations

i’m plugged-in directly to the synology ports without using an external USB hub.
putting it in usb2 gives the same results btw

I’m not sure why the USB isn’t being detected then. I would compare the output of lsusb -t to the link I posted to see if that is the issue. Maybe someone else on here is using that Synology machine. I have also see some issues with USB ports not providing enough power.

thanks for following up
on the synology the lsusb -t does not exist so unable to show more info.
from the lsusb command it looks like 4-1 uses 896mA

in my synology control panel there is one usb shown: USBDevice - Gloval Unichip Corp. VID:1a6e PID: 089a Rev. 1.00. I have not other devices connected so that must me the Coral. Not sure though if this info is of any use

Under my synology external devices tab i have no devices connected but i think that normally you would see external drives here.

I have rebooted the synology various time too after moving the Coral to a different usb for test and no results either.

can it have something to do with r&w rights? or drivers which are not available on the synology?
Did try sudo apt-get install libedgetpu1-std earlier, but apt-get is not recoginised.

uname -a outputs
Linux DS718 4.4.59+ #25426 SMP PREEMPT Wed Jul 8 03:21:29 CST 2020 x86_64 GNU/Linux synology_apollolake_718+

cat /proc/version outputs:
Linux version 4.4.59+ (root@build3) (gcc version 4.9.3 20150311 (prerelease) (crosstool-NG 1.20.0) ) #25426 SMP PREEMPT Wed Jul 8 03:21:29 CST 2020

not sure if thats usefull either?

This is the correct device. VID/PID are good.
What permissions do you have for this usb device? Here are mine:

crw-rw-r-- 1 root root 189, 129 Dec  2 09:35 /dev/bus/usb/002/002
patfee@DS718:/dev/bus/usb/004$ ls -l
total 0
crw------- 1 root root 189, 384 Dec  2 08:28 001
crw------- 1 root root 189, 385 Dec  2 08:28 002

i cant go in the 001 or 002 directory (not a file or directory)

changed the rights as above.
rebooted nas

started frigate again with no change:

Fontconfig error: Cannot load default config file
On connect called
Camera capture process started for fdc_frigate: 20
eating ffmpeg process...
Camera process started for fdc_frigate: 22
Starting detection process: 19
ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://hassio:[email protected]/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif -r 4 -f rawvideo -pix_fmt yuv420p pipe:Attempting to load TPU as usb:4
No EdgeTPU detected. Falling back to CPU.
* Serving Flask app "detect_objects" (lazy loading)
* Environment: development
* Debug mode: off

I don’t know why it isn’t being detected. I would see of you can get any of the getting started guides to work from the docs: https://coral.ai/docs/accelerator/get-started/#3-run-a-model-using-the-tensorflow-lite-api

Thanks Blake

Maybe as a final question, could it have something to do with user rights?
dont have a specific docker user configured

I don’t think so.

well, look at his permission rights. Only root can access the USB key.

@patfee how is docker run on the synology? by root user?

@S1eepy, using patfee as the user

I can’t wait to test your new HA integration: any hint when we can start to test these new Frigate features?

Thanks for your work!