Local realtime person detection for RTSP cameras

Which version are you using Ender?

I think that docker-compose file is from the original cpu-only version. There’s an example docker-compose in the latest readme files. Only one container

  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true
    image: frigate:latest
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - <path_to_config>:/config
    ports:
      - "5000:5000"
    environment:
      RTSP_PASSWORD: "password"

I did use the original one, but I added the usb passthrough volume and the privileged: true to it to make it work with the newer version. When I start up two containers simultaneously with docker compose, one of them fails to read the usb accelerator. Wondering if I need to put in some kind of time offset into starting up the containers, or if it’s a docker limitation where you can’t pass through the same usb device to two containers…

Sorry, my compose file listed above was from the CPU version… Here is one I think will work with Coral. I have moved over to my synology and am using the synology tools to manage the container now, but this compose was working on my old host, though there were other issues with compatibility with my cameras which were since solved:

version: "3"
services:

  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true
    image: frigate:latest
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /root/frigate/config:/config
    ports:
      - 5000:5000
    environment:
      - RTSP_PASSWORD=PASSWORD
      - RTSP_PASSWORD_RIGHT=PASSWORD
      - RTSP_PASSWORD_LEFT=PASSWORD
      - RTSP_PASSWORD_FRONT=PASSWORD
      - RTSP_PASSWORD_BACK=PASSWORD

Most of the important stuff is now in the config file rather than the docker compose/command line. If your cameras all use the same password, you can have just one password line… I had 5 passwords used for various cameras in different connection ways…

1 Like

You can’t have more than 1 container with the Coral. You have to configure all cameras to be processed with a single container. Just duplicate the configuration block under cameras for additional cameras.

Thanks, @scstraus, @blakeblackshear. I thought I had seen a post above saying multiple cameras required multiple containers. I guess that was for the CPU version. I’ll merge it all together and give it a shot in one container.

these vsync options have significantly decreased the queue full messages i was getting across 6 cameras, thanks a lot for sharing

I just got my coral delivered today and have been trying to set this up, thoguht I was close but reading more of the thread it sounds like some configs are for cpu while some are for tpu. Could someone please let me know if they see what is wrong with my config? I also not sure where the label_map file is, Ive downloaded this from above ‘http://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz’.

this is the error i get

ERROR: pull access denied for frigate, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

this is my config

root@cor:/opt/frigate# cat config/config.yml 
web_port: 5000

mqtt:
  host: 10.1.99.113
  topic_prefix: frigate
#  user: username # Optional -- Uncomment for use
#  password: password # Optional -- Uncomment for use

cameras:
  unifi:
    rtsp:
      user: blah
      host: 10.2.2.60
      port: 7447
      # values that begin with a "$" will be replaced with environment variable
      password: blahblah
      path: /LKNVLKSDNVKLN

    ################
    ## Optional mask. Must be the same dimensions as your video feed.
    ## The mask works by looking at the bottom center of the bounding box for the detected
    ## person in the image. If that pixel in the mask is a black pixel, it ignores it as a
    ## false positive. In my mask, the grass and driveway visible from my backdoor camera 
    ## are white. The garage doors, sky, and trees (anywhere it would be impossible for a 
    ## person to stand) are black.
    ################
    # mask: back-mask.bmp

    ################
    # Allows you to limit the framerate within frigate for cameras that do not support
    # custom framerates. A value of 1 tells frigate to look at every frame, 2 every 2nd frame, 
    # 3 every 3rd frame, etc.
    ################
    take_frame: 1

    ################
    # Optional hardware acceleration parameters for ffmpeg. If your hardware supports it, it can
    # greatly reduce the CPU power used to decode the video stream. You will need to determine which
    # parameters work for your specific hardware. These may work for those with Intel hardware that
    # supports QuickSync.
    ################
    # ffmpeg_hwaccel_args:
    #   - -hwaccel
    #   - vaapi
    #   - -hwaccel_device
    #   - /dev/dri/renderD128
    #   - -hwaccel_output_format
    #   - yuv420p
    
    regions:
      - size: 1920
        x_offset: 0
        y_offset: 0
        min_person_area: 5000
        threshold: 0.8
#      - size: 400
#        x_offset: 350
#        y_offset: 250
#        min_person_area: 2000
#        threshold: 0.5
#      - size: 400
#        x_offset: 750
#        y_offset: 250
#        min_person_area: 2000
#        threshold: 0.5

this is my compose file

root@cor:/opt/frigate# cat docker-compose.yml 
version: "3"
services:

  frigate1:
    container_name: tensorflow
    restart: unless-stopped
    image: frigate
    volumes:
      - /opt/frigate/config:/config:ro
      - /lab/debug:/lab/debug:rw
      - /opt/ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb:/frozen_inference_graph.pb:ro
    ports:
      - 5000:5000
    environment:
      - RTSP_URL=rtsp://10.2.2.2:7447/SVKMNSIFVHIFNSIVNS
      - MQTT_HOST=10.1.99.113
      - MQTT_TOPIC_PREFIX=cameras/1
      - DEBUG=0

Im running this on an ubuntu 18 vm thats on a dell r720 - 16 CPUs x Intel® Xeon® CPU E5-2690 0 @ 2.90GHz

I am usiing unifi cameras…going from the above thread i just used anything for UN PW sense its not requred, and the path i filled in with random charectors for this post.

1 Like

i did run docker build earlier…does it take a long time?

its sitting here, i stopped it last time thinking it was running the container i wanted but now maybe its actually installing still…

Removing intermediate container 436aedafb599
 ---> 1919f9873417
Step 6/24 : RUN apt-get -qq update && apt-get -qq install --no-install-recommends -y  python3  ffmpeg  build-essential  cmake  unzip  pkg-config  libjpeg-dev  libpng-dev  libtiff-dev  libavcodec-dev  libavformat-dev  libswscale-dev  libv4l-dev  libxvidcore-dev  libx264-dev  libgtk-3-dev  libatlas-base-dev  gfortran  python3-dev  libusb-1.0-0  python3-pip  python3-pil  python3-numpy  libc++1  libc++abi1  libunwind8  libgcc1  libva-drm2 libva2 i965-va-driver vainfo  && rm -rf /var/lib/apt/lists/*
 ---> Running in d7799c7a759c

i hate that i fight with something for hours, finally give up and ask for help…and then see what it is im doing that is dumb. sorry about that.

Step 6/24 argh

if someone can help with where the label_map is i should be able to figure this out. thank you

Hi Darbos

If you are using the latest version (or a recent version), the model and labels are already included. The build takes approx an hour on a rpi4 from memory, so shouldn’t be taking that long. You don’t need to build it though? Blake has uploaded images to docker hub. Also, see the readme on the latest version for a docker-compose example for use with coral

Hope that helps a little bit at least

1 Like

Thanks Onion. I think I might be missing a link or something, I have been following the directions at the top of this post
https://github.com/blakeblackshear/frigate/blob/700bd1e3efb59c1b01bebab3b8344e24aebcf657/README.md

it looks like i need to build first then run the container according to the directions, is there a link to the pre-built containers?

Ive made some progress, it looks like im stuck on accessing the coral though. im digging into that now.

root@cor:/opt/frigate# docker logs -f 4f019bb9b091
On connect called
Traceback (most recent call last):
  File "detect_objects.py", line 99, in <module>
    main()
  File "detect_objects.py", line 57, in main
    prepped_frame_queue
  File "/opt/frigate/frigate/object_detection.py", line 32, in __init__
    self.engine = DetectionEngine(PATH_TO_CKPT)
  File "/usr/local/lib/python3.6/dist-packages/edgetpu/detection/engine.py", line 72, in __init__
    super().__init__(model_path)
  File "/usr/local/lib/python3.6/dist-packages/edgetpu/basic/basic_engine.py", line 40, in __init__
    self._engine = BasicEnginePythonWrapper.CreateFromFile(model_path)
RuntimeError: Error in device opening (/sys/bus/usb/devices/1-1)!

1 Like

HI, Just for info I first tried with ubuntu 18 VM ESXI host and couldn’t get the coral to work. Moved to Ubuntu 18 on a standalone machine and worked straight away both with the Google Coral Examples and the frigate docker images.
You can try this google coral example to test if your setup will work with your Google Coral if you haven’t already:
https://coral.withgoogle.com/docs/accelerator/get-started/

Cool, that looks like the latest readme. The latest release is tagged v0.2.0 https://github.com/blakeblackshear/frigate/tree/v0.2.0

Is that the version you built?

Prebuilt images are here https://hub.docker.com/r/blakeblackshear/frigate

Looks like you are getting there though. Not had that error myself. I just used the /dev/bus/usb

Thanks man. That first link is the correct one, what I have been working with. Anychance any one has seen this error? It looks like its running now but after a few minutes it spits out this error, and looks like it goes back to runnin? but i also cant access to web url

root@cor:/opt/frigate# docker logs -f frigate
On connect called

Traceback (most recent call last):
  File "detect_objects.py", line 99, in <module>
    main()
  File "detect_objects.py", line 57, in main
    prepped_frame_queue
  File "/opt/frigate/frigate/object_detection.py", line 32, in __init__
    self.engine = DetectionEngine(PATH_TO_CKPT)
  File "/usr/local/lib/python3.6/dist-packages/edgetpu/detection/engine.py", line 72, in __init__
    super().__init__(model_path)
  File "/usr/local/lib/python3.6/dist-packages/edgetpu/basic/basic_engine.py", line 40, in __init__
    self._engine = BasicEnginePythonWrapper.CreateFromFile(model_path)
RuntimeError: Error in device opening (/sys/bus/usb/devices/1-2)!
On connect called
root@cor:/opt/frigate# cat docker-compose.yml 
  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true
    image: frigate:latest
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /opt/frigate/config:/config
    ports:
      - "5000:5000"
root@cor:/opt/frigate# cat config/config.yml
web_port: 5000

mqtt:
  host: 10.1.99.113
  topic_prefix: frigate

cameras:
  unifi:
    rtsp:
      host: 10.x.x.x
      port: 7447
      path: /longunifistring
      user: this
      password: could
    regions:
      - size: 1920
        x_offset: 0
        y_offset: 0
        min_person_area: 5000
        threshold: 0.8
1 Like

Your size can’t be 1920 because your image is only 1080 pixels tall. Your region is outside the bounds of your image.

Also, you will be much better off using a substream that is a lower resolution where your region size is closer to 300. There is no improvement in accuracy for using a 1080p stream if your region size is greater than 300. All regions are resized to 300 before detection is run, so you are just using extra CPU cycles to resize the image down. Rule of thumb is to choose a resolution that gets your smallest region as close to 300 as possible.

OK thank you. I just tried to change it to 1080 and got the same error. I actually just copied that from a person above that is using unifi as well. It looks to me like it cant connect to the usb, based on the last line in the error. however i can see it fine device (see below).

Just seen the second post, thats good info i assumed 1080, I will use the unifi stream 640x360 for this then, i assume adjust the size in config to 360 as well.

I havent even really dont anything to the region section yert, i was just trying to get her working first, so anything i need to change there is no issue.

ill make those changes and report back. I do have to go eat dinner but ill be back an a hour or two im not giving up! :slight_smile:

root@cor:/opt/frigate# lsusb

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 001 Device 005: ID 0e0f:0002 VMware, Inc. Virtual USB Hub

Bus 001 Device 004: ID 0e0f:0002 VMware, Inc. Virtual USB Hub

Bus 001 Device 003: ID 1a6e:089a Global Unichip Corp. <–this is the same name/how it looks in vmware when i add it to the vm.

Bus 001 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

root@cor:/opt/frigate#

Doing this through VMWare is tricky… the Coral likes to change Hardware IDs. There is a website out there that walks you through it, and will give you both hardware IDs to use, but don’t be surprised if you have to keep changing the USB device filters for passing into the VM to get it to recognize and work.

Honestly, I gave up on it after awhile (thankfully the RP4 came in.)

Also side note, I just read some article saying Hikvision and Dahua cams are now banned for import in the US.
https://ipvm.com/reports/aug-13-2019

Alright, I really need some help PLEASE. Ive spent the entire day on this now. I cant figure out the pre-built or manual install. after i do docker pull what do i need to do for the pre-built docker?

for the manual install here is my configs and output:

(venv) root@ai:/opt/frigate# cat docker-compose.yml 
  frigate:
    container_name: frigate:latest
    restart: unless-stopped
    privileged: true
    image: frigate:latest
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /opt/frigate:/config
    ports:
      - "5000:5000"
(venv) root@ai:/opt/frigate# cat config/config.yml 
web_port: 5000

mqtt:
  host: 10.1.99.113
  topic_prefix: frigate
#  user: username # Optional -- Uncomment for use
#  password: password # Optional -- Uncomment for use

cameras:
  unifi:
    rtsp:
      user: viewer
      host: 10.2.2.2
      port: 7447
      # values that begin with a "$" will be replaced with environment variable
#      password: $RTSP_PASSWORD
      path: /longunifistring

    ################
    ## Optional mask. Must be the same dimensions as your video feed.
    ## The mask works by looking at the bottom center of the bounding box for the detected
    ## person in the image. If that pixel in the mask is a black pixel, it ignores it as a
    ## false positive. In my mask, the grass and driveway visible from my backdoor camera 
    ## are white. The garage doors, sky, and trees (anywhere it would be impossible for a 
    ## person to stand) are black.
    ################
    # mask: back-mask.bmp

    ################
    # Allows you to limit the framerate within frigate for cameras that do not support
    # custom framerates. A value of 1 tells frigate to look at every frame, 2 every 2nd frame, 
    # 3 every 3rd frame, etc.
    ################
#    take_frame: 1

    ################
    # Optional hardware acceleration parameters for ffmpeg. If your hardware supports it, it can
    # greatly reduce the CPU power used to decode the video stream. You will need to determine which
    # parameters work for your specific hardware. These may work for those with Intel hardware that
    # supports QuickSync.
    ################
    # ffmpeg_hwaccel_args:
    #   - -hwaccel
    #   - vaapi
    #   - -hwaccel_device
    #   - /dev/dri/renderD128
    #   - -hwaccel_output_format
    #   - yuv420p
    
    regions:
      - size: 360
        x_offset: 0
        y_offset: 300
        min_person_area: 5000
        threshold: 0.5
#      - size: 400
#        x_offset: 350
#        y_offset: 250
#        min_person_area: 2000
#        threshold: 0.5
#      - size: 400
#        x_offset: 750
#        y_offset: 250
#        min_person_area: 2000
#        threshold: 0.5

(venv) root@ai:/opt/frigate# sudo docker run --rm \

> --privileged \

> -v /dev/bus/usb:/dev/bus/usb \

> -v /opt/frigate:/config:ro \

> -p 5000:5000 \

> blakeblackshear/frigate

standard_init_linux.go:211: exec user process caused "exec format error"

(venv) root@ai:/opt/frigate# docker logs frigate



(venv) root@ai:/opt/frigate# docker-compose up -d

Pulling frigate (frigate:latest)...

ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
1 Like