Local realtime person detection for RTSP cameras

I have:

RLC-520A Reolink RLC-520A - 5MP Person/Vehicle Detection Smart PoE Security Camera
RLC-822A H265 Reolink RLC-822A – 4K 8MP Person/Vehicle Detection PoE Camera with 3X Optical Zoom
RLC-820A H265 Reolink RLC-820A – Person/Vehicle Detection PoE Security IP Camera with 4K

These cameras 4k have rtsp h265 full resolution, and low resolution h264.

The question is record at full size

i see, did you tryed the substream? Because i have one of 10 cameras that doesnt have fullhd h264, but when i use substream, the option apears there.

in according to this docs from reolink

I use:
full size 4k h265
rtsp://admin:[email protected]:554//h265Preview_01_main

kiw size h264
rtsp://admin:[email protected]:554//h264Preview_01_sub

i didn’t see others url can works, rtmp seems doesn’t works

strange, it should work. If you want i can remote desktop(anydesk) with skype and try to figure it out.
Otherwise, i have no idea to go further

Hi, check my post: Local realtime person detection for RTSP cameras - #5000 by Artekus

I have RLC-820A too and Reolink send me custom firmware to have main stream h.264. But it’s not 4k, it’s 5MP I think. But I don’t care, it works fine :blush:

Hi,

thanks to reply, so the solution is use a 4k camera, like 2k camera. It’s ok if isn’t possible get a refund, but i bought them in amazon, so i get a refund now about these 4k camera and get 5mp reolink h264.

H265 is a mess, i looking for blue iris that seems works in h265, but in windows, so i use HA in proxmox in a nuc.

I have no choice to use 2k camera, 4k isn’t possible and it’s a pity.

There are some few cameras 4k in h264. Are very hard to find

thanks anyway, but i read a lot and seems that isn’t possible.

Reolink 4k camera have full size stream only in h265, and like @palito said, reolink can’t do a 4k stream in h264, maybe because hardware onboard, can’t manage it, i don’t know.

Anyway this issue, have no solution, only use h264 not 4k.

1 Like

Or try zoneminder

thanks, i’ll try now

With this configuration i can see on frigate the preview in low resolution h264.

But with this configuration, should be record in full size 4k, but not.

On frigate I can see that detect an human, in tracked object, but doesn’t save any files, and doesn’t create of course, any events.

Is normal? I guess no.

I’m happy if preview is on low res h264, but record is 4k.

Is possible and how with frigate?

Or i must try zoneminder, or refund and get 5mp reolink h264 and amen

  balcone_grande:

    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
          roles:
            - detect
        - path: rtsp://admin:[email protected]:554/
          roles:
            - clips
            - record
            - rtmp
    width: 640
    height: 360
    fps: 3
    detect:
      enabled: True

log:

ffmpeg.balcone_grande.clips_record_rtmp ERROR   : Could not write header for output file #1 (incorrect codec parameters ?): Function not implemented
ffmpeg.balcone_grande.clips_record_rtmp ERROR   : 
watchdog.balcone_grande        INFO    : Terminating the existing ffmpeg process...
watchdog.balcone_grande        INFO    : Waiting for ffmpeg to exit gracefully...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55c6b1a6c140] moov atom not found
/tmp/cache/balcone_grande-20210828205449.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: balcone_grande-20210828205449.mp4
ffmpeg.balcone_grande.clips_record_rtmp ERROR   : [flv @ 0x556005aca300] Video codec hevc not compatible with flv
ffmpeg.balcone_grande.clips_record_rtmp ERROR   : Could not write header for output file #1 (incorrect codec parameters ?): Function not implemented
ffmpeg.balcone_grande.clips_record_rtmp ERROR   : 
watchdog.balcone_grande        INFO    : Terminating the existing ffmpeg process...
watchdog.balcone_grande        INFO    : Waiting for ffmpeg to exit gracefully...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x564a4a43c140] moov atom not found
/tmp/cache/balcone_grande-20210828205459.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: balcone_grande-20210828205459.mp4
ffmpeg.balcone_grande.clips_record_rtmp ERROR   : [flv @ 0x55c11787edc0] Video codec hevc not compatible with flv
ffmpeg.balcone_grande.clips_record_rtmp ERROR   : Could not write header for output file #1 (incorrect codec parameters ?): 

Hello

I looking for enable HW acceleration on docker : PI4/4GB /arrch64 + Coral USB.
Docker image 0.9.0-rc2

I tried many combinaisons, and either container is not starting or cam is only green.

Without HW setting, everything is fine but inference speed isn’t the best :frowning: I’am expecting much better

Here is my yaml

# frigate.yml
detectors:
  coral:
    type: edgetpu
    device: usb
  # cpu1:
  #   type: cpu

# ffmpeg:
#   hwaccel_args:
#     - '-c:v'
#     - -hwaccel
#     - qsv
#     - -qsv_device
#     - h264_v4l2m2m
#     - h264_mmal
#     - h264_qsv
#     - cuda
#     - h264_cuvid
#     - vaapi
#     - -hwaccel_device
#     - /dev/dri/renderD128
#     - -hwaccel_output_format
#     - yuv420p
    
mqtt:
  host: 192.168.1.2
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: mqtt
  password: password
  stats_interval: 60

birdseye:
  enabled: False

cameras:


  contre-allee:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.3:554/h264D1?username=admin&password=password
          roles:
            - rtmp
            - record
        #- path: rtsp://192.168.1.3:554/h264CIF?username=admin&password=password
        - path: rtsp://192.168.1.3:554/h264?username=admin&password=password
          roles:
            - detect
    best_image_timeout: 15                                                        
    detect:                                                                       
      enabled: True                                                               
      max_disappeared: 15
      width: 2592
      height: 1944
      fps: 5
    objects:
      track:
        - airplane
        - bicycle
        - bird
        # - bus
        - car
        - cat
        - dog
        - motorcycle
        - person
        # - truck
      filters:
        airplane:
          mask:
            - 0,1944,2592,1944,2568,1404,2008,1006,1600,1366,1135,1334,1003,1241,1013,1032,0,932
        person:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
        bicycle:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
        truck:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
        motorcycle:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
        cat:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
        dog:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
        car:
          mask:
            - 2592,0,2592,1318,0,1213,0,0
            - 1634,1944,1523,1802,857,1757,684,1944
    motion:
      threshold: 50
    snapshots:
      enabled: true
      clean_copy: True
      timestamp: True
      bounding_box: True
      crop: False
      retain:
        default: 2

    record:
      enabled: True
      retain_days: 2
      events:
        enabled: False
        max_seconds: 300
        pre_capture: 10
        post_capture: 5
        objects:
          - airplane
          - bicycle
          - bird
          # - bus
          - car
          - cat
          - dog
          - motorcycle
          - person
        # - truck

logger:
  default: debug
  logs:
    frigate.video: debug

Is your Coral plugged into a USB3 port?

Also, I don’t think the GPU on the RPi4 has enough memory to decode that resolution.

Hi blakeblackshear,

Thanks for your reply Coral is connected on USB3. Even though i had to use Y cable for additional power. I do hope it does not reduce efficiency.
Regarding resolution I can use substream with lower size (line is commented above path).
As I said, i’m running quite fine, but not sure it’s optimized. I notice local ffmpeg and docker’s one aren’t using the same librairies.



So i’m a bit lost. Do you have any suggestion ?

Anyone experiencing problems with EZVIZ C6N? The resolution changes by itself all the time (I put standard definition on the Ezviz app, but it keep changing?

What Linux distro are you using as I could only get it to work with Raspberry Pi OS Lite with privileged: true on the docker container.
I tried Debian but it always said this in the docker log and couldn’t find out how to get it to work.

ERROR : [h264_v4l2m2m @ 0xaaaacc7156c0] Could not find a valid device

1 Like

I was using the RPIOS 64b lite… And also added privileged in docker-compose… No success ATM
I’m on my way to try ubuntu server as it seems to me that vaapi is supported.

Hello, it is quite a while that I am using HA for my SmartHome. On last week I started on person detection.
Since I have HA running fine on my Synology DS918+ in VMM I first tried to use the HA add-on for frigate. I got this running using this video on YouTube:

For the tryout I used an Eufy Indoor Cam Pan&Tilt with a resolution of 2K.

But with even configuring 3 cpu’s for detection the performance was not satisfying.
I then discovered that it is possible to install frigate externally on a Raspi 3 or 4 and use the Frigate NVR proxy to integrate the external frigate into HA.
I also purchased a coral usb to accelerate the detection on my Raspi 3 (32bit for now).

I found very little information on how to get such a setup running since I also never used docker on a Raspi.
But I could get it up an running using these information:
On actual Raspian lite.
docker and docker-compose on the Raspi:

Here are my config files: (frigate 0.8.4 for the beginning)
docker-compose.yaml

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:0.8.4-armv7
    devices:
      - /dev/bus/usb:/dev/bus/usb
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/frigate/config.yml:/config/config.yml:ro
      - /media/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: „verysecred“

I could leave out the line with renderD128 on a Raspi, correct?
/etc/frigate/config.yml:

mqtt:
  host: 192.168.x.xx
  user: homeassistant
  password: xxxxx

cameras:
  syn_cam_1:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]/live0
        #- path: rtsp://syno:[email protected]:554/Sms=1.unicast
        #- path: rtsp://syno:[email protected]:554/Sms=1.multicast
          roles:
            - detect   # only if you want object detection
            - clips    # only if you want to record video and stills
      #hwaccel_args:
      #  - -c:v
      #  - h264_mmal
    width: 1920     # e.g. 1280
    height: 1080   # e.g. 720
    fps: 5
    objects:
      track:       # add more object types under this if you want
        - person   # only if you want to detect persons
        - teddy bear
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 2

detectors:
  coral:
    type: edgetpu
    device: usb

I could not yet get hwaccel to work. If I activate these setting I only get a green video.
Any hint for that?

For the detection the provided resolution is much higher than needed, but I could not get any solution to reduce this.

Anyway the performance is satisfying for now. (Coral Inference Speed: about 68)
Thanks to all that are working on this great software: frigate and HA!

Hi, someone get stuck when viewing snapshot on iOS?
I cannot exit to app :))
I must uninstall app and reinstall it.

So hello again,
i have change to frigate 0.9.0 Release Candidate 2

now my reolink camera’s (RLC-520) are smearing…
Before in v0.8.4 everything was fine…
can someone help to solve this problem?
here is my config:

ffmpeg:
#  hwaccel_args:
#    - -hwaccel
#    - cuda
#    - -c:v 
#    - h264_cuvid
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 #-c:v h264_cuvid
  output_args:
    detect: -f rawvideo -pix_fmt yuv420p -filter:v fps=fps=5
    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    rtmp: -c copy -map 0 -f flv
      - -filter:v
#      - fps=fps=5
mqtt:
  host: 10.0.0.65
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  stats_interval: 60
  
cameras:
  Front:
    ffmpeg:
      output_args:   
        rtmp: -c copy -map 0 -f flv
      inputs:
#        - path: rtsp://admin:[email protected]/1
#          roles:
#            - detect
        - path: rtsp://admin:[email protected]/Streaming/Channels/1
          roles:
            - detect
            - rtmp
#            - clips
#    width: 720
#    height: 480
    width: 2560
    height: 1440
    fps: 5
    clips:
      enabled: false
      pre_capture: 5
      post_capture: 5
      objects:
        - person
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      mask:
        - 0,0,1769,0,1885,0,1897,152,1669,165,1457,255,989,485,529,740,0,1083
    zones:  
      Vorgarten:
        coordinates: 0,1440,1737,1440,2560,1440,2560,0,1870,0,1878,163,1664,173,1555,219,1395,288,1156,411,788,609,704,650,63,1046,0,1079
  Garten:
    ffmpeg:
      global_args:
        - '-hide_banner'
        - '-loglevel'
        - info
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -rw_timeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      output_args:   

#        rtmp: -c:v libx264 -an -f flv -an
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy


      inputs:
#        - path: rtmp://10.0.0.17/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=xxxx
#        - path: rtmp://admin:[email protected]/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=xxxx
#          roles:
#            - detect
        - path: rtmp://admin:[email protected]/bcs/channel0_main.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=xxxx
#        - path: rtsp://admin:[email protected]:554//h264Preview_01_main
          roles:
            - detect
            - rtmp
#            - clips
#    width: 640
#    height: 480
    detect:
#      width: 2560
#      height: 1920
      width: 2048
      height: 1536
      fps: 5
    rtmp:
      enabled: True
#    clips:
#      enabled: true
#      pre_capture: 5
#      post_capture: 5
#      objects:
#        - person
#      required_zones: []
#      retain:
#        default: 10
#        objects:
#          person: 15    
    motion:
      mask:
        - 79,242,454,49,895,52,1249,55,1472,30,1916,251,1898,309,2048,340,2048,0,0,0,0,256,96,314,128,261
    zones:  
      hinten_links:
        coordinates: 2048,1536,2048,331,1901,307,1903,270,1944,268,1470,36,1239,52,1018,51,908,43,872,44,437,42,377,75,137,189,77,232,124,298,81,304,0,250,0,1536
    objects:
      track:
        - person
        - dog
  Gartenhaus:
    ffmpeg:
      global_args:
        - '-hide_banner'
        - '-loglevel'
        - info
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -rw_timeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      output_args:   
#        rtmp: -c:v libx264 -an -f flv -an
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy


      inputs:
#        - path: rtmp://10.0.0.19/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=xxx
#        - path: rtmp://admin:[email protected]/bcs/channel0_sub.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=xxx
#          roles:
#            - detect
        - path: rtmp://admin:[email protected]/bcs/channel0_main.bcs?token=sdasdasd&channel=0&stream=0&user=admin&password=xxx
          roles:
            - detect
            - rtmp
#            - clips
#    width: 640
#    height: 480
    detect:
      width: 2048
      height: 1536
      fps: 5
    rtmp:
      enabled: True
    clips:
      enabled: true
      pre_capture: 5
      post_capture: 5
      objects:
        - person
      required_zones: []
      retain:
        default: 10
        objects:
          person: 15
    motion:
      mask:
        - 2048,569,1670,497,1676,346,1714,328,1552,216,1162,36,897,98,427,241,426,265,356,313,398,339,159,365,60,387,0,388,0,0,2048,0
    zones:  
      hinten_rechts:
        coordinates: 2048,1536,2048,561,1662,495,1670,335,1546,209,1162,22,424,233,443,276,358,307,364,346,51,373,0,393,0,1536
    objects:
      track:
        - person
        - dog
    
objects:
  track:
    - person
    - cat
    - dog
    - car 
  filters:
    person:
      #min_area: 5000
      #max_area: 100000
      #min_score: 0.6
      threshold: 0.7
      
detectors:
  coral:
    type: edgetpu
    device: usb

I apologize for the many comments in my config, but I’ve already played around a bit…

greetings chris

Hello! Can someone share some hints how I can setup SSL?