Local realtime person detection for RTSP cameras

Did you find out how to solve it?

In my case it worked till a month ago. Now I see same error as you.

I tried rebuild the container with just mqtt in config.yml but still no way to make it work again.

With automation and mqtt its possible

How can be possible use multiple cameras?

I have a NUC, with home assistant in proxmox, supervised.

I setup a camera, now i need to add more than one, so how can I do that?

I have tried to add another in frigate.yaml, but see only the last camera, and not all camera that i setup

I have a Reolink RLC-410-5MP that works

Now i have:
RLC-520A
RLC-822A ( this is h265 so maybe is hard to config and see live, but rtmp in vlc works, in browser is green )
RLC-820A ( this is h265 so maybe is hard to config and see live, but rtmp in vlc works, in browser is green )

I donā€™t understand how to setup these cameras, and also understand how to see live preview, ( I see that i canā€™t see h265 preview right? so i need to setup h264, and i have the link h264 and h265. )
I wish if I canā€™t manage h265 for live preview, have h264 preview, but clips record in h265 full size.

I have an old Dlink camera which uses MJPEG over HTTP. I tried using Frigate but it canā€™t read from the camera
Details are on Frigate Github

If the feed is green when viewing in the browser, Iā€™m going to guess you have the dimensions of the stream wrong in frigate.yaml. Look at the properties of the stream in vlc to get the correct dimensions and then update the frigate config.

Hi,

in first thanks to reply.

This is my /config/frigate.yml about my Reolink RLC-410-5MP
note. with this configuration, this camera works, only when i restart Frigate, i donā€™t understand why i have this warning:

 * Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
frigate.app                    WARNING : Camera ingresso has clips assigned to an input, but clips is not enabled.
frigate.app                    WARNING : Camera ingresso has record assigned to an input, but record is not enabled.
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
frigate.mqtt                   INFO    : Turning on clips for ingresso via mqtt
frigate.mqtt                   INFO    : Turning on snapshots for ingresso via mqtt
detector.cpu1                  INFO    : Starting detection process: 37
detector.cpu2                  INFO    : Starting detection process: 39
frigate.app                    INFO    : Camera processor started for ingresso: 46
detector.cpu3                  INFO    : Starting detection process: 41
frigate.app                    INFO    : Capture process started for ingresso: 47
detector.cpu4                  INFO    : Starting detection process: 42
########################################## MQTT SERVER 
mqtt:
  host: 192.168.178.36
  user: mqtt_user
  password: mqtt_user
########################################## CAMERA INGRESSO
cameras:
  ingresso:
    detect:
      enabled: True
    ffmpeg:
      hwaccel_args:
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      output_args:
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
      inputs:
          - path: rtmp://192.168.178.35/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=mypass
            roles:
              - detect
              - clips
              - record
              - rtmp
    width: 2560
    height: 1920
    fps: 3
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu
#motion:
#  mask:
#    - 0,1920,2560,1920,2193,599,2095,524,1909,396,1724,269,1423,260,704,442,230,599,0,742

Now i bought other 3 cameras and i donā€™t understand how to add them in this yaml, because i have tried without success.

In first i have tried to add Reolink RLC-822A that is 4k with optic zoom and h265.

1 - Recognize always the last camera in this yaml. ( so recognize frigate in this yaml, only balcone cameras, and not ingresso camera.
2 - I have tried to copy and paste the config about my Reolink RLC-410-5MP and change the ip address, the url and resolution, but i have the little resolution, not 4k, and record in low resolution, so useless.
If i put the rtsp h265, green screen in frigate.
3 - I wish to have record audio and video at full size, if is not possible have full size preview, i use h264, to avoid any encode on the fly and waste energy and cpu. ( with coral i guess isnā€™t possible use it to encode on the fly, right ) ?
4 - If is possible have command zoom in and out on Home assistant? i guess isnā€™t possible on frigate, but maybe is possible in preview window about this camera? but maybe have a no sense, because resolution of it is possible only at low resolution h264 ? Maybe was better get a 1080p h264 camera. Seems that h265 is a mess.

########################################## MQTT SERVER 
mqtt:
  host: 192.168.178.36
  user: mqtt_user
  password: mqtt_user
########################################## CAMERA INGRESSO
cameras:
  ingresso:
    detect:
      enabled: True
    ffmpeg:
      hwaccel_args:
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      output_args:
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
      inputs:
          - path: rtmp://192.168.178.35/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=mypass
#global_args: -analyzeduration 1000M -probesize 1000M
            roles:
              - detect
              - clips
              - record
              - rtmp
    width: 2560
    height: 1920
    fps: 3
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu
motion:
  mask:
    - 0,1920,2560,1920,2193,599,2095,524,1909,396,1724,269,1423,260,704,442,230,599,0,742
########################################## CAMERA BALCONE GRANDE
cameras:
  balcone:
    detect:
      enabled: True
    ffmpeg:
      hwaccel_args:
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      output_args:
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
      inputs:
          - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
            roles:
              - detect
              - clips
              - record
              - rtmp
    width: 640
    height: 360
    fps: 3
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu

I was not able to see h265 cameras, so you need lower.
And the last yaml is wrong, the detector should be only one on the top, read docs.
Hereā€™s mine:

#MQTT frigate/back/detect/set   ON/OFF to turn on or off tensorflow
database:
  path: /config/frigate.db
  
detectors:
  coral_pci:
    type: edgetpu
    device: pci


    
ffmpeg:
  # Optional: global ffmpeg args (default: shown below)
  global_args: -hide_banner -loglevel warning
  # Optional: global hwaccel args (default: shown below)
  # NOTE: See hardware acceleration docs for your specific device
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128
  # Optional: global input args (default: shown below)
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1
  # Optional: global output args
  output_args:
    # Optional: output args for detect streams (default: shown below)
    detect: -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
    #longloading-record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -an
    #longloading-clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -an

    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    # Optional: output args for clips streams (default: shown below)
    clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an


    
mqtt:
  # Required: host name
  host: xxxxxxx
  # Optional: port (default: shown below)
  port: zzzz
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate
  # Optional: user
  user: xxxxxxxxxx
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  password: xxxxxxxxxxx
  # Optional: interval in seconds for publishing stats (default: shown below)
  stats_interval: 60


cameras:

  CAM1:

    ffmpeg:
      inputs:
        - path: rtsp://xxx:xxxxx@xxxxxxx:554/cam/realmonitor?channel=1&subtype=0&authbasic=xxxxxxxxx
          roles:
            - detect
            - clips

    width: 1920
    height: 1080
    fps: 5
    best_image_timeout: 60
    motion:
      mask:
        - 1634,262,1744,890,1792,1045,1370,1080,963,994,26,982,22,1080,1920,1080,1920,0,343,0,457,226
    zones:
      zone_0:
        coordinates: 0,545,0,955,1266,996,1843,864,1750,156,1240,57,343,24,24,33
    detect:
      enabled: True
      max_disappeared: 25

    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
      objects:
        - person
      retain:
        default: 14
        #objects:
          #person: 15
    record:
      enabled: False
      retain_days: 30

    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: False
      retain:
        default: 14
        #objects:
          #person: 15

    mqtt:
      enabled: False
      timestamp: True
      bounding_box: True
      crop: True
      height: 270

    objects:
      track:
        - person
        - cat
      filters:
        person:
          min_area: 5000
          max_area: 100000
          min_score: 0.5
          threshold: 0.77
        cat:
          min_area: 1000
          max_area: 40000
          min_score: 0.5
          threshold: 0.77
  CAM2:

    ffmpeg:
      inputs:
        - path: rtsp://xxx:xxxxx@xxxxx:554/cam/realmonitor?channel=1&subtype=0&authbasic=xxxxxxx
          roles:
            - detect
            - clips

    width: 1920
    height: 1080
    fps: 5
    best_image_timeout: 60
    motion:
      mask:
        - 142,0,1920,0,1920,1051,1587,992,1126,870,1142,746,1209,646,1477,63,673,95,350,126,0,295,0,0
        - 742,221,1061,264,1063,358,752,652,656,602,744,504,886,421,697,380,650,252
    zones:
      zone_0:
        coordinates: 0,1080,955,1080,1920,1080,1124,864,1422,211,874,106,630,118,402,158,0,284
    detect:
      enabled: True
      max_disappeared: 25

    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
      objects:
        - person
      retain:
        default: 14
        #objects:
          #person: 15
    record:
      enabled: False
      retain_days: 30

    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: False
      retain:
        default: 14
        #objects:
          #person: 15

    mqtt:
      enabled: False
      timestamp: True
      bounding_box: True
      crop: True
      height: 270

    objects:
      track:
        - person
        - car
        - cat
      filters:
        person:
          min_area: 1000
          max_area: 100000
          min_score: 0.5
          threshold: 0.77
        cat:
          min_area: 1000
          max_area: 40000
          min_score: 0.5
          threshold: 0.77

Nice, works and now i can see 2 cameras, iā€™ll try to setup others.

Anyway how can i record in full size 4k?

And why h265 isnā€™t possibile in preview?

-Anyway how can i record in full size 4k?

Just change the resolution? I dont have 4k so iā€™m not sure.
from
width: 1920
height: 1080
to
width: 3840
height: 2160
-And why h265 isnā€™t possibile in preview?
No idea, maybe in future will be ok.

If i change resolution with an rtsp h264 low resolution, i have a preview:


So my question is:

How can we manage h265 camera to have preview in h264 and record in full size 4k ?

Show settings from camera interface

Balcone_grande is the cameras with green screen because we use an h264 stream from camera that is 640x360 and we setup 3840x2160

If i use the 4k stream rtsp, green screen anyway with 3840x2160 because maybe is h265 the rtsp 4k
264

########################################## MQTT SERVER 
mqtt:
  host: 192.168.178.36
  user: mqtt_user
  password: mqtt_user

detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu

cameras:

  ingresso:

    ffmpeg:
      hwaccel_args:
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      output_args:
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

      inputs:
        - path: rtmp://192.168.178.35/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=mypass
          roles:
            - detect
            - clips
            - record
            - rtmp

    width: 2560
    height: 1920
    fps: 3
    detect:
      enabled: True

    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
      
  balcone_grande:

    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
          roles:
            - detect
            - clips
            - record
            - rtmp
    width: 3840
    height: 2160
    fps: 3
    detect:
      enabled: True
      
    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5

i dont see camera settings with 4k in camera UI.
You need to set the source for 4k and h264


265

########################################## MQTT SERVER 
mqtt:
  host: 192.168.178.36
  user: mqtt_user
  password: mqtt_user
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu

cameras:

  ingresso:

    ffmpeg:
      hwaccel_args:
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      output_args:
#        rtmp: -c copy -f flv
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

      inputs:
        - path: rtmp://192.168.178.35/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=mypass
          roles:
            - detect
            - clips
            - record
            - rtmp

    width: 2560
    height: 1920
    fps: 3
    detect:
      enabled: True

    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
      
  balcone_grande:

    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h265Preview_01_main
          roles:
            - detect
            - clips
            - record
            - rtmp
    width: 3840
    height: 2160
    fps: 3
    detect:
      enabled: True
      
    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5

I see h265 in camera UI, can you change to h264?

Why you have double slashes between 554 and h265 ? put only one

And you have wrong yaml configs still. Take my as template and rewrite it.

Done, but resolution of h264 is low in preview ( and maybe i can accept it, but record also at low resolution, so useless )
264

########################################## MQTT SERVER 
mqtt:
  host: 192.168.178.36
  user: mqtt_user
  password: mqtt_user
detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu

cameras:

  ingresso:

    ffmpeg:
      hwaccel_args:
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
      output_args:
#        rtmp: -c copy -f flv
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy

      inputs:
        - path: rtmp://192.168.178.35/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=mypass
          roles:
            - detect
            - clips
            - record
            - rtmp

    width: 2560
    height: 1920
    fps: 3
    detect:
      enabled: True

    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
      
  balcone_grande:

    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
          roles:
            - detect
            - clips
            - record
            - rtmp
    width: 640
    height: 360
    fps: 3
    detect:
      enabled: True
      
    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5

You cant make 4k with h264???

Of course not, not exist 4k resolution h264 for reolink.

hmm
From the perspective of resolution, there are 5 H.264 IP camera types: 720p, 1080p, 4MP, 5MP and 4K cameras