Local realtime person detection for RTSP cameras

bremby · July 23, 2023, 5:31pm

Hi, I have 2 cameras that are properly working. Both of them have the VIDEOMAIN for RECORDING and VIDEOSUB for DETECTION

See the config

cameras:
 camera1:
  enabled: True
  
  ffmpeg:
    inputs:
    - path: rtsp://user::password@IPAddress1:88/videoMain
      hwaccel_args: preset-rpi-64-h264
      #input_args: preset-rtsp-generic
      input_args: preset-rtsp-restream
      roles:
       - record
   
    - path: rtsp://user::password@IPAddress1:88/videoSub
      hwaccel_args: preset-rpi-64-h264
      #input_args: preset-rtsp-generic
      input_args: preset-rtsp-restream
      roles:
       - detect
    output_args:
      record: preset-record-generic
      
  detect:
    enabled: True 
    width: 1280
    height: 720
    fps: 5

  record:
   enabled: True
   retain:
     days: 7
     mode: motion
   events:
     pre_capture: 15
     post_capture: 15
     retain:
       default: 5
       mode: active_objects
       objects:
        person: 5
     objects:
      - person

  snapshots:
    enabled: True
    timestamp: True
    bounding_box: True
    retain:
      default: 5
      objects:
        person: 5

 camera2:
  enabled: True
  
  ffmpeg:
    inputs:
    - path: rtsp://user::password@IPAddress2:88/videoMain
      hwaccel_args: preset-rpi-64-h264
      #input_args: preset-rtsp-generic
      input_args: preset-rtsp-restream
      roles:
       - record
   
    - path: rtsp://user:password@IPAddress2:88/videoSub
      hwaccel_args: preset-rpi-64-h264
      #input_args: preset-rtsp-generic
      input_args: preset-rtsp-restream
      roles:
       - detect

    output_args:
      record: preset-record-generic
      
  detect:
    enabled: True 
    width: 640
    height: 360
    fps: 5
  record:
   enabled: True
   retain:
     days: 7
     mode: motion
   events:
     pre_capture: 15
     post_capture: 15
     retain:
       default: 5
       mode: active_objects
       objects:
        person: 5
     objects:
      - person
  snapshots:
    enabled: True
    timestamp: True
    bounding_box: True
    retain:
      default: 5
      objects:
        person: 5
  rtmp:
    enabled: False

now I would like to switch to the go2rtc, but I don’t understand how to properly match the stream.
if I add a single entry it works

go2rtc:
  streams:
    camera1:
      - rtsp://user:password@camera1:88/videoMain

but how I can add the videosub? and how I can add the camera2?

this is wrong because as far I understood the name must be unique

`go2rtc:
  streams:
    camera1:
      - rtsp://user:password@IPAddress1:88/videoSub
    camera1:
      - rtsp://user:password@IPAddress1:88/videoMain

do I have to define different cameras? and arrange to split the detect and record across the main and the sub definition?
eg:

cameres:
  camera1_main:
     ......
     roles:
       - record
  camera1_sub:
     ......
      roles:
       - detect
  camera2_main:
     ......
      roles:
       - record
  camera2_sub:
     ......
      roles:
       - detect

tmjpugh · July 24, 2023, 8:04am

If sub is detection only you do not need to add to go2rtc. Just keep under ffmpeg with detect role as you currently have.

bremby · July 24, 2023, 8:25pm

thank yoy, i got it
i have another question, i see only the jsmepg option, what i need to do to add go2rtc or mse?

crzynik · July 25, 2023, 12:45am

bremby · August 25, 2023, 9:54am

@crzynik , I think that I have it

`go2rtc:
  streams:
    camera1_main:
      - rtsp://user:[email protected]:88/videoMain
    camera1_sub:
      - rtsp://user:[email protected]:88/videoSub
    camera2_main:
      - rtsp://user:[email protected]:88/videoMain
    camera2_sub:
      - rtsp://user:[email protected]:88/videoSub

  webrtc:
    candidates:
      - X.X.X.X:8555
      - stun:8555

rtmp:
  enabled: False`

this is the go2trc dashboard

if I click on the stream I see RTC

but in the frigate page I see only jsmpeg

this is the go2rtc config

api:
  origin: '*'
log:
  format: text
rtsp:
  default_query: mp4

I’m missing something?
do you have any suggestions on my config?

crzynik · August 25, 2023, 12:16pm

the name of the go2rtc streams don’t match the name of the camera

bremby · August 25, 2023, 6:23pm

OMG OMG , thank you so much @crzynik I didn’t really go it since you said it

I guess I should go for something like this, right?
As I have MAIN and SUB streams with different roles, I should also create different cameras

cameras:
 camera1_main:
  enabled: True

  ffmpeg:
    inputs:
    - path: rtsp://127.0.0.1:8554/camera1_main
      input_args: preset-rtsp-restream
      roles:
       - record
  record:
   enabled: True
   retain:
     days: 7
     mode: motion
   events:
     pre_capture: 15
     post_capture: 15
     retain:
       default: 5
       mode: active_objects
       objects:
        person: 5
     objects:
      - person

  snapshots:
    enabled: True
    timestamp: True
    bounding_box: True
    retain:
      default: 5
      objects:
        person: 5
        
  birdseye:
    enabled: True
    restream: False

 camera1_sub:
  enabled: True

  ffmpeg:
    inputs:
    - path: rtsp://127.0.0.1:8554/camera1_sub
      input_args: preset-rtsp-restream
      roles:
       - detect
    output_args:
      record: preset-record-generic

  detect:
    enabled: True
    width: 1280
    height: 720
    fps: 5

bremby · August 25, 2023, 10:35pm

I found the answer here use both primary stream and substream with go2rtc? · blakeblackshear/frigate · Discussion #5947 (github.com), it seem more easy

brpeterso · September 1, 2023, 4:36pm

I see there is an option in the Frigate card to hide the camera from UI. Is there an option to hide the camera from UI based on sensor/switch state? I have three indoor cameras that only turn on when the house is in Away/Night mode. Currently these cameras take up space when scrolling through the frigate card when I’m at home and it would be great if they disappeared from the UI during these times. I’m hoping I’m just missing something here, but I haven’t had any luck when searching. TIA.

Mind · September 8, 2023, 8:42am

Can you please show me the whole config within the yaml for the doorbird? Thanks!!

bremby · September 16, 2023, 5:07am

Hi,

I see this message in the log every 10 sec

2023-09-16 06:53:53.451374237 [2023-09-16 06:53:53] watchdog.camera1 ERROR : No new recording segments were created for camera1 in the last 120s. restarting the ffmpeg record process…
2023-09-16 06:53:53.451498541 [2023-09-16 06:53:53] watchdog.camera1 INFO : Terminating the existing ffmpeg process…
2023-09-16 06:53:53.451591478 [2023-09-16 06:53:53] watchdog.camera1 INFO : Waiting for ffmpeg to exit gracefully…

I made any mistakes in the config?

can you check my config

detectors:
coral:
type: edgetpu
device: usb

go2rtc:
streams:
camera1:
- rtsp://user:[email protected]:88/videoMain

webrtc:
candidates:
- 1.1.1.12:8555
- stun:8555

rtmp:
enabled: False

birdseye:
enabled: True
mode: continuous
#restream: True

objects:
track:

person

- car

bicycle

motorcycle

bird

cat

dog

ffmpeg:
hwaccel_args: preset-intel-qsv-h264

cameras:
camera1:
enabled: True

ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/camera1_main
#hwaccel_args: preset-intel-qsv-h264
#hwaccel_args: preset-rpi-64-h264
#input_args: preset-rtsp-generic
input_args: preset-rtsp-restream
roles:
- record
- path: rtsp://user:[email protected]:88/videoSub
  #hwaccel_args: preset-rpi-64-h264
  #input_args: preset-rtsp-generic
  input_args: preset-rtsp-restream
  roles:
   - detect
output_args:
  record: preset-record-generic
record:
enabled: True
retain:
days: 7
mode: motion
events:
pre_capture: 15
post_capture: 15
retain:
default: 5
mode: active_objects
objects:
person: 5
objects:
- person

snapshots:
enabled: True
timestamp: True
bounding_box: True
retain:
default: 5
objects:
person: 5

detect:
enabled: True
width: 1280
height: 720
fps: 5

here my camera settings

VLC stream

bremby · September 22, 2023, 8:31pm

Thank you the community support i got my errror, i was referring to the wrong camera name

0la1 · October 4, 2023, 11:06am

Hi everyone,

I am trying to get Frigate up and running and got stuck with errors. The log is:

2023-10-04 11:00:25.252235006  [2023-10-04 13:00:25] watchdog.Eingang               ERROR   : Ffmpeg process crashed unexpectedly for Eingang.
2023-10-04 11:00:25.257906533  [2023-10-04 13:00:25] watchdog.Eingang               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-10-04 11:00:25.258024327  [2023-10-04 13:00:25] ffmpeg.Eingang.detect          ERROR   : [rtsp @ 0x559bf2106c40] method OPTIONS failed: 401 Unauthorized
2023-10-04 11:00:25.258101239  [2023-10-04 13:00:25] ffmpeg.Eingang.detect          ERROR   : rtsp://*:*@IP:554/cam/realmonitor?channel=1&subtype=1: Server returned 401 Unauthorized (authorization failed)
2023-10-04 11:00:25.262051502  [2023-10-04 13:00:25] watchdog.Eingang               ERROR   : No new recording segments were created for Eingang in the last 120s. restarting the ffmpeg record process...
2023-10-04 11:00:25.262178804  [2023-10-04 13:00:25] watchdog.Eingang               INFO    : Terminating the existing ffmpeg process...
2023-10-04 11:00:25.262256908  [2023-10-04 13:00:25] watchdog.Eingang               INFO    : Waiting for ffmpeg to exit gracefully...
2023-10-04 11:00:25.293230352  [2023-10-04 13:00:25] frigate.video                  ERROR   : Eingang: Unable to read frames from ffmpeg process.
2023-10-04 11:00:25.293416767  [2023-10-04 13:00:25] frigate.video                  ERROR   : Eingang: ffmpeg process is not running. exiting capture thread...
2023-10-04 11:00:35.268555395  [2023-10-04 13:00:35] watchdog.Eingang               ERROR   : Ffmpeg process crashed unexpectedly for Eingang.
2023-10-04 11:00:35.268750736  [2023-10-04 13:00:35] watchdog.Eingang               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-10-04 11:00:35.269093626  [2023-10-04 13:00:35] ffmpeg.Eingang.detect          ERROR   : [rtsp @ 0x55d7808ccc40] method OPTIONS failed: 401 Unauthorized
2023-10-04 11:00:35.269340046  [2023-10-04 13:00:35] ffmpeg.Eingang.detect          ERROR   : rtsp://*:*@IP:554/cam/realmonitor?channel=1&subtype=1: Server returned 401 Unauthorized (authorization failed)
2023-10-04 11:00:35.276318604  [2023-10-04 13:00:35] watchdog.Eingang               ERROR   : No new recording segments were created for Eingang in the last 120s. restarting the ffmpeg record process...
2023-10-04 11:00:35.276449757  [2023-10-04 13:00:35] watchdog.Eingang               INFO    : Terminating the existing ffmpeg process...
2023-10-04 11:00:35.276496843  [2023-10-04 13:00:35] watchdog.Eingang               INFO    : Waiting for ffmpeg to exit gracefully...
2023-10-04 11:00:35.319631440  [2023-10-04 13:00:35] frigate.video                  ERROR   : Eingang: Unable to read frames from ffmpeg process.
2023-10-04 11:00:35.319721573  [2023-10-04 13:00:35] frigate.video                  ERROR   : Eingang: ffmpeg process is not running. exiting capture thread...

The frigate.yaml is:

mqtt:
  host: HAOS IP       #mqttserver ohne port
  user: mqtt                #mqttuser                                              
  password: PASSWORD           #mqttpassword
database:
  path: /config/frigate.db       

detectors:
  coral:
    type: edgetpu
    device: usb
#  cpu1:
#    type: cpu
#    num_threads: 3

cameras:
  Eingang:                                                            #Kameraname
    ffmpeg:
      inputs:
        - path: rtsp://USER:PASSWORD@IP:554/cam/realmonitor?channel=1&subtype=0    #RTSP Stream High Res
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://USER:PASSWORD@IP:554/cam/realmonitor?channel=1&subtype=1          #RTPS Stream Low Res
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      enable: True                                                        
      width: 640
      height: 480
      fps: 5
    objects:
      track:
        - person
        - cat
        - dog 
    snapshots:
      enabled: True
      timestamp: False
      bounding_box: True
      retain:
        default: 30
    record:
      enabled: True
      events:
        retain:
          default: 14

My camera stream is:

I can view the stream via VLC player withouth any further authentification, so the RTSP link works. I have also disabled any motion detection on the camera.

Does someone has a clue?

crzynik · October 4, 2023, 1:08pm

the camera might be UDP only, that or the password is being entered incorrectly in frigate (nothing should be url encoded)

0la1 · October 4, 2023, 1:25pm

Hi, I have had the camera configured in WebRTC with the same link and it is working. Also I veryfied the link (incl. user and password) with VLC which instantly connects.
The camera logs indeed show that the login is blocked, but how can the exact same link connect via VLC and not via frigate?

danbutter · October 4, 2023, 2:00pm

Limit of number of logged in users?

0la1 · October 4, 2023, 2:02pm

There is only one user logged in at a time.

0la1 · October 4, 2023, 3:12pm

I thought I will give it another go, with a bare minimum config.

mqtt:
  enabled: False

cameras:
  Eingang: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://USER:PASSWORD@IP:554/cam/realmonitor?channel=1&subtype=0 # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 640 # <---- update for your camera's resolution
      height: 480 # <---- update for your camera's resolution

s6-rc: info: service s6rc-fdholder: starting
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service s6rc-fdholder successfully started
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service log-prepare: starting
s6-rc: info: service log-prepare successfully started
s6-rc: info: service nginx-log: starting
s6-rc: info: service go2rtc-log: starting
s6-rc: info: service frigate-log: starting
s6-rc: info: service nginx-log successfully started
s6-rc: info: service go2rtc-log successfully started
s6-rc: info: service go2rtc: starting
s6-rc: info: service frigate-log successfully started
s6-rc: info: service go2rtc successfully started
s6-rc: info: service go2rtc-healthcheck: starting
s6-rc: info: service frigate: starting
s6-rc: info: service go2rtc-healthcheck successfully started
s6-rc: info: service frigate successfully started
s6-rc: info: service nginx: starting
s6-rc: info: service nginx successfully started
s6-rc: info: service legacy-services: starting
2023-10-04 15:07:29.181820949  [INFO] Preparing go2rtc config...
2023-10-04 15:07:29.182304873  [INFO] Starting Frigate...
2023-10-04 15:07:29.183198890  [INFO] Starting NGINX...
s6-rc: info: service legacy-services successfully started
2023-10-04 15:07:29.219446889  [INFO] Got IP address from supervisor: 192.168.20.10
2023-10-04 15:07:29.252151148  [INFO] Got WebRTC port from supervisor: 8555
2023-10-04 15:07:29.464616081  [INFO] Starting go2rtc...
2023-10-04 15:07:29.550105779  17:07:29.550 INF go2rtc version 1.2.0 linux/amd64
2023-10-04 15:07:29.550318647  17:07:29.550 INF [api] listen addr=:1984
2023-10-04 15:07:29.550497632  17:07:29.550 INF [rtsp] listen addr=:8554
2023-10-04 15:07:29.550645796  17:07:29.550 INF [srtp] listen addr=:8443
2023-10-04 15:07:29.550760151  17:07:29.550 INF [webrtc] listen addr=:8555
2023-10-04 15:07:30.239732211  [2023-10-04 17:07:30] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-10-04 15:07:30.249851306  [2023-10-04 17:07:30] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-10-04 15:07:30.250552766  [2023-10-04 17:07:30] peewee_migrate                 INFO    : Starting migrations
2023-10-04 15:07:30.254130669  [2023-10-04 17:07:30] peewee_migrate                 INFO    : There is nothing to migrate
2023-10-04 15:07:30.263742074  [2023-10-04 17:07:30] detector.cpu                   INFO    : Starting detection process: 443
2023-10-04 15:07:30.264079800  [2023-10-04 17:07:30] frigate.app                    INFO    : Output process started: 445
2023-10-04 15:07:30.266067681  [2023-10-04 17:07:30] frigate.detectors              WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
2023-10-04 15:07:30.267765004  [2023-10-04 17:07:30] frigate.app                    INFO    : Camera processor started for Eingang: 448
2023-10-04 15:07:30.271605976  [2023-10-04 17:07:30] frigate.app                    INFO    : Capture process started for Eingang: 450
2023-10-04 15:07:39.183326481  [INFO] Starting go2rtc healthcheck service...

Now it is stuck at the go2rtc healthcheck and nothing happens further.

xbmcnut · October 17, 2023, 12:08am

Anyone seen this Frigate behaviour on Firefox 118.01 and recent versions? All the cameras flicker when viewing the cameras page. It’s OK when on Birdseye view. Been doing it for about 6~8 weeks. Doesn’t do it on Chrome or Edge. Frigate 0.12.1-367d724 & go2rtc version 1.8.1
20231017-Frigate-Firefox

danbutter · October 24, 2023, 10:08pm

This has always happened in firefox for me, but not frigate specifically. Any home assistant camera. I am on 118.0.2 and everything is smoooth like it never has been before. Gotta admit I don’t use firefox as much as I used to, but no. No flickering here.