Local realtime person detection for RTSP cameras

retained mqtt message?

1 Like

Oh baby! Loving the new Birdseye feature!

Here is my full working config for reference. Note that I did not do full time recording before or after 2x. I use my Ubiquiti UDMP for long term storage of recordings. I have six cameras being pulled into Frigate but just have the config for one in the code below. I use a NUC with Corals.

detectors:
  coral_pci:
    type: edgetpu
    device: pci
  coral_usb:
    type: edgetpu
    device: usb

mqtt:
  host: core-mosquitto
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: XXXXXXX
  password: XXXXXXX
birdseye:
  enabled: True
  width: 1280
  height: 720
  quality: 8
  mode: objects

ffmpeg:
  global_args:
    -hide_banner
    -loglevel warning
  input_args:
    -avoid_negative_ts make_zero
    -fflags +genpts+discardcorrupt
    -rtsp_transport tcp
    -stimeout 5000000
    -use_wallclock_as_timestamps 1
  output_args:
    detect:
      -f rawvideo 
      -pix_fmt yuv420p
    record: 
      -f segment
      -segment_time 60
      -segment_format mp4
      -reset_timestamps 1
      -strftime 1 -c copy -an
    rtmp:
      -c copy
      -f flv
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

record:
  enabled: true
  retain_days: 0
  events:
    retain:
      default: 30

snapshots:
  retain:
    default: 10
    objects:
      person: 15
      
cameras:
####FRONT DOOR####  
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://XXXXX
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 1600
      height: 1200
      fps: 10
    motion:
      mask:
        - 0,411,0,720,670,664,1242,658,1298,559,1600,540,1600,0,0,0
    zones:
      patio:
        coordinates: 1600,1200,1264,934,1285,743,1086,709,966,743,737,763,390,803,0,1116,0,1200
    snapshots:
      enabled: true
      bounding_box: true
      crop: false
      timestamp: false

4 Likes

New image pushed to docker (stable-amd64) yesterday, container now cannot start. Iā€™m looking through docs saw there were some breaking changes so I made the adjustments in my config. Still no go.

Looking at the container logs I see its having issues hitting run.sh:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
ifelse: fatal: unable to exec /run.sh: No such file or directory
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Iā€™m running it with docker-compose on a separate box from my HA. It had been running successfully for months. Absolutely love this project and greatly appreciate the work and effort that goes into it.


I cant watch clips. On two differwnt HA servers and through app and webbrowser pwa mobile. After 2.0 update. Was working perfectly before update

I had the same error, for me a HA restart fixed it.

You probably have a warning message in your logs about not having the record role assigned to an input with recording enabled. You need add record where you removed clips.

1 Like

Top of config:

detectors:
  coral_pci:
    type: edgetpu
    device: pci


    
ffmpeg:
  # Optional: global ffmpeg args (default: shown below)
  global_args: -hide_banner -loglevel warning
  # Optional: global hwaccel args (default: shown below)
  # NOTE: See hardware acceleration docs for your specific device
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128
  # Optional: global input args (default: shown below)
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1
  # Optional: global output args
  output_args:
    # Optional: output args for detect streams (default: shown below)
    detect: -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
    #longloading-record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -an
    #longloading-clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -an

    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    # Optional: output args for clips streams (default: shown below)
    #clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an

and Each cam have same setup on one server 11cams other server 11cams

  CAM_garaz:

    ffmpeg:
      inputs:
        - path: rtsp://xxx:xxxx@192xxxx:554/cam/realmonitor?channel=1&subtype=0&authbasic=xxxx
          roles:
            - detect
            #- clips


    zones:
      zone_0:
        coordinates: 0,1080,1388,1080,1920,1080,1920,660,1792,476,1412,110,581,47,309,161,0,386
    detect:
      enabled: True
      max_disappeared: 25
      width: 1920
      height: 1080
      fps: 5
      #best_image_timeout: 60

    record:
      enabled: True
      retain_days: 0
      events:
        post_capture: 10
        pre_capture: 5
        objects:
          - person
        retain:
          default: 5
    
    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: False
      bounding_box: True
      crop: False
      retain:
        default: 5
        #objects:
          #person: 15
    mqtt:
      enabled: False
      timestamp: True
      bounding_box: True
      crop: True
      height: 270
    objects:
      track:
        - person
      filters:
        person:
          min_area: 1000
          max_area: 100000
          min_score: 0.5
          threshold: 0.77

@blakeblackshear

same after restart

Some other settings are broking on update?

[2021-10-08 07:28:42] frigate.video                  INFO    : CAM1: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
[2021-10-08 07:28:42] frigate.video                  INFO    : CAM1: ffmpeg process is not running. exiting capture thread...
[2021-10-08 07:28:52] watchdog.CAM1                  ERROR   : FFMPEG process crashed unexpectedly for CAM1.
[2021-10-08 07:28:52] watchdog.CAM1                  ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2021-10-08 07:28:52] watchdog.CAM1                  ERROR   : You may have invalid args defined for this camera.

@blakeblackshear

Sorry, I also update and got errosā€¦ and I donā€™t know where to startā€¦

cameras -> schuurachter -> ffmpeg -> inputs -> 0 -> roles -> 1
  value is not a valid enumeration member; permitted: 'record', 'rtmp', 'detect' (type=type_error.enum; enum_values=[<CameraRoleEnum.record: 'record'>, <CameraRoleEnum.rtmp: 'rtmp'>, <CameraRoleEnum.detect: 'detect'>])
cameras -> schuurachter -> clips
  extra fields not permitted (type=value_error.extra)
cameras -> schuurachter -> fps
  extra fields not permitted (type=value_error.extra)
cameras -> schuurachter -> height
  extra fields not permitted (type=value_error.extra)
cameras -> schuurachter -> width
  extra fields not permitted (type=value_error.extra)
cameras -> parkeerplaats -> ffmpeg -> inputs -> 0 -> roles -> 1
  value is not a valid enumeration member; permitted: 'record', 'rtmp', 'detect' (type=type_error.enum; enum_values=[<CameraRoleEnum.record: 'record'>, <CameraRoleEnum.rtmp: 'rtmp'>, <CameraRoleEnum.detect: 'detect'>])
cameras -> parkeerplaats -> clips
  extra fields not permitted (type=value_error.extra)
cameras -> parkeerplaats -> fps
  extra fields not permitted (type=value_error.extra)
cameras -> parkeerplaats -> height
  extra fields not permitted (type=value_error.extra)
cameras -> parkeerplaats -> width
  extra fields not permitted (type=value_error.extra)
cameras -> voortuin -> ffmpeg -> inputs -> 0 -> roles -> 1
  value is not a valid enumeration member; permitted: 'record', 'rtmp', 'detect' (type=type_error.enum; enum_values=[<CameraRoleEnum.record: 'record'>, <CameraRoleEnum.rtmp: 'rtmp'>, <CameraRoleEnum.detect: 'detect'>])
cameras -> voortuin -> clips
  extra fields not permitted (type=value_error.extra)
cameras -> voortuin -> fps
  extra fields not permitted (type=value_error.extra)
cameras -> voortuin -> height
  extra fields not permitted (type=value_error.extra)
cameras -> voortuin -> width
  extra fields not permitted (type=value_error.extra)
cameras -> terras -> ffmpeg -> inputs -> 0 -> roles -> 1
  value is not a valid enumeration member; permitted: 'record', 'rtmp', 'detect' (type=type_error.enum; enum_values=[<CameraRoleEnum.record: 'record'>, <CameraRoleEnum.rtmp: 'rtmp'>, <CameraRoleEnum.detect: 'detect'>])
cameras -> terras -> clips
  extra fields not permitted (type=value_error.extra)
cameras -> terras -> fps
  extra fields not permitted (type=value_error.extra)
cameras -> terras -> height
  extra fields not permitted (type=value_error.extra)
cameras -> terras -> width
  extra fields not permitted (type=value_error.extra)
[cmd] python3 exited 1
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.

those are easy, you need to rewrite your config, move as per documentation. I was also shocked at first but after 30minutes all fixed.
Remove all mention on clips.
Add record True with retention false (if you donā€™t use 24/7 recording) and move your settings frrom clips to record

clips is no longer a valid role and should be removed

FPS not needed

This is an example of 1-5 cams:


cameras:
# FRONT Camera ########################################################################################################
  voordeur:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
            - clips
#            - rtmp
    width: 2560
    height: 1440
    fps: 6
    snapshots:
      enabled: true
      crop: false
      bounding_box: True
    objects:
      track:
        - person
        - dog
        - cat
    motion:
      mask:
        - '1090,81,1216,0,1382,34,1532,99,1518,259,1426,363,1123,247'
        - '1766,498,1753,669,1778,722,1829,773,1863,825,1906,884,1924,955,1953,1065,2237,1169,2255,1128,2295,1046,2338,946,2378,841,2410,743,2215,647,2190,582,2138,512,2010,402,1859,351'
    clips:
      # Required: enables clips for the camera (default: shown below)
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: Number of seconds before the event to include in the clips (default: shown below)
      pre_capture: 2
      # Optional: Number of seconds after the event to include in the clips (default: shown below)
      post_capture: 2
      # Optional: Objects to save clips for. (default: all tracked objects)
      objects:
        - person
        - dog
        - cat
      # Optional: Restrict clips to objects that entered any of the listed zones (default: no required zones)
      required_zones: []
      # Optional: Camera override for retention settings (default: global values)
      retain:
        # Required: Default retention days (default: shown below)
        default: 10
        # Optional: Per object retention days
        objects:
          person: 200
          dog: 20
          cat: 200

What should I do with the ā€œclipsā€ section then?

Frigate Integration will not connect after update and followed the new config based on item 3 in new document.

`Frigate
URL you use to access Frigate (ie. http://frigate:5000/)

If you are using HassOS with the addon, the URL should be http://ccab4aaf-frigate:5000/

Home Assistant needs access to port 5000 (api) and 1935 (rtmp) for all features.

The integration will setup sensors, cameras, and media browser functionality.

Sensors:

Stats to monitor frigate performance
Object counts for all zones and cameras
Cameras:

Cameras for image of the last detected object for each camera
Camera entities with stream support (requires RTMP)
Media Browser:

Rich UI with thumbnails for browsing event clips
Rich UI for browsing 24/7 recordings by month, day, camera, time
API:

Notification API with public facing endpoints for images in notifications
Failed to connect
 
URL
http://ccab4aaf-frigate:5000
`
New  Config.yml Frigate

URL you use to access Frigate (ie. `http://frigate:5000/`)

If you are using HassOS with the addon, the URL should be `http://ccab4aaf-frigate:5000/`

Home Assistant needs access to port 5000 (api) and 1935 (rtmp) for all features.

The integration will setup sensors, cameras, and media browser functionality.

Sensors:

* Stats to monitor frigate performance
* Object counts for all zones and cameras

Cameras:

* Cameras for image of the last detected object for each camera
* Camera entities with stream support (requires RTMP)

Media Browser:

* Rich UI with thumbnails for browsing event clips
* Rich UI for browsing 24/7 recordings by month, day, camera, time

API:

* Notification API with public facing endpoints for images in notifications

Failed to connect

URL

SUBMIT
```
mqtt:
  host: 192.168.0.74
  topic_prefix: frigate
  client_id: frigate
  user: mikey
  password: xxxxxx


detectors:
  cpu1:
    type: cpu
    num_threads: 3

cameras:
    camera1:front_driveway:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/h264Preview_01_sub
          roles:
            - detect
    width: 1280
    height: 720
````
 Any idea where i am  going wrong as  i am trying to setup agin in HASSO
  CAM_obyvak:

    ffmpeg:
      inputs:
        - path: rtsp://xxxx:xxxx@xxxx:554/cam/realmonitor?channel=1&subtype=0&authbasic=xxxxx
          roles:
            - detect



    zones:
      zone_0:
        coordinates: 0,1080,1429,1080,1855,583,1876,490,1309,380,1333,0,0,0
    detect:
      enabled: True
      max_disappeared: 25
      width: 1920
      height: 1080
      fps: 5
      #best_image_timeout: 60

    record:
      enabled: False
      retain_days: 0
      events:
        post_capture: 10
        pre_capture: 5
        objects:
          - person
        retain:
          default: 5
    rtmp:
      enabled: False
    snapshots:
      enabled: True
      timestamp: False
      bounding_box: True
      crop: False
      retain:
        default: 5
        #objects:
          #person: 15
    mqtt:
      enabled: False
      timestamp: True
      bounding_box: True
      crop: True
      height: 270
    objects:
      track:
        - person
      filters:
        person:
          min_area: 1000
          max_area: 100000
          min_score: 0.5
          threshold: 0.77

No 24/7 recording

I totally do not get this:

cameras:
# FRONT Camera ########################################################################################################
  voordeur:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
            #- clips
#            - rtmp
    detect:
      width: 2560
      height: 1440
#    fps: 6
    snapshots:
      enabled: true
      crop: false
      bounding_box: True
    objects:
      track:
        - person
        - dog
        - cat
    motion:
      mask:
        - '1090,81,1216,0,1382,34,1532,99,1518,259,1426,363,1123,247'
        - '1766,498,1753,669,1778,722,1829,773,1863,825,1906,884,1924,955,1953,1065,2237,1169,2255,1128,2295,1046,2338,946,2378,841,2410,743,2215,647,2190,582,2138,512,2010,402,1859,351'
#    clips:
      # Required: enables clips for the camera (default: shown below)
      # This value can be set via MQTT and will be updated in startup based on retained value
#      enabled: True
      # Optional: Number of seconds before the event to include in the clips (default: shown below)
#      pre_capture: 2
      # Optional: Number of seconds after the event to include in the clips (default: shown below)
#      post_capture: 2
      # Optional: Objects to save clips for. (default: all tracked objects)
#      objects:
#        - person
#        - dog
#        - cat
      # Optional: Restrict clips to objects that entered any of the listed zones (default: no required zones)
#      required_zones: []
      # Optional: Camera override for retention settings (default: global values)
#      retain:
        # Required: Default retention days (default: shown below)
#        default: 10
        # Optional: Per object retention days
#        objects:
#          person: 200
#          dog: 20
#          cat: 200

with this errors:

[services.d] starting services


[services.d] done.


[2021-10-08 16:15:28] frigate.app                    INFO    : Starting Frigate (0.9.1-800f33e)


Error parsing config: 26 validation errors for FrigateConfig


ffmpeg -> output_args -> clips


  extra fields not permitted (type=value_error.extra)


cameras -> voordeur -> ffmpeg -> inputs -> 0 -> roles -> 1


  value is not a valid enumeration member; permitted: 'record', 'rtmp', 'detect' (type=type_error.enum; enum_values=[<CameraRoleEnum.record: 'record'>, <CameraRoleEnum.rtmp: 'rtmp'>, <CameraRoleEnum.detect: 'detect'>])


cameras -> voordeur -> clips


  extra fields not permitted (type=value_error.extra)


cameras -> voordeur -> fps


  extra fields not permitted (type=value_error.extra)


cameras -> voordeur -> height


  extra fields not permitted (type=value_error.extra)


cameras -> voordeur -> width


  extra fields not permitted (type=value_error.extra)

whatā€™s wrong there?

did you save the config before restart? maybe save and restart again

Its still saying you have clips as a role and heig tht/width/fps under FFMPEG section but above looks correct so I thinking it not saved before you try restart

I really didā€¦ Not

Another question. How do I delete all recordings? and database?

If its a one time thing just delete the db file in config and remove all the media files?