Local realtime person detection for RTSP cameras

pcwii · October 9, 2021, 2:13am

Oh (light bulb turns on between my ears). All i need to do is add my second camera to my existing config.yml and it will utilize the coral tpu already configured for that container. I was certainly over thinking this. Thanks for the quick response!

amirite · October 9, 2021, 6:13am

Hey what is the best order of operations for this update? I read about the breaking changes and changed my config, but how should I be updating each part of an HA install. I’m thinking I’d:

Stop the Frigate Add-on
Update the addon
Update & save the Config file
Update the Integration via HACS
Restart Home Assistant and start the Addon back up

Does that sound right?

EDIT: Everything worked perfectly after these steps, expect the Coral TPU doesn’t seem to auto-detect anymore so I had to add that default config entry to my setup as well. Everything else worked great

Klagio · October 9, 2021, 2:51pm

I don’t have HACS, since I have an installation under proxmox, not sure how to install it … so which is the version of the custom_component 2.1.0?

amirite · October 9, 2021, 2:53pm

I just happen to use HACS, which has it’s own updater for integrations. I believe this is a required step for this update though. You’d have to just pull the latest changes from the repo in your setup I think?

mcpaha · October 9, 2021, 6:28pm

Not sure if this is right thread but wasn’t able to find correct thread for lovelace frigate card…

Any idea why only one camera clips are visible?

xbmcnut · October 10, 2021, 6:11am

For anyone trying to migrate over to the new formatting, here is my working config with a lot of the settings moved to global defaults as all six cameras have the same stream settings. I have listed the config for one Hikvision camera and one Dahua one; both utilising the 3rd stream and 1280 x 720, h.264, 6fps. HA is running on an i7 NUC as is Frigate (HA add-on). Not an expert but working for me. FYI, CPU is averaging 33% with this 6 camera config. With only 4 cameras, it was sitting at about 21%.

mqtt:
  host: <nasip)

detectors:
  coral:
    type: edgetpu
    device: usb

birdseye:
  enabled: True
  width: 1280  
  height: 720
  quality: 8
  mode: objects   

ffmpeg:
  hwaccel_args: #Intel NUC < v10
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p    

detect:
  width: 1280 # <---- update for your camera's resolution
  height: 720 # <---- update for your camera's resolution
  fps: 6

objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person

record: # <----- Enable recording
  enabled: True
  retain_days: 0
  events:
    max_seconds: 300 # default value
    pre_capture: 2 # default = 5
    post_capture: 5
    objects:
      - person
    required_zones: []
    retain:
      default: 5 # default = 10

snapshots: # <----- Enable snapshots
  enabled: True
  bounding_box: True 
  # Optional: crop the snapshot (default: shown below)
  crop: False
  # Optional: height to resize the snapshot to (default: original size)
  #height:    

cameras:
  driveway_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://username:password@<hikvisionip>:554/Streaming/Channels/103 #(3rd stream h.264, 720p, 6fps)
          roles:
            - detect
            - rtmp
            - record           
    motion:
      mask:
        - 1280,282,1219,234,1144,210,982,159,886,117,819,132,592,720,1280,720
        - 601,0,510,73,0,103,0,0           

  path_frigate: # <------ Name the camera    
    ffmpeg:
      inputs:
        - path: rtsp://username:password@<dahuaip>:554/cam/realmonitor?channel=1&subtype=02 #(3rd stream h.264, 720p, 6fps)
          roles:
            - detect
            - rtmp
            - record             
    motion:
      mask:    
        - 1035,0,985,569,617,687,385,291,369,0

xbmcnut · October 10, 2021, 6:15am

What are the record settings for the camera with the missing clips?

mcpaha · October 10, 2021, 6:55am

Default settings that every camera detect, record, rtmp. And I see with Frigate UI all the cameras events & clips. That’s why I am asking this because I don’t get it that only one camera clips are visible in home assistant…

sezgin9716 · October 10, 2021, 8:30am

Since the latest changes to Frigate NVR in Home assistant the Frigate NVR not working

s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2021-10-10 11:27:02] frigate.app                    INFO    : Starting Frigate (0.9.1-800f33e)
Error parsing config: 6 validation errors for FrigateConfig
cameras -> garden-cam -> ffmpeg -> inputs -> 0 -> roles -> 2
  value is not a valid enumeration member; permitted: 'record', 'rtmp', 'detect' (type=type_error.enum; enum_values=[<CameraRoleEnum.record: 'record'>, <CameraRoleEnum.rtmp: 'rtmp'>, <CameraRoleEnum.detect: 'detect'>])
cameras -> garden-cam -> clips
  extra fields not permitted (type=value_error.extra)
cameras -> garden-cam -> fps
  extra fields not permitted (type=value_error.extra)
cameras -> garden-cam -> height
  extra fields not permitted (type=value_error.extra)
cameras -> garden-cam -> width
  extra fields not permitted (type=value_error.extra)
etectors
  extra fields not permitted (type=value_error.extra)
[cmd] python3 exited 1
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.

gr1nch · October 10, 2021, 12:25pm

The errors indicate that your config is now broken. This happened to me too, I was using the clips option which is deprecated in the newer versions. Check out the master documentation site in the config sections and examples -

gr1nch · October 10, 2021, 12:30pm

Switching from the native Frigate integration on HASS, because performance was awful on a RPi4 without a Coral tpu, using the Frigate NVR Proxy integration, all my lovelace frigate cards can not now display clips or snapshots. Anyone worked out how to fix this?

tmjpugh · October 10, 2021, 1:30pm

Did the camera capture any clips?

What are record settings/retain settings? (Post config)

jasonshearer · October 10, 2021, 2:14pm

I am seeing about the same performance on a NUC8i5BEH with a Coral processing six cameras. With the new version of Frigate I actually enabled three days of retention. Hoping to stay around ~60% utilization of my 1TB SSD. I have longer retention on my Ubiquiti UDMP.

sender · October 10, 2021, 4:22pm

So since last update a lot of this changed and indeed a lot of them are breaking.
Speaking about breaking. My “clips” which are still called clips in the UI, do not play anymore on pressing them. I have recording configured and it seems there is given the fact there is a “tile”. Just don’t play anymore

ukro · October 10, 2021, 4:44pm

^^^^^
@blakeblackshear

ukro · October 10, 2021, 4:45pm

Same here, waiting for balckbear to help.Maybe will report github issue

blakeblackshear · October 10, 2021, 4:57pm

If they are new events, you most likely never added the record role to an input. See if you have a warning in the logs on startup.

ukro · October 10, 2021, 5:04pm

I am sorry,i think its my fault somewhere.
Its working.
I don’t know what i did…
So now there will be no clips? It will be saved in recordings?
So as i can see it is not cached in ram to create clips,it is going directly to recodings.
Now if i have 11cameras, they will be constantly writing to the NAS?

scote · October 10, 2021, 5:41pm

I have tidied up my config moving common ffmpeg options to global config. Made adjustments for new config requirements. At the same time I got ffmpeg to resize output down to 1280x720 for detector and drop fps to 5. I am running 3 cams at 1600x900 10fps decoding on nvidia gt640 with an 8yr old Haswell i5-4670. This resulted in CPU use dropping from 11.5% to 6.4% as reported by HA Supervisor. I believe the resize is being done by GPU decoder so no need for separate streams. I wonder if the fps change could be done on input too further lowering CPU?

mqtt:
  host: 192.168.50.191
  user: *******
  password: **********
ffmpeg:
  input_args:
    - -vsync
    - 0
    - -c:v
    - h264_cuvid
    - -resize
    - 1280x720
  output_args:
    detect: -f rawvideo -pix_fmt yuv420p -vf fps=5
detect:
  width: 1280
  height: 720
  fps: 5
objects:
  track:
    - person
    - car
    - truck
    - bicycle
    - dog
    - cat
detectors:
  coral:
    type: edgetpu
    device: usb
detect:
  enabled: True
snapshots:
  enabled: True

record:
  enabled: True
  retain_days: 0
  events:
    max_seconds: 300
    pre_capture: 5
    post_capture: 5
    retain:
      default: 10
      objects:
        person: 30

cameras:
  back:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.144:8554/unicast
          roles:
            - detect
            - rtmp
            - record
    motion:
      mask:
        - 0,179,531,109,577,0,0,0
    zones:
      door:
        coordinates: 1051,179,780,720,1068,720,1241,141
    mqtt:
      enabled: False

  side:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.215:8554/unicast
          roles:
            - detect
            - rtmp
            - record
    motion:
      mask:
        - 845,529,875,663,963,672,1004,574,1600,683,1600,0,113,0,157,630
    snapshots:
      enabled: False
    mqtt:
      enabled: False

  garden:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.133:8554/unicast
          roles:
            - detect
            - rtmp
            - record
    mqtt:
      enabled: False

sender · October 10, 2021, 7:42pm

I think I did?

cameras:
# FRONT Camera ########################################################################################################
  voordeur:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
            - record
    detect:
      enabled: True
      max_disappeared: 2
      width: 2560
      height: 1440
      fps: 6
    record:
      enabled: False
      retain_days: 0
      events:
        post_capture: 10
        pre_capture: 5
        objects:
        - person
        - dog
        - cat
        retain:
          default: 10      
    snapshots:
      enabled: true
      crop: false
      bounding_box: True
    objects:
      track:
        - person
        - dog
        - cat
    motion:
      mask:
        - '1090,81,1216,0,1382,34,1532,99,1518,259,1426,363,1123,247'
        - '1766,498,1753,669,1778,722,1829,773,1863,825,1906,884,1924,955,1953,1065,2237,1169,2255,1128,2295,1046,2338,946,2378,841,2410,743,2215,647,2190,582,2138,512,2010,402,1859,351'

And I do not see anything in the log except after a while (dont know how long):

2021-10-10 18:21:54] frigate.object_processing      WARNING : Unable to save snapshot for 1633882905.655275-6lyobp.
[2021-10-10 18:21:54] frigate.object_processing      WARNING : Unable to create clean png because frame 1633882908.033336 is not in the cache
[2021-10-10 18:21:54] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633882905.655275-6lyobp.
[2021-10-10 18:21:54] frigate.object_processing      WARNING : Unable to create jpg because frame 1633882908.033336 is not in the cache
[2021-10-10 18:22:48] frigate.object_processing      WARNING : Unable to create jpg because frame 1633882963.404733 is not in the cache
[2021-10-10 18:22:48] frigate.object_processing      WARNING : Unable to save snapshot for 1633882961.399878-ot2xpx.
[2021-10-10 18:22:48] frigate.object_processing      WARNING : Unable to create clean png because frame 1633882963.404733 is not in the cache
[2021-10-10 18:22:48] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633882961.399878-ot2xpx.
[2021-10-10 18:22:48] frigate.object_processing      WARNING : Unable to create jpg because frame 1633882963.404733 is not in the cache
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883010.038617 is not in the cache
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to save snapshot for 1633883007.885112-n15t09.
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to create clean png because frame 1633883010.038617 is not in the cache
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633883007.885112-n15t09.
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883010.038617 is not in the cache
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883010.038617 is not in the cache
[2021-10-10 18:23:35] frigate.object_processing      WARNING : Unable to send mqtt snapshot for 1633883007.885112-n15t09.
[2021-10-10 18:28:32] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883306.875549 is not in the cache
[2021-10-10 18:28:32] frigate.object_processing      WARNING : Unable to save snapshot for 1633883304.396746-x48343.
[2021-10-10 18:28:32] frigate.object_processing      WARNING : Unable to create clean png because frame 1633883306.875549 is not in the cache
[2021-10-10 18:28:32] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633883304.396746-x48343.
[2021-10-10 18:28:32] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883306.875549 is not in the cache
[2021-10-10 18:36:46] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883801.721082 is not in the cache
[2021-10-10 18:36:46] frigate.object_processing      WARNING : Unable to save snapshot for 1633883800.617849-u5z8n1.
[2021-10-10 18:36:46] frigate.object_processing      WARNING : Unable to create clean png because frame 1633883801.721082 is not in the cache
[2021-10-10 18:36:46] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633883800.617849-u5z8n1.
[2021-10-10 18:36:46] frigate.object_processing      WARNING : Unable to create jpg because frame 1633883801.721082 is not in the cache
[2021-10-10 18:41:59] frigate.object_processing      WARNING : Unable to create jpg because frame 1633884109.39016 is not in the cache
[2021-10-10 18:41:59] frigate.object_processing      WARNING : Unable to save snapshot for 1633884107.864138-4qufnf.
[2021-10-10 18:41:59] frigate.object_processing      WARNING : Unable to create clean png because frame 1633884109.39016 is not in the cache
[2021-10-10 18:41:59] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633884107.864138-4qufnf.
[2021-10-10 18:41:59] frigate.object_processing      WARNING : Unable to create jpg because frame 1633884109.39016 is not in the cache
[2021-10-10 18:45:26] frigate.object_processing      WARNING : Unable to create jpg because frame 1633884321.581425 is not in the cache
[2021-10-10 18:45:26] frigate.object_processing      WARNING : Unable to save snapshot for 1633884320.011118-9mf083.
[2021-10-10 18:45:26] frigate.object_processing      WARNING : Unable to create clean png because frame 1633884321.581425 is not in the cache
[2021-10-10 18:45:26] frigate.object_processing      WARNING : Unable to save clean snapshot for 1633884320.011118-9mf083.
[2021-10-10 18:45:26] frigate.object_processing      WARNING : Unable to create jpg because frame 1633884321.581425 is not in the cache

Hence me asking to "how to delete the database other than just delete the file below, or is that teh way and will it be recreated after I restart the container?


database:
  path: /media/frigate/db/frigate.db