Local realtime person detection for RTSP cameras

You are probably running with --rm.

Thank you for reply, i’m using frigate addon. And i’m trying to figure out how to resetup the addon to make additional changes when the addon is started.
Is the --rm in your default setup/image?
Can it be removed?

H all and @blakeblackshear

I note a few people are having problems with filters and have read some of the fixes. But is anyone have trouble with a mask in a filter?

I have this in global config:

  track:
    - car
    - person
    - bicycle

And this under my front camera:

    objects:
      track:
        - car
        - person
        - bicycle
      mask:
        - 0,174,331,0,0,0

However the mask is ignored?

At the camera level, mask should be under ‘motion:’, or specified as part of the object filter - example:

    motion:
      delta_alpha: 0.6
      threshold: 18
      mask:
        - 1463,946,1508,1061,1286,1161,0,1809,0,0,1006,0,1767,0,1732,184,1640,178,1594,181,1548,204,1509,206,1462,247,1424,276,1459,421,1492,635
        - 3072,0,2491,603,2385,416,2305,327,2276,316,2178,376,2082,308,1965,242,2070,0
    objects:
      track:
        - person
        - car
        - dog       
      filters:
        car: 
          mask: 
            - 1042,942,1506,929,1517,338,1159,325       
        dog:
          mask:
            - 0,1779,0,2048,2919,2048,1623,1057,1466,1059
        person:
          mask:
            - 0,1779,0,2048,2919,2048,1623,1057,1466,1059

EDIT: Just RTFM-ed myself… seems you’re correct, mask can also apply directly below object:

… did you try it as per the example, just in case?

objects:
  track:
    - person
    - car
  mask: 0,0,1000,0,1000,200,0,200

Looks like Google Coral USB TPUs are starting to come into stock, my order from Mouser I put in back in the beginning of March just shipped. Hopefully others will see them soon too.

I have managed to mount a network share to the host system HASSOS to allow frigate to save to network share
Tutorial:
https://community.home-assistant.io/t/solved-hassos-mount-nas-network-share/

Other tutorial that i did before with proxmox container is depreciated

1 Like

I’ve setup an automation via nodeRED to turn off detection of my driveway camera for a few minutes when our front door is opened. I have zigbee door sensors so use those to turn off the Frigate detection for 3 minutes, then turn detection back on. It obviously doesn’t record a clip of any detections, so not exactly what you asked for but it works for my use case.

I’m a newbie to Node-Red, and have tried a few different examples for android notifications but have never got it working correctly. I use a yaml automation for android notifications which works ok, but would rather keep everything in NodeRED.

Ive done something similar in yaml with 2 automations.

- id: '1612466944061'
  alias: Camera de-activate
  description: Deactivate front camera triggers on front door opening
  trigger:
  - platform: state
    entity_id: binary_sensor.front_door
    from: 'off'
    to: 'on'
  condition: []
  action:
  - service: switch.turn_off
    data: {}
    entity_id: group.camera_disable
  mode: single

- id: '1612467240719'
  alias: Camera re-activate
  description: Deactivate front camera triggers on front door opening
  trigger:
  - platform: state
    entity_id: binary_sensor.front_door
    from: 'on'
    to: 'off'
  condition: []
  action:
  - delay:
      hours: 0
      minutes: 15
      seconds: 0
      milliseconds: 0
  - service: switch.turn_on
    data: {}
    entity_id: group.camera_disable
  mode: single

and grouped the camera’s I’m using

Camera_disable:
  name: Camera disable
  icon: mdi:cctv
  entities:
    - switch.doorcam_detect
    - switch.drivewaycam_detect

It works well enough for us.

Any sugestions on settings?
when i remove this,which is recomended in docs it is working.

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

I’ve got green screens on all 12 cameras,stream is H264
But i want to enable the passthrough integrated GPU that i pass to be operational.
Intel <10gen settings

Thank you.
If anybody have NUCs without corral, maybe your settings should work? please let me know.

frigate.video                  INFO    : CAM4: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM3: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM5: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM3: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM2: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM2: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM5: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM1: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM1: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM7: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have 
ffmpeg.CAM_technicka.detect    ERROR   : Consider increasing the value for the 'analyzeduration' and 'probesize' options
ffmpeg.CAM_technicka.detect    ERROR   : [h264_v4l2m2m @ 0x55d40a815800] Could not find a valid device
ffmpeg.CAM_technicka.detect    ERROR   : [h264_v4l2m2m @ 0x55d40a815800] can't configure decoder
ffmpeg.CAM_technicka.detect    ERROR   : Error while opening decoder for input stream #0:0 : Invalid argument
ffmpeg.CAM8.detect             ERROR   : [h264_v4l2m2m @ 0x55595fafa580] Could not find a valid device
ffmpeg.CAM8.detect             ERROR   : [h264_v4l2m2m @ 0x55595fafa580] can't configure decoder

i don’t know how to try this suggestion?!

Okay so this is working:

Is this the prefered solution for integrated GPU?

I am unable to start the add-on. I get the following error is the Supervisor Log:

21-04-28 07:20:43 ERROR (SyncWorker_5) [supervisor.docker] Can’t start addon_ccab4aaf_frigate: 500 Server Error for http+docker://localhost/v1.41/containers/dd9eacd7ee8db5eee7889870ab035d7307e7185b3eda88652b2c2aaaf31160a5/start: Internal Server Error ("driver failed programming external connectivity on endpoint addon_ccab4aaf_frigate (09d24995e8e6a564e6453d9a6efe77eed595814c38fce80807

Did you tryed supervisor->system->reboot host?
source

Yes, still does seem to have ay affect :slight_smile:
I’ll continue my fault finding

Hi,

so I tried lowering the “max_disappeared” setting with no visible change. Also the lowering the “best_image” did not produce more snapshots.

Could you post your node-red code for me please? Have not worked with node red before, this is new land for me.
Thank you very much.

Cheers Chris

Do you know where I can get a copy of the firmware? The original firmware it was shipped with only gave me the option for h265 too, which is one of the reasons I swapped.

How would I go about transcoding the stream to include the frigate detection boxes? I have tried this:

ffmpeg:
  global_args: 
    -hide_banner 
    -loglevel warning
  input_args: 
    -avoid_negative_ts make_zero 
    -fflags +genpts+discardcorrupt 
    -rtsp_transport tcp 
    -stimeout 5000000
  output_args:
    clips: 
      -f segment 
      -segment_time 300 
      -segment_format mp4 
      -reset_timestamps 1 
      -strftime 1 
      -c copy 
      -an

but that give me the error:

Unable to create clip for 04_Side_Gate and event 1619622504.318758-5yi399. There were no cache files for this event.

I can’t find the information to tell me what each of the arguments do, or what the options are to change them too. I’ve looked here. Configuration | Frigate (blakeblackshear.github.io)

Also my camera is quite high up, and some objects can be quite small, is there a way of increasing person detection when they are smaller in the frame? I have the following at the moment:

          min_area: 0
          min_score: 0.2
          threshold: 0.3

Increase the ‘-detect’ resolution stream worked for me in a similar situation, I ended up using 3072x2048 (max resolution mode from hikvision fisheye). This way, a person can easily fill 300x300px.

Thanks. I’ll wait for my coral adaptor to arrive tomorrow before doing that.
I’m trying to reencode the stream, but i’m getting these errors:

ffmpeg.06_Rear_Garden.clips ERROR : Guessed Channel Layout for Input Stream #0.1 : mono
ffmpeg.07_West_Fence.clips ERROR : [rtsp @ 0x5630f1244840] max delay reached. need to consume packet
ffmpeg.06_Rear_Garden.clips ERROR : [hevc @ 0x5584da18b700] Could not find ref with POC 0
ffmpeg.07_West_Fence.clips ERROR : [rtsp @ 0x5630f1244840] RTP: missed -63704 packets
ffmpeg.07_West_Fence.clips ERROR : Guessed Channel Layout for Input Stream #0.1 : mono
ffmpeg.07_West_Fence.clips ERROR : [hevc @ 0x5630f12cec40] Could not find ref with POC 4
ffmpeg.07_West_Fence.clips ERROR : [rtsp @ 0x5630f1244840] max delay reached. need to consume packet
ffmpeg.07_West_Fence.clips ERROR : [rtsp @ 0x5630f1244840] RTP: missed 1864 packets

my current ffmpeg looks like this:

ffmpeg:
  global_args: 
    -hide_banner 
    -loglevel warning
  input_args: 
    - -avoid_negative_ts
    - make_zero
    - -fflags
    - nobuffer
#    - -rtsp_transport tcp
#    - -stimeout 5000000
  output_args:
    clips: 
      -f segment
      -segment_time 300
      -segment_format mp4
      -reset_timestamps 1
      -strftime 1
      -c:v libx264
      -an

first, awesome work!

Everything works with only CPU processing, but I have a problem with the USB Coral Stick.
I’m not sure if the Coral stick is damaged or if it’s some other kind of error.
I am using Proxmox as host system running HassOS.

lsusb on the host looks like this:



however, it is not completely displayed to me, is this correct?

in the log, it doesn’t look very good, anyway:

* Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.app                    INFO    : Camera processor started for back: 49
frigate.app                    INFO    : Capture process started for back: 50
detector.coral                 INFO    : Starting detection process: 46
frigate.mqtt                   INFO    : MQTT connected
Process detector:coral:
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.edgetpu                INFO    : No EdgeTPU detected.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...
frigate.events                 INFO    : Exiting event processor...
frigate.record                 INFO    : Exiting recording maintenance...
frigate.object_processing      INFO    : Exiting object processor...
frigate.watchdog               INFO    : Exiting watchdog...
frigate.events                 INFO    : Exiting event cleanup...
frigate.stats                  INFO    : Exiting watchdog...
peewee.sqliteq                 INFO    : writer received shutdown request, exiting.
root                           INFO    : Waiting for detection process to exit gracefully...

the config file I have changed to that:

detectors:
  coral:
    type: edgetpu
    device: usb
mqtt:
........

is there anything I could check? or is it possibly the host machine?

thanks a lot.

Haven’t tried it myself, but I want to if the PCIe corals ever come back in stock. I also run proxmox and HassOS and from what I’ve read in the past you need to install the drivers on the proxmox host. I never did understand if the passthrough to HassOS worked without any other modifications though.

I can’t find it on the fly, only in use with the PCIE card.

do you know where you read that?

does not make much sense for me… i mean the deconz USB stick you can also simply pass through.

@sender - While I’m not on LXC, I am running Frigate on Unraid/Docker and HA in a VM with Frigate addon. I am thinking though your problem lies somewhere within the Frigate deployment and i can share my working configuration config.yml file if you like to compare. I started from scratch and now have pretty much every option turned on and working. I also use 2 cpu option (no Coral or any such device). Let me know.