Local realtime person detection for RTSP cameras

Jooced · April 25, 2021, 11:08pm

Walt made an incredible node-red automation for the native iPhone HA app here:

I’ve tried to recreate this for Android but I’m not having any luck. Has anyone created a flow for node-red and Android app with some advanced functionality? I’d really like to do two things, be able to mute notifications for X minutes off the notification and disable notifications if a door has been recently opened (don’t notify drive way camera for 10 minutes if the front door was opened).

I’m just not there with my programming know-how even if node-red is doing most of it.

asherw83 · April 25, 2021, 11:54pm

If you haven’t already, attempt to flash back to Annke firmware. My C800 has a dropdown to choose between h264 and h265.
I’m surprised Hikvision doesn’t have this same option. Try latest firmware maybe?
Best of luck!

tupac · April 26, 2021, 8:15am

Hi,

thank you @blakeblackshear! It’s great tool

I’m looking for install it on a jetson nano 2GB (for me google coral little bit expensive and my HA is running on intel nuc virtualbox). As I see in the forum it is possible. Can Frigate work on a jetson nano 2gb? And how can I install it on jetson nano, e.g. ubuntu + Frigate in docker?

Than you in advance!

konradwalsh · April 26, 2021, 8:18am

Well done to you,… I was worried the naysayers would throw you off… Thanks for contributing back to us

ukro · April 26, 2021, 8:25am

Thx m8

ukro · April 26, 2021, 8:30am

Look this guy use it

If you know linux, partly you can use this tutorial, but only part as its for intel integrated GPU
https://community.home-assistant.io/t/network-share-on-frigate-hassos-with-nabucasa-ssl-and-proxmox-lxc-container/
I’m now also interested in this jetson. If/when i buy i might make a tutorial.

ukro · April 26, 2021, 8:41am

Do somebody use Frigate on Raspberry pi 4? How is the performance?
Because i read this

TensorFlow Lite on Raspberry Pi 4 can achieve performance comparable to NVIDIA’s Jetson Nano at a fraction of the cost.

https://towardsdatascience.com/3-ways-to-install-tensorflow-2-on-raspberry-pi-fe1fa2da9104

Dmitriy_Marchevsky · April 26, 2021, 9:01am

works with two 720p streams at 5 fps detection, inference speed is 200-300ms

ukro · April 26, 2021, 9:07am

Thank you for reply, can you explain what is inference speed?
Do you think it can handle any more cameras as 5fps?

ukro · April 26, 2021, 8:48pm

@blakeblackshear Hi, please can you explain why whenever i stop frigate the container just disapear from the listing in portainer.
P.S. cache is cleared. Other containers for example ssh are in stop state when i stop them from HASS GUI.
Thank you

blakeblackshear · April 26, 2021, 9:10pm

You are probably running with --rm.

ukro · April 26, 2021, 9:34pm

Thank you for reply, i’m using frigate addon. And i’m trying to figure out how to resetup the addon to make additional changes when the addon is started.
Is the --rm in your default setup/image?
Can it be removed?

imellor · April 27, 2021, 9:21am

H all and @blakeblackshear

I note a few people are having problems with filters and have read some of the fixes. But is anyone have trouble with a mask in a filter?

I have this in global config:

  track:
    - car
    - person
    - bicycle

And this under my front camera:

    objects:
      track:
        - car
        - person
        - bicycle
      mask:
        - 0,174,331,0,0,0

However the mask is ignored?

iMiMx · April 27, 2021, 10:20am

At the camera level, mask should be under ‘motion:’, or specified as part of the object filter - example:

    motion:
      delta_alpha: 0.6
      threshold: 18
      mask:
        - 1463,946,1508,1061,1286,1161,0,1809,0,0,1006,0,1767,0,1732,184,1640,178,1594,181,1548,204,1509,206,1462,247,1424,276,1459,421,1492,635
        - 3072,0,2491,603,2385,416,2305,327,2276,316,2178,376,2082,308,1965,242,2070,0
    objects:
      track:
        - person
        - car
        - dog       
      filters:
        car: 
          mask: 
            - 1042,942,1506,929,1517,338,1159,325       
        dog:
          mask:
            - 0,1779,0,2048,2919,2048,1623,1057,1466,1059
        person:
          mask:
            - 0,1779,0,2048,2919,2048,1623,1057,1466,1059

EDIT: Just RTFM-ed myself… seems you’re correct, mask can also apply directly below object:

… did you try it as per the example, just in case?

objects:
  track:
    - person
    - car
  mask: 0,0,1000,0,1000,200,0,200

noebl1 · April 27, 2021, 2:20pm

Looks like Google Coral USB TPUs are starting to come into stock, my order from Mouser I put in back in the beginning of March just shipped. Hopefully others will see them soon too.

ukro · April 27, 2021, 3:42pm

I have managed to mount a network share to the host system HASSOS to allow frigate to save to network share
Tutorial:
https://community.home-assistant.io/t/solved-hassos-mount-nas-network-share/

Other tutorial that i did before with proxmox container is depreciated

dan123 · April 27, 2021, 8:08pm

I’ve setup an automation via nodeRED to turn off detection of my driveway camera for a few minutes when our front door is opened. I have zigbee door sensors so use those to turn off the Frigate detection for 3 minutes, then turn detection back on. It obviously doesn’t record a clip of any detections, so not exactly what you asked for but it works for my use case.

I’m a newbie to Node-Red, and have tried a few different examples for android notifications but have never got it working correctly. I use a yaml automation for android notifications which works ok, but would rather keep everything in NodeRED.

denver · April 27, 2021, 8:29pm

Ive done something similar in yaml with 2 automations.

- id: '1612466944061'
  alias: Camera de-activate
  description: Deactivate front camera triggers on front door opening
  trigger:
  - platform: state
    entity_id: binary_sensor.front_door
    from: 'off'
    to: 'on'
  condition: []
  action:
  - service: switch.turn_off
    data: {}
    entity_id: group.camera_disable
  mode: single

- id: '1612467240719'
  alias: Camera re-activate
  description: Deactivate front camera triggers on front door opening
  trigger:
  - platform: state
    entity_id: binary_sensor.front_door
    from: 'on'
    to: 'off'
  condition: []
  action:
  - delay:
      hours: 0
      minutes: 15
      seconds: 0
      milliseconds: 0
  - service: switch.turn_on
    data: {}
    entity_id: group.camera_disable
  mode: single

and grouped the camera’s I’m using

Camera_disable:
  name: Camera disable
  icon: mdi:cctv
  entities:
    - switch.doorcam_detect
    - switch.drivewaycam_detect

It works well enough for us.

ukro · April 27, 2021, 8:42pm

Any sugestions on settings?
when i remove this,which is recomended in docs it is working.

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

I’ve got green screens on all 12 cameras,stream is H264
But i want to enable the passthrough integrated GPU that i pass to be operational.
Intel <10gen settings

Thank you.
If anybody have NUCs without corral, maybe your settings should work? please let me know.

frigate.video                  INFO    : CAM4: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM3: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM5: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM3: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM2: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM2: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM5: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM1: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : CAM1: ffmpeg process is not running. exiting capture thread...
frigate.video                  INFO    : CAM7: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have

ffmpeg.CAM_technicka.detect    ERROR   : Consider increasing the value for the 'analyzeduration' and 'probesize' options
ffmpeg.CAM_technicka.detect    ERROR   : [h264_v4l2m2m @ 0x55d40a815800] Could not find a valid device
ffmpeg.CAM_technicka.detect    ERROR   : [h264_v4l2m2m @ 0x55d40a815800] can't configure decoder
ffmpeg.CAM_technicka.detect    ERROR   : Error while opening decoder for input stream #0:0 : Invalid argument
ffmpeg.CAM8.detect             ERROR   : [h264_v4l2m2m @ 0x55595fafa580] Could not find a valid device
ffmpeg.CAM8.detect             ERROR   : [h264_v4l2m2m @ 0x55595fafa580] can't configure decoder

i don’t know how to try this suggestion?!

Okay so this is working:

Is this the prefered solution for integrated GPU?

lawrencez · April 28, 2021, 7:21am

I am unable to start the add-on. I get the following error is the Supervisor Log:

21-04-28 07:20:43 ERROR (SyncWorker_5) [supervisor.docker] Can’t start addon_ccab4aaf_frigate: 500 Server Error for http+docker://localhost/v1.41/containers/dd9eacd7ee8db5eee7889870ab035d7307e7185b3eda88652b2c2aaaf31160a5/start: Internal Server Error ("driver failed programming external connectivity on endpoint addon_ccab4aaf_frigate (09d24995e8e6a564e6453d9a6efe77eed595814c38fce80807