Local realtime person detection for RTSP cameras

pcwii · March 8, 2021, 7:59pm

I have attempted with the ReoLink Configuration example and without. I have also tried disabling all roles except detect: and I have removed all hwaccel_args:
for what it is worth I have tried both RTMP streams and RTSP streams from the camera (all display fine in vlc)
I still have a green camera display and logs are repeatable showing these now.

ffmpeg.porch.detect            ERROR   : Option rtsp_transport not found.
frigate.video                  INFO    : porch: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : porch: ffmpeg process is not running. exiting capture thread...

thundergreen · March 8, 2021, 8:37pm

Check your resolution of the cam if it matches in your config

pcwii · March 8, 2021, 9:22pm

I think I am ok here but not certain. if I have width / height correct.

So for what it is worth. I changed to an RTSP stream again. updated my camera FPS to match my configuration. All looks better now. Thanks for the support.

umbramalison · March 8, 2021, 9:38pm

can i change the ffmpeg binaries in the frigate docker container. or can I run frigate without docker and use the host’s ffmpeg?

i’ve got a jetson nano, with a custom ffmpeg that I want to use in frigate.

Any ideas? I don’t know where to start!

ukro · March 8, 2021, 10:36pm

I have frigate nvr integration. Is it possible to save the clips on my network share? As it is taking a lot of space on my HASS OS.

Eddie1974 · March 9, 2021, 9:53am

HI all,

I have a latest person image as a button, how would i play the latest clip associated with that latest person image? I want to use the media player as a popup window.

mcpaha · March 9, 2021, 1:32pm

Hi!

Maybe this is a stupid question but I’d like to use frigate so that it records clips when there is movement (sensitivy is selectable) and also does object detection if it can. Now it seems (or don’t know how to conf) that it saves clips only when it understand configured object.

Also like to know how to manually clean old events (or just lower retention day) and why HA Media browser doesn’t show thumbnails from pics & clips (however all of these working if clicked).

sender · March 9, 2021, 1:50pm

I bought myself a coral usb… expect questions the next period when it arrives :-).

for now please help me with Local realtime person detection for RTSP cameras

ThaNerd · March 9, 2021, 6:29pm

How do I fix the time on the clips? It’s not showing the right time…

So far, I traced the following:
Human and Car works well but…
85% dog = my wife wearing a coat with a fur hoodie and fur earmuffs
75% dog = a squirell
70% dog = some cat
Never traced an actual dog or cat, will def need training…

noebl1 · March 9, 2021, 6:45pm

@omayhemo I had placed an order from Mouser last week for the Google Coral USB TPU, Mouser sent me an email yesterday saying ETA is now 3/16 for shipment (web site still says 3/15), so maybe that’s the next batch arrival?

sender · March 9, 2021, 7:26pm

Yes 1400 pieces at 15 march I saw…

noebl1 · March 9, 2021, 7:38pm

haha sorry I meant more their email seemed to lock in the eta vs the site that’s changed a few times

ThaNerd · March 9, 2021, 8:57pm

Just updated my docker and 2 of my 3 cameras are green. Used to work flawlessly before.

Both are Dlink DCS-2670L (specs)

Edit: had to unplug both cameras to make it work again weird. Both cameras were working on Blueiris the whole time so something happened to frigate when I upgraded my docker…

champ26 · March 9, 2021, 10:19pm

weird. updated HA to 2021.3.3 from 2021.3.0 and then upgraded frigate from 1.12 to 1.13, and now seems to update fine.

sender · March 10, 2021, 9:35am

sender · March 10, 2021, 11:36am

Can I please have “some” help :-)… the fun is going away

Tried a mask… not helping…

Starting migrations,
peewee_migrate                 INFO    : Starting migrations,
There is nothing to migrate,
peewee_migrate                 INFO    : There is nothing to migrate,
frigate.mqtt                   INFO    : MQTT connected,
detector.cpu1                  INFO    : Starting detection process: 44,
detector.cpu2                  INFO    : Starting detection process: 46,
frigate.app                    INFO    : Camera processor started for voordeur: 49,
frigate.app                    INFO    : Camera processor started for schuurachter: 50,
frigate.app                    INFO    : Camera processor started for parkeerplaats: 58,
frigate.app                    INFO    : Capture process started for voordeur: 63,
frigate.app                    INFO    : Capture process started for schuurachter: 69,
frigate.app                    INFO    : Capture process started for parkeerplaats: 71,
Fatal Python error: Fatal Python error: Bus error,
,
Thread 0xBus error,
,
00007f662461b700 (most recent call first):,
  File "Thread 0x/u00007f662461b700s (most recent call first):,
r  File /"l/iubs/rp/lib/python3.8/threading.yptyho"n, line 3302. in 8w/aitth,
r  File e"/usr/lib/pythona3d.i8ng/.mupylt"i, line p302r in owcaeists,
ing/queues.py", line 227 in _feed  File ,
  File ""//uussrr//lliibb//ppyytthhoon3n3.8.8/t/hmruelatdiinprg.opceys"s, line i870 in nrg/uqn,
ue  File "ue/ussr/lib/python3.8/thr.epayd"i, line n227g in ._pfy", line 932 in _bootstrap_inner,
  File "/usr/libe/epdyt,
h  File o"n3/u.8s/rt/hlrieba/pdyitngh.onp3y."8, line /890t in hr_eboaoditnstrap,
,
Current thread 0x00007f6626e1c700 (most recent call first):,
  File "/optg/frigate/frigate/video.py", line 117 in capture_frames,
  File "/opt/frigate/frigate/video.py", line 231 in run,
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner,
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap,
,
Thread 0x00007f662961d700 (most recent call first):,
  File "/opt/frigate/frigate/video.py", line 205 in run,
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner,
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap,
,
Thread 0x00007f662be1e700 (most recent call first):,
  File "/opt/frigate/frigate/log.py", line 71 in run,
  File "/usr/lib/python3.8/threading.py", line 932 in _bootstrap_inner,
  File "/usr/lib/python3.8/threading.py", line 890 in _bootstrap,
,
Thread 0x00007f663d814740 (most recent call first):,
  File "/usr/lib/python3.8/threading.py", line 1027 in _wait_for_tstate_lock,
  File "/usr/lib/python3.8/threading.py", line 1011 in join,
  File "/opt/frigate/frigate/video.py", line 245 in cap.tuprye"_, line c870a in mreunr,
a  File ,
"  File /u"sr//ulsirb//lpiytbh/opny3t.h8/otnh3r.e8a/dmiungl.tpiyp"r, line o932c in e_sbsoiontgst/rparpo_cinensesr.,
p  File "y"/u, line s108r in /rluinb,
/p  File y"t/huosnr3./8l/itbh/rpeaydtihnog.np3y."8, line /890 in m_ublotoitpsrtorcaeps,
s,
iCurrent thread 0xn00007f6626e1c700g (most recent call first):,
/  File p"r/oocpets/sf.ripgya"te, line /f315r in i_gbaotoet/vsitdreoa.pp,
y  File ", line "117/ in ucsarp/tluirbe/_pfyratmheosn,
3  File ."8//omput/lftriipgraotcee/sfrsiignagt/ep/ovpideeno_.fpyo"r, line k231. in pryu"n,
, line   File 75" in /_ulsaru/lnicb/hp,
yt  File h"o/n3u.s8r//thlrieba/dpinygt.hpoyn"3, line .9328 in /_mbuolottisptrraopc_eisnsnienr,
g  File /"p/oupser/nl_ifbo/rpkyt.hpoyn"3, line .198 in /_th_rienaidti_n_g.,
p  File y"", line 890/ in u_sbro/otlsitbr/app,
y,
tThread 0xh00007f662961d700o (most recent call first):,
n  File 3"./8o/pmtu/lftriipgartoec/efsrsigiantge//cvoindteeo.xpty."py, line 205" in , line r277u in n,
_  File P"o/puesnr,
/  File l"i/bu/spry/tlhoinb3/.p8y/tthhroena3d.i8n/gm.uplyt"i, line p932 in r_obcoeostsstirnagp/_cionnnteer,
x  File t"./puys"r/, line l224i in b_/Ppyotpheonn,
3  File ."8//utshrre/aldiibn/gp.yptyh"o, line n890 in 3_.b8o/omtustlrtaipp,
r,
oThread 0xc00007f662be1e700e (most recent call first):,
s  File s"i/nopgt//pfrroicgeastse./pfryi"ga, line t121e in /slotga.rpty,
"  File , line "71/ in orupnt,
/  File f"r/iugsra/tlei/bf/rpiygtahtoen3/.a8p/pt.hprye"a, line di182n in gs.tpayr"t, line _932c in _abmoeortas_tcraapp_tiunrneer_,
p  File r"/oucsers/sleisb/,
py  File t"h/ono3p.t8//fthrriegaadtien/gf.pryi"g, line a890t in e_/baopopt.sptyr"ap, line ,
228,
 in Thread 0x00007f663d814740s (most recent call first):,
t  File a"r/tu,
s  File r/"l/iobp/tp/ython3f.r8i/gtahtree/afdriingga.tpey", line 1027 in _wait_for_tstate_lo/c_k_,
  File m"a/iuns_r_/.lipby/"py, line t15ho in n<3.m8o/dtuhrleea>d,
in  File g."p/yu"s, line r1011/ in ljiobi/np,
y  File t"h/oonp3t/.f8r/irguantpey/.fpryig"at, line e87/ in v_irdueno_.cpyo"de, line ,
245 in   File ca"p/tuusre_camre/rlai,
b  File "//puysr/lib/python3.8/multtihporonc3e.s8s/inrgu/npproyc.epsys".p, line y194" in , line 108_ in rruunn_,
m  File o"d/uulser_/alsi_b/mpayitnh,
on3.8/multiprocessing/process.py", line 315 in _bootstrap,
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75 in _launch,
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19 in __init__,
  File "/usr/lib/python3.8/multiprocessing/context.py", line 277 in _Popen,
  File "/usr/lib/python3.8/multiprocessing/context.py", line 224 in _Popen,
  File "/usr/lib/python3.8/multiprocessing/process.py", line 121 in start,
  File "/opt/frigate/frigate/app.py", line 182 in start_camera_capture_processes,
  File "/opt/frigate/frigate/app.py", line 228 in start,
  File "/opt/frigate/frigate/__main__.py", line 15 in <module>,
  File "/usr/lib/python3.8/runpy.py", line 87 in _run_code,
  File "/usr/lib/python3.8/runpy.py", line 194 in _run_module_as_main,

ThaNerd · March 10, 2021, 2:50pm

Anyone knows where this data comes from? It’s stamped on my snapshots in hassio.

2021-03-10_9-48-50

The time is off, but I looked and my docker PC as the proper time and my hassio has the proper time, so where does this come from?

noebl1 · March 10, 2021, 3:09pm

I saw this too, mine’s running in Kubernetes however. I think it looks like it may be defaulting to Zulu / GMT?

homeassistant.mi · March 10, 2021, 5:15pm

Hi
I have problem with my “Green” camera, i try many parameters to ffpmeg and nothing works. Can you maybe help me to configure this camera

Screenshot from 2021-03-10 18-12-47

i have config


cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
          roles:
            - rtmp
            - detect
    width: 1920
    height: 1080
    fps: 20

and in logs i see

frigate.app                    INFO    : Capture process started for front_door: 41
frigate.video                  INFO    : front_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : front_door: ffmpeg process is not running. exiting capture thread...
ffmpeg.front_door.detect       ERROR   : [flv @ 0x1ada200] Video codec hevc not compatible with flv
ffmpeg.front_door.detect       ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): Function not implemented
ffmpeg.front_door.detect       ERROR   : 
frigate.video                  INFO    : front_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : front_door: ffmpeg process is not running. exiting capture thread...
ffmpeg.front_door.detect       ERROR   : [flv @ 0x213a700] Video codec hevc not compatible with flv
ffmpeg.front_door.detect       ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): Function not implemented
ffmpeg.front_door.detect       ERROR   : 
frigate.video                  INFO    : front_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : front_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and

Video stream IS
Screenshot from 2021-03-10 18-14-47

Sorb · March 10, 2021, 5:42pm

Hi,
first of all: Thanks for this great Integration!
I have a similar looking issue like homeassistant.mi.

Background: I have a Reolink Lumus Cam, that doesn’t natively expose a RTSP feed. So I am using neolink to translate from the proprietary protocol to RTSP. In VLC the streams are working fine. In Frigate I get the following errors:

* Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
frigate.app                    INFO    : Camera processor started for garage: 38
frigate.app                    INFO    : Capture process started for garage: 40
frigate.video                  INFO    : garage: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate.video                  INFO    : garage: ffmpeg process is not running. exiting capture thread...
detector.coral                 INFO    : Starting detection process: 36
frigate.edgetpu                INFO    : Attempting to load TPU as usb:0
frigate.edgetpu                INFO    : TPU found
ffmpeg.garage.detect           ERROR   : [h264 @ 0x561971d75980] missing picture in access unit with size 22
ffmpeg.garage.detect           ERROR   : [h264 @ 0x561971d75980] no frame!
ffmpeg.garage.detect           ERROR   : [rtsp @ 0x561971d71ac0] decoding for stream 0 failed
ffmpeg.garage.detect           ERROR   : [rtsp @ 0x561971d71ac0] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
ffmpeg.garage.detect           ERROR   : Consider increasing the value for the 'analyzeduration' and 'probesize' options
ffmpeg.garage.detect           ERROR   : Guessed Channel Layout for Input Stream #0.1 : stereo
ffmpeg.garage.detect           ERROR   : Input #0, rtsp, from 'rtsp://10.0.0.2:8554/garage':
ffmpeg.garage.detect           ERROR   :   Metadata:
ffmpeg.garage.detect           ERROR   :     title           : Session streamed with GStreamer
ffmpeg.garage.detect           ERROR   :     comment         : rtsp-server
ffmpeg.garage.detect           ERROR   :   Duration: N/A, start: 1615396902.443125, bitrate: 512 kb/s
ffmpeg.garage.detect           ERROR   :     Stream #0:0: Video: h264, none, 90k tbr, 90k tbn, 180k tbc
ffmpeg.garage.detect           ERROR   :     Stream #0:1: Audio: pcm_s16be, 16000 Hz, stereo, s16, 512 kb/s
ffmpeg.garage.detect           ERROR   : [flv @ 0x561971d94080] dimensions not set
ffmpeg.garage.detect           ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
ffmpeg.garage.detect           ERROR   : Stream mapping:
ffmpeg.garage.detect           ERROR   :   Stream #0:0 -> #0:0 (copy)
ffmpeg.garage.detect           ERROR   :   Stream #0:1 -> #0:1 (copy)
ffmpeg.garage.detect           ERROR   :   Stream #0:0 -> #1:0 (h264 (native) -> rawvideo (native))
ffmpeg.garage.detect           ERROR   :     Last message repeated 1 times

The configuration for my camera looks like this:

detectors:
  coral:
    type: edgetpu
    device: 'usb:0'

mqtt:
  host: 10.0.0.2
  port: 1883
  topic_prefix: frigate
  client_id: <user>
  user: <user>
  password: <pass>

ffmpeg:
  global_args: 
    -hide_banner 
    -loglevel info
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

cameras:
  garage:
    ffmpeg:
      inputs:
        - path: 'rtsp://10.0.0.2:8554/garage'
          roles:
            - detect
            - rtmp
    width: 1920
    height: 1080
    fps: 5

objects:
  track:
    - person
    - cat
    - dog
    - bird

Any idea?