Local realtime person detection for RTSP cameras

That sucks, that definitely doesn’t make it easy! It may be worth trying one of the amazon adapters before hacking into wiring. I would assume using the coral at usb2 must also limit its FPS processing as well as interface speed?

Not that I noticed, but it’s hard to tell because The Coral would keep shutting down on the USB3 card and I only have 2 cameras currently.

you wrote you have AMD CPU this case you dont have Intel Quick sync as it is not an Intel CPU. I have never used AMD CPU but not bad if you managed to run all that on that 3.2GHz 8core AMD. Also excellent low power consumption. Yep, traditional gaming GPUs will need 150-250 watt power, thats why i am leaning towards low consumption solutions like xavier nx.

Recording clips …
Is it possible to enable (disable) recording clips via MQTT?
I’d like to record clips during night and not during day. Is there any way to control it?
Thanks

Not currently. Please open an issue on Github.

Hi all,
When i start Frigate, the log is showing this. " frame 1602032964.611396 is not in memory store."…

What does that mean?

Check your camera fps at /debug/stats

You may need to set your fps on the camera if frigate shows it is higher than you expect.

I have my cameras set at streaming 15 fps. And in frigate config fps: 5. What do you mean set my fps on the camera?

Usually those messages mean that the framerate frigate is detecting is high and you are running out of memory to store all the extra frames. What does /debug/stats show?

Past duration 0.833763 too large 125928kB time=00:00:10.60 bitrate=97321.0kbits/s dup=20 drop=40 speed=1.13x
Past duration 0.746208 too large 48708kB time=00:00:04.10 bitrate=97321.0kbits/s dup=20 drop=8 speed=1.47x
Past duration 0.822884 too large 80784kB time=00:00:08.60 bitrate=76951.5kbits/s dup=0 drop=31 speed=1.24x
Past duration 0.988441 too large 36828kB time=00:00:03.10 bitrate=97321.0kbits/s dup=19 drop=2 speed=2.01x
Input #0, rtsp, from ‘rtsp://admin:Elizabeth1980#@10.0.0.208:554/cam/realmonitor?channel=1&subtype=1’:=1.81x
Metadata:
title : Media Server
Duration: N/A, start: 1602039688.937956, bitrate: N/A
Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 704x576, 15 fps, 25.08 tbr, 90k tbn, 30 tbc
Stream mapping:= 11 q=-0.0 size= 173448kB time=00:00:14.60 bitrate=97321.0kbits/s dup=20 drop=61 speed=1.13x
Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[swscaler @ 0x5582aca15c20] deprecated pixel format used, make sure you did set range correctly76 speed= 1.1x
Output #0, rawvideo, to ‘pipe:’:
Metadata:
title : Media Server
encoder : Lavf57.83.100
Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 704x576, q=2-31, 97320 kb/s, 10 fps, 10 tbn, 10 tbc
Metadata:
encoder : Lavc57.107.100 rawvideo

Hope this helps…

The logs alone don’t tell me much. Please open a bug report on github with all the info in the template. It is hard to troubleshoot with bits and pieces of information here.

How is v0.6.0 RC3 treating everybody? I’m still on 0.5.0, but have some time and considering upgrading.

Is there an lxc version of frigate?

I’ve recently migrated to proxmox and would love to run it in its own lxc container but have no idea how. are there any step by steps around?

Hi, can you clarify how you managed to have coral work in LXC?

I have added this into LXC conf:

lxc.cgroup.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb/002/002 dev/bus/usb/002/002 none bind,optional,create=file

I can see the coral in the docker LXC:

root@docker:/opt/frigate/config# lsusb 
Bus 002 Device 002: ID 18d1:9302 Google Inc. 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 8087:0a2b Intel Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

However, it seems frigate can’t detect it:

2020-10-11T21:07:51.979143188Z Received: 15
2020-10-11T21:07:52.082771075Z z5: stop event set. exiting capture thread...
2020-10-11T21:07:52.083120619Z z5: frame 1602450472.082454 is not in memory store.
2020-10-11T21:07:55.790456947Z Exiting watchdog...
2020-10-11T21:07:55.823678925Z Exiting event processor...
2020-10-11T21:08:01.892273982Z Exiting object processor...
2020-10-11T21:08:07.720776592Z On connect called
2020-10-11T21:08:08.731079236Z ffprobe -v panic -show_error -show_streams -of json "rtsp://frigate:[email protected]:554/cam/realmonitor?channel=1&subtype=2"
2020-10-11T21:08:08.731838014Z Starting detection process: 23
2020-10-11T21:08:08.732171396Z Attempting to load TPU as usb
2020-10-11T21:08:14.831658215Z {'streams': [{'index': 0, 'codec_name': 'h264', 'codec_long_name': 'H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10', 'profile': 'High', 'codec_type': 'video', 'codec_time_base': '1/10', 'codec_tag_string': '[0][0][0][0]', 'codec_tag': '0x0000', 'width': 1920, 'height': 1080, 'coded_width': 1920, 'coded_height': 1080, 'has_b_frames': 0, 'sample_aspect_ratio': '1:1', 'display_aspect_ratio': '16:9', 'pix_fmt': 'yuvj420p', 'level': 40, 'color_range': 'pc', 'color_space': 'bt709', 'color_transfer': 'bt709', 'color_primaries': 'bt709', 'chroma_location': 'left', 'field_order': 'progressive', 'refs': 1, 'is_avc': 'false', 'nal_length_size': '0', 'r_frame_rate': '100/1', 'avg_frame_rate': '5/1', 'time_base': '1/90000', 'start_pts': 108000, 'start_time': '1.200000', 'bits_per_raw_sample': '8', 'disposition': {'default': 0, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}}]}
2020-10-11T21:08:14.832154454Z Creating ffmpeg process...
2020-10-11T21:08:14.832163900Z ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://frigate:[email protected]:554/cam/realmonitor?channel=1&subtype=2 -f rawvideo -pix_fmt rgb24 pipe:
2020-10-11T21:08:14.838795918Z Camera_process started for z5: 37
2020-10-11T21:08:14.840259151Z Starting process for z5: 37
2020-10-11T21:08:14.854397028Z  * Serving Flask app "detect_objects" (lazy loading)
2020-10-11T21:08:14.854420228Z  * Environment: production
2020-10-11T21:08:14.854425119Z    WARNING: This is a development server. Do not use it in a production deployment.
2020-10-11T21:08:14.854530571Z    Use a production WSGI server instead.
2020-10-11T21:08:14.854573983Z  * Debug mode: off
2020-10-11T21:08:33.788384954Z Attempting to load TPU as pci:0
2020-10-11T21:08:33.791101215Z No EdgeTPU detected. Falling back to CPU.

This is what debug shows:

{"coral":{"detection_start":0.0,"fps":0.0,"inference_speed":10.0,"pid":23},"plasma_store_rc":null,"z5":{"camera_fps":5.0,"detection_fps":0.0,"ffmpeg_pid":35,"frame_info":{"detect":1602450595.734781,"process":0.0,"read":1602450595.734781},"pid":37,"process_fps":5.0,"read_start":0.0,"skipped_fps":0.0}}

New release candidate!

Changes:

  • Raspberry Pi 3/4 Support
  • Update ffmpeg to 4.3.1
  • Support multiple coral devices
  • Reduce CPU usage of ffmpeg and motion detection
  • Replace underlying memory store with shared_memory and update to Python 3.8
  • Smaller docker images due to switch to multistage builds
  • shm-size parameter no longer necessary for the majority of scenarios

Breaking Changes:

  • MQTT snapshots no longer draw regions or bounding boxes
  • Default output_args for cameras have changed. If you specified custom output parameters, you will need to update.
  • Updated config for specifying coral device. See new detectors section in the example config.
  • New json structure at /debug/stats to handle multiple detectors

Images:

  • x86: blakeblackshear/frigate:0.7.0-rc1
  • Raspberry Pi 3/4 (32bit): blakeblackshear/frigate:0.7.0-rc1-armv7hf
  • Raspberry Pi 4 (64bit): blakeblackshear/frigate:0.7.0-rc1-arm64
6 Likes

Try passing the entire bus 002 to the container instead.

lxc.cgroup.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb/002 dev/bus/usb/002 none bind,optional,create=dir

tried without luck. device still visible thru lsusb inside LXC, but frigate still not able to use it.

Just upgraded to 0.7.0 rc2 and saw a a nice reduction in cpu on both instances of frigate. I did notice the graph for coral_fps and interface time no longer work and I see this is in the logs.

TemplateError(‘UndefinedError: ‘mappingproxy object’ has no attribute ‘coral’’) while processing template ‘Template("{{ states.sensor.frigate_debug.attributes[“coral”][“fps”] }}")’ for attribute ‘_state’ in entity ‘sensor.frigate_coral_fps’

Error while processing template: Template("{{ states.sensor.frigate_debug.attributes[“coral”][“fps”] }}")

That is listed in the breaking changes. The structure of that endpoint changed to accommodate multiple corals for the same frigate instance.

I did see that JSON was updated, im just not sure what i should be doing to fix it.