Local realtime person detection for RTSP cameras

[wjbeckett (Backslash)]

I have a question regarding notifications,
I’m currently getting notifications for object detections as soon as they are detected, which is amazing, however I’m getting multiple notifications at once. What I mean is When someone is standing at the door, I will get notifications every time they move slightly. Same with a car coming up the driveway, I’ll get multiple notifications as the car moves.

Is there a way that I can get the notification when the person or car is first detected, and not for the duration of the event?

The aim here is to have a notification as soon as a person or car is detected within my zones and send me an image of that person or car.

Any help is greatly appreciated!

[tmjpugh]

Add delay at end of automation Actions to prevent retriggers for, as an example, 30 seconds

delay:
  hours: 0
  minutes: 0
  seconds: 30
  milliseconds: 0

You also may need to add a tag into action for notification to allow overwrite of previous vs multiple display

[borodom]

USB Coral Stick changes PID/VID after Initialization. How do you mount it to your container ?

[surge919]

i use the --privileged flag.

inside frigate it shows up with lsusb as

2-1 1a6e:089a:0100 00 3.10 5000MBit/s 896mA 1IF ( ffffffd1ffffffb2ffffffdbffffffad)

Synology says it will not be supported and they don’t plan to bring back any support for USB devices outside of storage and UPS

but since Frigate does see something, I’m hoping someone can figure something out

[borodom]

The ffffffd1ffffffb2ffffffdbffffffad is a Google Coral stick ID after initialization. It works for me without any issue however I run it on DSM 6 not 7.

I use this code to run container:

docker run -d \
  --name frigate \
  --privileged \
  --restart=unless-stopped \
  --shm-size 2gb \
  --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \
  --device /dev/dri/renderD128 \
  -v /dev/bus/usb:/dev/bus/usb \
  -v /volume1/docker/frigate/config:/config:ro \
  -v /volume2/docker_data/frigate:/media/frigate:rw \
  -v /etc/localtime:/etc/localtime:ro \
  -e FRIGATE_RTSP_PASSWORD='<password>' \
  -p 5000:5000 \
  blakeblackshear/frigate:stable-amd64

Mind the

-v /dev/bus/usb:/dev/bus/usb

this way of mounting usb works for me without any issues. Also check if you gave right permissions to required folders.

I hope it helps.

[surge919]

I have the same settings

Unless there is something you really want from DSM 7, I’d stick with 6.

I don’t have enough storage to backup and reinstall 6 so I’m looking at a separate machine for Frigate

Here’s the output showing no coral detected

2021-07-08T20:22:19.448415177Z detector.coral                 INFO    : Starting detection process: 35
2021-07-08T20:22:19.448686076Z frigate.edgetpu                INFO    : Attempting to load TPU as usb
2021-07-08T20:22:19.455197218Z frigate.edgetpu                INFO    : No EdgeTPU detected.
2021-07-08T20:22:19.456100523Z Process detector:coral:
2021-07-08T20:22:19.478449900Z Traceback (most recent call last):
2021-07-08T20:22:19.478917683Z   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
2021-07-08T20:22:19.479110068Z     delegate = Delegate(library, options)
2021-07-08T20:22:19.479388598Z   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
2021-07-08T20:22:19.479557040Z     raise ValueError(capture.message)
2021-07-08T20:22:19.479837488Z ValueError
2021-07-08T20:22:19.480054156Z
2021-07-08T20:22:19.480109151Z During handling of the above exception, another exception occurred:
2021-07-08T20:22:19.480194064Z
2021-07-08T20:22:19.480318247Z Traceback (most recent call last):
2021-07-08T20:22:19.480507434Z   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
2021-07-08T20:22:19.480571709Z     self.run()
2021-07-08T20:22:19.480686076Z   File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
2021-07-08T20:22:19.480742155Z     self._target(*self._args, **self._kwargs)
2021-07-08T20:22:19.480865588Z   File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
2021-07-08T20:22:19.480924442Z     object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
2021-07-08T20:22:19.481059375Z   File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
2021-07-08T20:22:19.481114133Z     edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
2021-07-08T20:22:19.481272397Z   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
2021-07-08T20:22:19.481366660Z     raise ValueError('Failed to load delegate from {}\n{}'.format(
2021-07-08T20:22:19.481624925Z ValueError: Failed to load delegate from libedgetpu.so.1.0
2021-07-08T20:22:19.481781154Z
2021-07-08T20:22:29.767606628Z [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5636e7e29140] moov atom not found
2021-07-08T20:22:29.768297845Z /media/frigate/recordings/driveway-20210708162057.mp4: Invalid data found when processing input

[scottyep10]

I’m running the latest non-stable frigate tag, 0.9.0-92aa16c-amd64, and I was wondering what the new birdseye tab is supposed to do? I currently have 3 cameras fed into frigate, but the birdeyes page is blank with just the frigate logo showing. I realize this software isn’t finished yet and I’m taking a risk by running software that hasn’t been fully developed and tested. I’m more just curious what features are being added. Feel free to tell me to bug off and wait for the official release

[tmjpugh]

@scottyep10

and I was wondering what the new birdseye tab is supposed to do?

Shows camera with recent detection

On this page it doesn’t quite explain it but it does show related issues and discussion of feature within the issue

Release 0.9.0

[surge919]

@surge919

inside frigate it shows up with lsusb as

2-1 1a6e:089a:0100 00 3.10 5000MBit/s 896mA 1IF ( ffffffd1ffffffb2ffffffdbffffffad)

This may have been an error on my part. I md using –privileged but I don’t see anything when I try lsusb from within frigate

[scottyep10]

@tmjpugh

On this page it doesn’t quite explain it but it does show related issues and discussion of feature within the issue

Thanks for the link. I think if I want birdseye to always show something then I need to change the config mode from the default “objects” to “continuous”. I’ll give that a shot.

Edit: Continuous worked! Thanks again for the link

[surge919]

when using a coral device, how much does the CPU matter?

I have an older machine with an Intel Core 2 Duo E8400 ( 3.0GHz) and 4 GB RAM
How would that compare to running Frigate on a Pi4?

[danps]

Hi,
Does anyone know if it’s possible to manually trigger a full-frame object detection with Frigate? I understand that the designed behaviour is that frigate checks for motion, and only upon motion detection, then triggers object detection, and only for the area of the image which had motion. This is fantastic design from a CPU/TPU resource utilisation standpoint, but occasionally I’d like to be able to send the entire frame of video to the TPU for object detection. An example might be that when a car moves onto my driveway, Frigate detects it correctly as it was moving, however, if that car then parks, and is no longer moving, Frigate will no longer detect it. Any way to do this? Thanks

[Yves_IF (Yves)]

@blakeblackshear thanks for this great work.

I looking for a way, to integrate the Event-View in Lovelace.

I like to use it for other user, so the complete frigate-webpage is not an option. I tried with the iframe lovelace card but i just can display the whole page.

have someone an idea or a (similar) solution?

Thanks a lot

[danbutter]

Do you want it to look like that screen shot…in a table?
Or just see the last detected object from each cam?

[Juggler (SR)]

I’m setting up Frigate for the first time, transitioning from ZoneMinder. I have several Dahua cameras that allow for a sub stream. Reading through the documentation, it looks like it is best to do detect on a lower-resolution sub-stream, while the clip/record is on a higher stream.

My cameras allow me to create a H.264 sub-stream with a resolution of 640x480, with the mainstream at a full 1080p. Is this sufficient resolution for detection? Should I be doing the detection on the 1080p stream instead?

[tmjpugh]

@Juggler

My cameras allow me to create a H.264 sub-stream with a resolution of 640x480, with the mainstream at a full 1080p. Is this sufficient resolution for detection? Should I be doing the detection on the 1080p stream instead?

Yes. 640x480 is good for detect.

Take a look at this post

Local realtime person detection for RTSP cameras

In theory, yes. The models are mostly trained on square images, so when the detection runs, it will squish a rectangle into the square. I believe this will throw off the aspect ratio and decrease accuracy. Also of note, the fast models are all trained on 300x300 pixel images, and your region is resized to that before processing by tensorflow. I chose my camera resolution so that my smallest region would be as close to that size as possible. Because the images are resized, tensorflow will have a…

[Yves_IF (Yves)]

yes, i want it to look like the upper screenshot
(i know the possibility with the last screenshot i have it with the camera stream camera.cam-name_person)

[ukro]

Hi guyz,
stationary object is parked infront of camera. There is a lot of events. Why it doesnt see that the object doesnt exit the zone.It is making new events every 10seconds

detect:
  enabled: True
  max_disappeared: 25

[flyize]

I have a v2 Wyze cam. According to the docs, I’ve got the resolution wrong. Here’s what I’ve tried so far:

* w:1920 h:1080 - Completely green screen
* w:1920 h:1088 (An odd resolution that VLC showed) - This just gives me a green tinged extremely broken picture

What am I doing wrong?

edit: I had a typo

[danbutter]

Okay. That’s what I would have suggested since it has the same data, but if that isn’t what you want it isn’t helpful to you.
Sorry I can’t help.