Local realtime person detection for RTSP cameras

It honestly appears to run really well apart from those disconnects, so I can’t complain at all!

has anybody gotten this to run with like an Nvidia card trying to figure it out

Installed 0.8.0-beta1 and updated my config.yml.

No local port on 5000

7927e60dc1f9:/opt/frigate$ ss -tlpn
State                Recv-Q               Send-Q                             Local Address:Port                              Peer Address:Port              Process               
LISTEN               0                    4096                                  127.0.0.11:34643                                  0.0.0.0:*                                       
LISTEN               0                    128                                    127.0.0.1:5001                                   0.0.0.0:*                  users:(("python3",pid=1,fd=52))

There are a lot of breaking changes with the 0.8.0 release. I found it easier just to rebuild my configuration. The 0.7.X configuration will not work.

Fount the issue… As I start the container as non-root user, nginx could not be started. A few changes to the compose file and it starts now.

@blakeblackshear here’s an easier method in the MQTT binary sensor that gives the best of both worlds:

Take this, which worked with the original ON/OFF payloads:

- platform: mqtt
  name: "Motion BYC Person"
  state_topic: "frigate/byc/person"
  payload_on: "ON"
  payload_off: "OFF"
  device_class: motion
  availability_topic: "frigate/available"

…and change it to this:

- platform: mqtt
  name: "Motion BYC Person"
  state_topic: "frigate/byc/person"
  payload_on: "ON"
  payload_off: "OFF"
  device_class: motion
  availability_topic: "frigate/available"
  value_template: >
    {% if value == "0" or value == '' %}
      OFF
    {% else %}
      ON
    {% endif %}

Tested it quite a bit and it seems to work fine. If it’s NULL or 0, the binary sensor stays off. Anything else, e.g. 1 or 10 objects detected, and it turns to on.

FYI your work on Frigate has been outstanding. I basically replaced the brains in Blue Iris with Frigate via MQTT integration. It works amazingly well. Thanks much!

1 Like

@blakeblackshear my issue with camera’s jumping around keeps coming back. If I watch the actual source rtsp stream it’s good, but through frigate it jumps around to different streams.

I uploaded a quick clip to show what i mean. This is while watching the re-broadcast rtmp stream, I get the same if I hit IP_address:5000/camera_name

Interesting. I have no idea why this would be happening. Frigate does zero processing of the RTSP feed before re-broadcasting it as RTMP. The ffmpeg process streams that directly to nginx without even decoding, so it is basically just a pipe for what ffmpeg read from the rtsp stream. Not a line of code touches that stream.

The fact that it happens at the mjpeg endpoint is also interesting because that is what frigate reads from ffmpeg’s other output. I’m guessing you are using the same feed for both rtmp and detect.

Because this happens in both places, I don’t believe the issue originates in the actual frigate code. Somehow the ffmpeg process is receiving frames from 2 different cameras intermittently.

What type of cameras are these and what does your RTSP input look like?

They are Reolink B800 cameras with the RTSP stream coming from the NVR. And yes, same feed for rtmp, detect and clips. I actually noticed this happening a few days ago with the stable release which is what prompted me to move to the beta.

The RTSP link is: rtsp://user:[email protected]:XXX/h264Preview_02_sub

While watching the feed bouncing around through the mjpeg endpoint on my laptop, I loaded the same RTSP link into Tiny Cam Pro on my phone and it’s solid. I then created a camera entity with the same RTSP link and it’s also solid. If I reboot frigate enough times, it works as it should until I edit anything in the config again.

What if you setup the same RTSP feed as an ffmpeg camera directly in HomeAssistant? Editing the config on frigate won’t really change anything other than cause ffmpeg to restart. I wouldn’t be surprised if running ffmpeg anywhere with that RTSP endpoint shows the same behavior.

That’s what I mean when I said I created a camera entity.

camera:
   - platform: ffmpeg
     name: Reolink1
     input: !secret reolink1

And it’s a solid stream.

?? Now what if you’re using a GPU like an Nvidia card would you put GPU instead of CPU and then PCI if you have the answer thanks

Sorry for the delayed response, but yepp, just enabled save clips, added the appropriate roles, and that’s it.

The problem wound up being - I had save_clips enabled, but didn’t have the cameras set up with the ‘clips’ role.

Below is my config so you can see what I needed to do to make it work. 1xReolink RLC-420, 2xReolink RLC-410

Hi! I had similar problem with Reolink NVR. This was happening when using RTSP streams. When I changed to RTMP streams issue went away.

When the stream jumps is it the actual live stream? I’ve seen an issue where the HA stream of a ‘live’ feed is actually a view of the past.

I’m getting a lot of errors in the terminal output when running frigate:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x555725053140] moov atom not found
/tmp/cache/driveway-20201216144516.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: driveway-20201216144516.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5555881a1140] moov atom not found
/tmp/cache/backyard-20201216144514.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: backyard-20201216144514.mp4
watchdog.driveway              INFO    : Terminating the existing ffmpeg process...
watchdog.driveway              INFO    : Waiting for ffmpeg to exit gracefully...
watchdog.backyard              INFO    : Terminating the existing ffmpeg process...
watchdog.backyard              INFO    : Waiting for ffmpeg to exit gracefully...

Any idea what that is all about??

Those are related to the process that checks the cached video files used for clips. Frigate writes 10 second segments directly from the camera to disk. Before assembling a clip from those cached segments, it uses ffprobe to check the integrity of the mp4 file. The errors you are seeing are from that ffprobe check. For some reason, those cache segments have invalid data. Most of the time, it is because the ffmpeg process saving them exited before it was finished writing the file (ie. the “Terminating the existing ffmpeg process…”).

Is it anything I can fix? Or is it normal?

It is expected to happen when ffmpeg loses connection to your camera. If you can get the camera stream to stabilize, that series of errors should go away. I would try setting to ffmpeg logs on one camera to something higher than fatal to see why ffmpeg is exiting.

Great, thanks for that. Its a camera I actually wasn’t planning on monitoring with Frigate and was just using it for testing purposes so I’ll probably just remove it from the config and be on my way. Although all my cameras are the same and that particular one was the only one giving me issues. If I have the same problem with the others I’ll do as you suggest.

Thanks for the great program (yes, I still call them that and not apps)!