Local realtime person detection for RTSP cameras

dtaylorl · June 24, 2021, 12:17am

Okay, just tried that. Same error

Edit: just got it to work. Not really sure what I did, besides trying to put the config file in a different directory. Now my mqtt isn’t configured properly so I’ll see if I can solve that.

blakeblackshear · June 24, 2021, 12:21pm

Frigate publishes to frigate/available=online in it’s on_connect callback. You should see a log message in Frigate of MQTT connected right before it sends it. The sensors in Home Assistant use that topic to determine whether or not the entity is available as a part of the normal “last will” mqtt spec. Either Frigate is not sending that message because it does not detect a reconnection for some reason, or the message is not sent to Home Assistant. Since they go unavailable, then Home Assistant must be receiving the frigate/available=offline message from your broker.

noebl1 · June 25, 2021, 12:04am

Thank you, this was super helpful! I’ll keep an eye out for the online / offline messages and go from there in debugging.

Danzal · June 25, 2021, 7:08pm

I know h265 playback is a browser issue, but are safari and ie 11 not compatible?

ukro · June 25, 2021, 8:18pm

Please suggestion.
I have 10cameras i want use NODERED when object person is found make something.
How can i read person object from the cameras? What are the settings. As what i see in MQTT settings it saves only pictures.

UPDATE:
SO i found the events.
but

  "after": {
    "id": "1624653706.05705-jb0kpx",
    "camera": "CAM8",
    "frame_time": 1624653710.852007,
    "label": "person",

I need to check the ID and hold in memory and wait if it changes? and the object is person then do something?
If somebody can share his flow i would be very grateful

UPDATE 1:
Solved it, if anybody need help, PM me.

Artekus · June 25, 2021, 9:07pm

Use the HACS integration for Frigate along with node-red-contrib-home-assistant-websocket. You’ll have everything you need.

palito · June 25, 2021, 9:41pm

Yaaay! I have succeeded with installing Reolink RLC-810A cameras to Frigate NVR

Well, it isn’t working on full 4k resolution (3840x2160) because there is some error with image being “concentrated” to top 2 milimeters of the screen which I just reported back to support, but on 2560x1440 it works without errors in Frigate and also I don’t see any drops in the frames or cluttered image as it was before. You are asking what changed, right? So the change which Reolink did was switch of the main stream from h.265 to h.264. And now all issues are gone and I’m able to use susbstream for decode and main stream for record, clips and even rtmp. But let’s go step by step…

What to do when you would like to use Reolink RLC-810A cameras in Frigate NVR:

write to Reolink support using Submit a request – Reolink Support. Ask for a new firmware which solve network drops of the connection to Blue Iris (that’s how they call it). My cameras are Hardware No. IPC_523128M8MP and I got firmware file of version v3.0.0.236_21040831 called IPC_523128M8MP.236_21040831.RLC-810A.IMX415.8MP.REOLINK.pak.
When you get the firmware file proceed with uploading the firmware to the camera. I suggest update the firmware with Update Configure File checked (WARNING: this will restore all configuration in the camera to factory defaults and you’ll lose all configuration!), because some options are not visible anymore in the new firmware so I believe it’s better to reset them to defaults.
Set new settings to the camera > connect to camera using IP address (default login credentials are “admin” and no password), navigate to Settings (gear icon in top right corner):
- Camera
  - Display
    - Date & Time: Hide
    - Watermark: disabled
  - Stream
    - Clear
      - resolution: 2560*1440 (after changing this you’ll need to login because camera will be rebooted)
      - FPS: 25
      - Max bitrate: 8192
    - Fluent
      - resolution: 640*360
      - FPS: 10
      - Max bitrate: 512
  - Audio and Light
    - Record Audio: enabled
  - Info
    - Camera name:
- Surveliance - here I disable all
- System
  - User management - change password for admin
The camera will reboot and it will put you back to login screen. Here select Stream type: Clear and login. Did you notice there isn’t any Balanced in the setting anymore? Did you notice you can view Clear stream in the browser? That’s because the codec for main stream was changed from h.265 to h.264. Now all works much more smoother We only need to wait for Reolink to fix the 4k resolution. But let’s continue…

Now in frigate.yml you can set something like this:

cameras:  
  camera_name:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
          roles:
            - detect
        - path: rtsp://admin:[email protected]:554/
          roles:
            - record
            - clips
            - rtmp
      global_args: -hide_banner -loglevel info
      output_args:
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #removed -an parameter to enable audio
        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #removed -an parameter to enable audio
    width: 640
    height: 360
    fps: 5
    objects:
      track:
        - person
        - car
    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
    rtmp:
      enabled: True

Restart Frigate NVR add-on in Home Assistant Supervisor and verify the log

And that’s it. Now thanks to the new firmware the Reolink RLC-810A is working in Frigate NVR better than ever before because recording and clips can be played directly from the browser thanks to the h.264 codec. The only issue is that the 4k resolution of main stream is not working but I hope they will solve it immediatelly when more of us will be requesting this to be fixed. So please write them feedback that we love change of the codec to h.264 and ask them to fix that 4k resolution

Update: I’ve been in discussion with Reolink support every day and their senior engineer confirmed it’s not possible to have 8MP resolution with h.264.
So it’s either h.264 with all-browser view on lower resolution with RTMP feature in Frigate working flawlessly or h.265 on full resolution. I choose simplicity and better support in browsers over resolution.

ukro · June 26, 2021, 7:57am

Thank you.
Nevertheless the websocket is reading but the result is in the same spot. So does it have some solution if the result is the same dont trigger?
Or i need to save the ID and check for changes?

UPDATE:1
Solved it, it is actualy seeing if the message was already sent and not sending again.
If anybody need help, PM me.

DrSpaldo · June 27, 2021, 2:47am

@SpikeyGG, although it was a bit of a journey (I had to watch a few videos), I ended up using the authentication with Authelia. There are quite a few steps with different components, however, I can see the Frigate UI and view clips, screenshots, events, etc.

Here is the guide I used from Ibracorp - https://youtu.be/kw_pohbKE3Y

DrSpaldo · June 27, 2021, 3:01am

Good post @palito . I ended up having to use RTMP stream for my Reolink RLC-410-5MP. I do like how you have worked out another method to keep your model using RTSP and stable.

SpikeyGG · June 27, 2021, 3:05am

That’s really awesome, thanks for sharing it. It would be cool to wrap my whole system around some consistent authentication platform like that. I’ve saved the video so I can look into doing it down the road.

DrSpaldo · June 27, 2021, 4:26am

No worries. It took me a day or so watching the videos and then switching it over to live. Works well so far and gives an extra layer of protection… With NGINX you can not have to expose ports to the internet and also make things go through cloudflare.

My next step is to see if I can make HA go through it without any issues

mcpaha · June 27, 2021, 10:20am

Hi!

Just installed one Reolink to my summer cottage. My car has been parked to its location and Frigate gives me constantly events from my car.

I’d like to get events when car is arriving or leaving but not from the car which is parked. How to do this? If I put mask to that location then Frigate doesn’t nitice if somebody is walking from that area?

tmjpugh · June 27, 2021, 4:49pm

Mask bottom half of where car normally park (tires to bottom windows)may reduce confidence to prevent capture

This should still allow person detect

poornachandratejasvi · June 27, 2021, 5:34pm

Hi, can u share how did u make coral mount to docker in synology

mcpaha · June 27, 2021, 5:37pm

Thx, @tmjpugh . However it would be really awesome that we could define zones where we could combine motion + object detection together.

didi767 · June 27, 2021, 6:18pm

Hi guys,

quick question, I have set some masks on the main feed (1080p) and then realized that I can use the sub-stream for the “detect”, but when I switched it, the mask I created earlier apply to this one and because of the lower resolution it covers way more that I want it to be (it lost all the possible motion area).
Is there a way to make this mask for the higher resolution? or maybe I should create the mask based on the lower resolution?

Thanks!
Didi

ukro · June 27, 2021, 9:15pm

Hey Y’all,
finaly i setup today a perimeter security with Frigate.
If i turn on the perimeter and some person object apears around the house, it turn on the 12V siren with ESP01 tasmota relay . I have 10s alarm with 1minutes pause delay. Works like a charm !!! @blakeblackshear LiveLongAndProsper ! <3

ukro · June 28, 2021, 1:29pm

Why the score differ in one event?

And clip was generated, but i can play only from nas(because of h265,changed it to h264)
So when i examen the clip, close to the dryer was my wife and trolley, she was riding the troley. And was infront of the dryer. That is strange why it saved the dryer around 10 seconds after she passed by with a babytroley…
I have issued a ticket on github

noebl1 · June 28, 2021, 5:51pm

Ok I was able to catch it this time with a rolling upgrade watching thru MQTT Explorer. Frigate had available=online before the upgrade. The EMQX (Mosquitto broker) upgrade started, and I saw Frigate go available=offline, I see two messages about 2 mins apart (guessing it rolled to a new node, the that node rolled so it went to another node as it was then upgraded): frigate.mqtt INFO : MQTT connected

It stayed at available=offline until I redeployed the frigate pod. I’m going to see if it’s possible to reproduce with Mosquitto and if the broker bounces twice within a couple minutes if that’s the key or something with EQMX that Frigate doesn’t like on a reconnect.