Local realtime person detection for RTSP cameras

Frigate publishes to frigate/available=online in it’s on_connect callback. You should see a log message in Frigate of MQTT connected right before it sends it. The sensors in Home Assistant use that topic to determine whether or not the entity is available as a part of the normal “last will” mqtt spec. Either Frigate is not sending that message because it does not detect a reconnection for some reason, or the message is not sent to Home Assistant. Since they go unavailable, then Home Assistant must be receiving the frigate/available=offline message from your broker.

Thank you, this was super helpful! I’ll keep an eye out for the online / offline messages and go from there in debugging.

I know h265 playback is a browser issue, but are safari and ie 11 not compatible?

Please suggestion.
I have 10cameras i want use NODERED when object person is found make something.
How can i read person object from the cameras? What are the settings. As what i see in MQTT settings it saves only pictures.

UPDATE:
SO i found the events.
but

  "after": {
    "id": "1624653706.05705-jb0kpx",
    "camera": "CAM8",
    "frame_time": 1624653710.852007,
    "label": "person",

I need to check the ID and hold in memory and wait if it changes? and the object is person then do something?
If somebody can share his flow i would be very grateful :slight_smile:

UPDATE 1:
Solved it, if anybody need help, PM me.

Use the HACS integration for Frigate along with node-red-contrib-home-assistant-websocket. You’ll have everything you need.

1 Like

Yaaay! :tada: I have succeeded with installing Reolink RLC-810A cameras to Frigate NVR :star_struck:

Well, it isn’t working on full 4k resolution (3840x2160) because there is some error with image being “concentrated” to top 2 milimeters of the screen which I just reported back to support, but on 2560x1440 it works without errors in Frigate and also I don’t see any drops in the frames or cluttered image as it was before. You are asking what changed, right? So the change which Reolink did was switch of the main stream from h.265 to h.264. And now all issues are gone and I’m able to use susbstream for decode and main stream for record, clips and even rtmp. But let’s go step by step


What to do when you would like to use Reolink RLC-810A cameras in Frigate NVR:

  • write to Reolink support using Submit a request – Reolink Support. Ask for a new firmware which solve network drops of the connection to Blue Iris (that’s how they call it). My cameras are Hardware No. IPC_523128M8MP and I got firmware file of version v3.0.0.236_21040831 called IPC_523128M8MP.236_21040831.RLC-810A.IMX415.8MP.REOLINK.pak.
  • When you get the firmware file proceed with uploading the firmware to the camera. I suggest update the firmware with Update Configure File checked (WARNING: this will restore all configuration in the camera to factory defaults and you’ll lose all configuration!), because some options are not visible anymore in the new firmware so I believe it’s better to reset them to defaults.
  • Set new settings to the camera > connect to camera using IP address (default login credentials are “admin” and no password), navigate to Settings (gear icon in top right corner):
    • Camera
      • Display
        • Date & Time: Hide
        • Watermark: disabled
      • Stream
        • Clear
          • resolution: 2560*1440 (after changing this you’ll need to login because camera will be rebooted)
          • FPS: 25
          • Max bitrate: 8192
        • Fluent
          • resolution: 640*360
          • FPS: 10
          • Max bitrate: 512
      • Audio and Light
        • Record Audio: enabled
      • Info
        • Camera name:
    • Surveliance - here I disable all
    • System
      • User management - change password for admin
  • The camera will reboot and it will put you back to login screen. Here select Stream type: Clear and login. Did you notice there isn’t any Balanced in the setting anymore? Did you notice you can view Clear stream in the browser? That’s because the codec for main stream was changed from h.265 to h.264. Now all works much more smoother :slight_smile: We only need to wait for Reolink to fix the 4k resolution. But let’s continue

  • Now in frigate.yml you can set something like this:
    cameras:  
      camera_name:
        ffmpeg:
          inputs:
            - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
              roles:
                - detect
            - path: rtsp://admin:[email protected]:554/
              roles:
                - record
                - clips
                - rtmp
          global_args: -hide_banner -loglevel info
          output_args:
            record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #removed -an parameter to enable audio
            clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #removed -an parameter to enable audio
        width: 640
        height: 360
        fps: 5
        objects:
          track:
            - person
            - car
        clips:
          enabled: True
          pre_capture: 5
          post_capture: 5
        rtmp:
          enabled: True
    
  • Restart Frigate NVR add-on in Home Assistant Supervisor and verify the log

And that’s it. Now thanks to the new firmware the Reolink RLC-810A is working in Frigate NVR better than ever before because recording and clips can be played directly from the browser thanks to the h.264 codec. The only issue is that the 4k resolution of main stream is not working but I hope they will solve it immediatelly when more of us will be requesting this to be fixed. So please write them feedback that we love change of the codec to h.264 and ask them to fix that 4k resolution :slight_smile:

Update: I’ve been in discussion with Reolink support every day and their senior engineer confirmed it’s not possible to have 8MP resolution with h.264.
So it’s either h.264 with all-browser view on lower resolution with RTMP feature in Frigate working flawlessly or h.265 on full resolution. I choose simplicity and better support in browsers over resolution.

10 Likes

Thank you.
Nevertheless the websocket is reading but the result is in the same spot. So does it have some solution if the result is the same dont trigger?
Or i need to save the ID and check for changes?

UPDATE:1
Solved it, it is actualy seeing if the message was already sent and not sending again.
If anybody need help, PM me.

@SpikeyGG, although it was a bit of a journey (I had to watch a few videos), I ended up using the authentication with Authelia. There are quite a few steps with different components, however, I can see the Frigate UI and view clips, screenshots, events, etc.

Here is the guide I used from Ibracorp - https://youtu.be/kw_pohbKE3Y

1 Like

Good post @palito . I ended up having to use RTMP stream for my Reolink RLC-410-5MP. I do like how you have worked out another method to keep your model using RTSP and stable.

That’s really awesome, thanks for sharing it. It would be cool to wrap my whole system around some consistent authentication platform like that. I’ve saved the video so I can look into doing it down the road.

No worries. It took me a day or so watching the videos and then switching it over to live. Works well so far and gives an extra layer of protection
 With NGINX you can not have to expose ports to the internet and also make things go through cloudflare.

My next step is to see if I can make HA go through it without any issues :wink:

Hi!

Just installed one Reolink to my summer cottage. My car has been parked to its location and Frigate gives me constantly events from my car.

image

I’d like to get events when car is arriving or leaving but not from the car which is parked. How to do this? If I put mask to that location then Frigate doesn’t nitice if somebody is walking from that area?

1 Like

Mask bottom half of where car normally park (tires to bottom windows)may reduce confidence to prevent capture

This should still allow person detect

Hi, can u share how did u make coral mount to docker in synology

Thx, @tmjpugh . However it would be really awesome that we could define zones where we could combine motion + object detection together.

Hi guys,

quick question, I have set some masks on the main feed (1080p) and then realized that I can use the sub-stream for the “detect”, but when I switched it, the mask I created earlier apply to this one and because of the lower resolution it covers way more that I want it to be (it lost all the possible motion area).
Is there a way to make this mask for the higher resolution? or maybe I should create the mask based on the lower resolution?

Thanks!
Didi

Hey Y’all,
finaly i setup today a perimeter security with Frigate.
If i turn on the perimeter and some person object apears around the house, it turn on the 12V siren with ESP01 tasmota relay :slight_smile:. I have 10s alarm with 1minutes pause delay. Works like a charm !!! @blakeblackshear LiveLongAndProsper ! <3

1 Like

Why the score differ in one event?

And clip was generated, but i can play only from nas(because of h265,changed it to h264)
So when i examen the clip, close to the dryer was my wife and trolley, she was riding the troley. And was infront of the dryer. That is strange why it saved the dryer around 10 seconds after she passed by with a babytroley

I have issued a ticket on github

Ok I was able to catch it this time with a rolling upgrade watching thru MQTT Explorer. Frigate had available=online before the upgrade. The EMQX (Mosquitto broker) upgrade started, and I saw Frigate go available=offline, I see two messages about 2 mins apart (guessing it rolled to a new node, the that node rolled so it went to another node as it was then upgraded): frigate.mqtt INFO : MQTT connected

It stayed at available=offline until I redeployed the frigate pod. I’m going to see if it’s possible to reproduce with Mosquitto and if the broker bounces twice within a couple minutes if that’s the key or something with EQMX that Frigate doesn’t like on a reconnect.

Anyone know why below automation magically stop working after server restart?

Trace shows “fail” for template portion matching camera but automation should match camera in MQTT message.
I provide a sample message below that fails for this.

AUTOMATION

- id: '1621092684971'
  alias: NOTIFY_FrigateEntryGateMotionInOut
  description: ''
  trigger:
  - platform: mqtt
    topic: frigate/events
  condition:
  - condition: state
    entity_id: automation.gate_autoopenvisitormode_2
    state: 'off'
  - condition: or
    conditions:
    - condition: template
      value_template: '{{ trigger.payload_json["after"]["current_zones"] == ["outside"]
        }}'
    - condition: template
      value_template: '{{ trigger.payload_json["after"]["entered_zones"] == ["outside"]
        }}'
    - condition: template
      value_template: '{{ trigger.payload_json["after"]["current_zones"] == ["inside"]
        }}'
    - condition: template
      value_template: '{{ trigger.payload_json["after"]["entered_zones"] == ["inside"]
        }}'
  - condition: template
    value_template: '{{ trigger.payload_json["after"]["camera"] == ["Frigate_Gate02"]
      }}'
  action:
  - service: notify.mobile_app_iphone
    data:
      message: A {{trigger.payload_json["after"]["label"]}} is {{ trigger.payload_json["after"]["current_zones"]
        }} the Gate.{{trigger.payload_json["after"]["top_score"]}}
      data:
        tag: NOTIFY_EntryGateMotionInOut
        image: https://server.com/api/frigate/notifications/{{ trigger.payload_json["after"]["id"]
          }}/thumbnail.jpg?for=android
        actions:
        - action: OPEN_ENTRY_GATE
          title: Open Gate
          destructive: true
        - action: URI
          title: APP
          uri: /lovelace/3

MQTT MESSAGE

{
    "before": {
        "id": "1624898615.82943-xmyv4l",
        "camera": "Frigate_Gate02",
        "frame_time": 1624898615.82943,
        "snapshot_time": 0,
        "label": "car",
        "top_score": 0,
        "false_positive": true,
        "start_time": 1624898615.82943,
        "end_time": null,
        "score": 0.82421875,
        "box": [
            464,
            52,
            597,
            149
        ],
        "area": 12901,
        "region": [
            361,
            0,
            661,
            300
        ],
        "current_zones": [],
        "entered_zones": []
    },
    "after": {
        "id": "1624898615.82943-xmyv4l",
        "camera": "Frigate_Gate02",
        "frame_time": 1624898615.936224,
        "snapshot_time": 1624898615.936224,
        "label": "car",
        "top_score": 0.82421875,
        "false_positive": false,
        "start_time": 1624898615.82943,
        "end_time": null,
        "score": 0.83203125,
        "box": [
            467,
            49,
            599,
            149
        ],
        "area": 13200,
        "region": [
            380,
            0,
            680,
            300
        ],
        "current_zones": [
            "outside"
        ],
        "entered_zones": [
            "outside"
        ]
    },
    "type": "new"
}

EDIT
Figured it out after I posted it.
I forget NO bracket was needed around camera name

“camera”
NOT [“camera”]

1 Like