Local realtime person detection for RTSP cameras

Interesting… I’ll take a look. The broker shows it connected and I see the messages coming in with MQTT Explorer so it’s had me perplexed. Does the integration have it’s own MQTT Client or going through the HA MQTT Client? Thanks!

EDIT: Interesting only a couple messages came across from this. No errors, very strange. Possibly is an issue with the broker.

Does anyone know if events are removed from database after restart?

Ive been rebooting a lot recently and notice that all previous events are missing after reboot. Is there a way to get these back or make it more persistent?

I did check that db files are mounted in docker so that not an issue.

Greetings,
pls help. I finally was able to install coral pci to HASSOS but the frigate addon doesnt see it

frigate.edgetpu                INFO    : Attempting to load TPU as pci:0
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0

HASSOS
lspci

06:10.0 Class 0880: 1ac1:089a

UPDATE:
Sorted out by adding this lines:

detectors:
  coral_pci:
    type: edgetpu
    device: pci

Successfully installed Frigate on HassOS, but since I’m running Home Assistant in VirtualBox, I’m trying to do a Docker install to take better advantage of hardware acceleration. Unfortunately, I’m finding Docker a bit confusing. I’m on Debian, and have successfully installed Docker Engine and Docker Compose, but I’m having trouble getting my Compose file right. Am I correct that in this section:

    volumes:
      - /etc/localtime:/etc/localtime:ro
      - <path_to_config_file>:/config/config.yml:ro
      - <path_to_directory_for_media>:/media/frigate

That ‘<path_to_config_file>’ and ‘<path_to_directory_for_media>’ should be replaced with something else? So far anything I’ve tried has thrown some sort of error about finding the config file. My Compose file is at ~/frigate/docker-compose.yml and my config file is at ~/frigate/config/config.yml

Yes

Replace with the location the file or folder can be found on the server, like /frigate/config/config.yml

Make sure Linux permissions allow frigate access to files or folder.

Post error msgs for help

Thanks. And does the right side of the equation stay as is? (i.e. /config/config.yml)

Error message I’m getting is

Error parsing config: [Errno 21] Is a directory: '/config/config.yml'

I’ve definitely got a config.yml file, not just a directory.

@blakeblackshear I was able to reproduce it, but I don’t understand it. I use the free version of EMQX for messaging/MQTT (it’s an enterprise level messaging server), and have a cluster inside my kubernetes cluster. Inside this cluster there’s a few nodes running in different pods on different k8s nodes, and the end point is set up to be able to connect to any of the nodes in the cluster with the same name/IP. So I can do a rolling upgrade and the clients may have a momentary very brief blip and reconnect to the next node, so mqtt/messaging never really goes down. I just did a rolling upgrade as 4.3.4 of EMQX released today and had some fixes I was waiting for. After it rolled through the deployment and upgraded all the nodes, mqtt clients reconnected automatically. When the Frigate client reconnected, I see it connected to the MQTT broker in EMQX, but it’s not sending anything really through except some random occasional messages for one of the three cameras.

Also yes I get EMQX is overkill in a home stack, however it’s light weight, has a decent web interface for status/monitoring, super easy to deploy a cluster with a helm chart, and my favorite part is I have it integrated with OpenLDAP with a custom OU for MQTT devices. So I can add new MQTT users on the fly without needing to restart the broker.

EDIT: I also have about 30 or so MQTT clients at any given time between all my IoT devices, and so far only seem to notice this with Frigate. I also made sure that Frigate’s MQTT client string is unique, and it’s not using shared credentials.

Finaly everything working with coral pci, i am so happy !!!
<3 @blakeblackshear

1 Like

Everything is ok. I see in logs, Is this important?

frigate.app                    INFO    : Capture process started for doorbell: 101
W :131] Could not set performance expectation : 94 (Inappropriate ioctl for device)

Yes

Is the error after you make correction

Yes, that’s the error I get when I’ve made the correction. Any other value seems to give a not found error so I guess this is improvement, but I’m not sure why it thinks my config file is a directory

This may help with error


volumes:
  - type: bind
    source: /frigate/config/config.yml
    target: /config/config.yml

Okay, just tried that. Same error :confused:

Edit: just got it to work. Not really sure what I did, besides trying to put the config file in a different directory. Now my mqtt isn’t configured properly so I’ll see if I can solve that.

Frigate publishes to frigate/available=online in it’s on_connect callback. You should see a log message in Frigate of MQTT connected right before it sends it. The sensors in Home Assistant use that topic to determine whether or not the entity is available as a part of the normal “last will” mqtt spec. Either Frigate is not sending that message because it does not detect a reconnection for some reason, or the message is not sent to Home Assistant. Since they go unavailable, then Home Assistant must be receiving the frigate/available=offline message from your broker.

Thank you, this was super helpful! I’ll keep an eye out for the online / offline messages and go from there in debugging.

I know h265 playback is a browser issue, but are safari and ie 11 not compatible?

Please suggestion.
I have 10cameras i want use NODERED when object person is found make something.
How can i read person object from the cameras? What are the settings. As what i see in MQTT settings it saves only pictures.

UPDATE:
SO i found the events.
but

  "after": {
    "id": "1624653706.05705-jb0kpx",
    "camera": "CAM8",
    "frame_time": 1624653710.852007,
    "label": "person",

I need to check the ID and hold in memory and wait if it changes? and the object is person then do something?
If somebody can share his flow i would be very grateful :slight_smile:

UPDATE 1:
Solved it, if anybody need help, PM me.

Use the HACS integration for Frigate along with node-red-contrib-home-assistant-websocket. You’ll have everything you need.

1 Like

Yaaay! :tada: I have succeeded with installing Reolink RLC-810A cameras to Frigate NVR :star_struck:

Well, it isn’t working on full 4k resolution (3840x2160) because there is some error with image being “concentrated” to top 2 milimeters of the screen which I just reported back to support, but on 2560x1440 it works without errors in Frigate and also I don’t see any drops in the frames or cluttered image as it was before. You are asking what changed, right? So the change which Reolink did was switch of the main stream from h.265 to h.264. And now all issues are gone and I’m able to use susbstream for decode and main stream for record, clips and even rtmp. But let’s go step by step…

What to do when you would like to use Reolink RLC-810A cameras in Frigate NVR:

  • write to Reolink support using Submit a request – Reolink Support. Ask for a new firmware which solve network drops of the connection to Blue Iris (that’s how they call it). My cameras are Hardware No. IPC_523128M8MP and I got firmware file of version v3.0.0.236_21040831 called IPC_523128M8MP.236_21040831.RLC-810A.IMX415.8MP.REOLINK.pak.
  • When you get the firmware file proceed with uploading the firmware to the camera. I suggest update the firmware with Update Configure File checked (WARNING: this will restore all configuration in the camera to factory defaults and you’ll lose all configuration!), because some options are not visible anymore in the new firmware so I believe it’s better to reset them to defaults.
  • Set new settings to the camera > connect to camera using IP address (default login credentials are “admin” and no password), navigate to Settings (gear icon in top right corner):
    • Camera
      • Display
        • Date & Time: Hide
        • Watermark: disabled
      • Stream
        • Clear
          • resolution: 2560*1440 (after changing this you’ll need to login because camera will be rebooted)
          • FPS: 25
          • Max bitrate: 8192
        • Fluent
          • resolution: 640*360
          • FPS: 10
          • Max bitrate: 512
      • Audio and Light
        • Record Audio: enabled
      • Info
        • Camera name:
    • Surveliance - here I disable all
    • System
      • User management - change password for admin
  • The camera will reboot and it will put you back to login screen. Here select Stream type: Clear and login. Did you notice there isn’t any Balanced in the setting anymore? Did you notice you can view Clear stream in the browser? That’s because the codec for main stream was changed from h.265 to h.264. Now all works much more smoother :slight_smile: We only need to wait for Reolink to fix the 4k resolution. But let’s continue…
  • Now in frigate.yml you can set something like this:
    cameras:  
      camera_name:
        ffmpeg:
          inputs:
            - path: rtsp://admin:[email protected]:554//h264Preview_01_sub
              roles:
                - detect
            - path: rtsp://admin:[email protected]:554/
              roles:
                - record
                - clips
                - rtmp
          global_args: -hide_banner -loglevel info
          output_args:
            record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #removed -an parameter to enable audio
            clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #removed -an parameter to enable audio
        width: 640
        height: 360
        fps: 5
        objects:
          track:
            - person
            - car
        clips:
          enabled: True
          pre_capture: 5
          post_capture: 5
        rtmp:
          enabled: True
    
  • Restart Frigate NVR add-on in Home Assistant Supervisor and verify the log

And that’s it. Now thanks to the new firmware the Reolink RLC-810A is working in Frigate NVR better than ever before because recording and clips can be played directly from the browser thanks to the h.264 codec. The only issue is that the 4k resolution of main stream is not working but I hope they will solve it immediatelly when more of us will be requesting this to be fixed. So please write them feedback that we love change of the codec to h.264 and ask them to fix that 4k resolution :slight_smile:

Update: I’ve been in discussion with Reolink support every day and their senior engineer confirmed it’s not possible to have 8MP resolution with h.264.
So it’s either h.264 with all-browser view on lower resolution with RTMP feature in Frigate working flawlessly or h.265 on full resolution. I choose simplicity and better support in browsers over resolution.

10 Likes

Thank you.
Nevertheless the websocket is reading but the result is in the same spot. So does it have some solution if the result is the same dont trigger?
Or i need to save the ID and check for changes?

UPDATE:1
Solved it, it is actualy seeing if the message was already sent and not sending again.
If anybody need help, PM me.