Local realtime person detection for RTSP cameras

What this mean in HA log?

21-02-26 13:49:47 WARNING (MainThread) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR
21-02-26 13:49:47 WARNING (MainThread) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR Beta (outdated)

Hello and sorry for the noob questions. I have looked for answers but finally decided to ask for help.

A little background. I worked in Home Automation starting in the 1980’s and retired a few years back. I’ve designed products for multiple companies and have more than a casual knowledge of home systems and software. That said, my knowledge is a bit stale and I’m entirely new to Home Assistant and frigate. My linux knowledge is basic but I can usually make things work.

I have HA currently deployed in my loft running Supervised HA on a Pi 4b with 4G ram & Debian 10. Debian was used because I wanted to eliminate the SD card and go with a more robust SSD.

My loft is being used as a testbed for a very large system for a home I’m building in Costa Rica. The loft has 38 Hue Lights, 8 TPLink Light dimmers, Lutron Pro bridge to use Lutron Pico remotes, a tempest weather station, 1-wire environmental sensors, ecobee thermostats, and a variety of Audio/Video devices. The home in CR will be 3-4 times as many devices, but I intend to test any integrations here before shipping stuff to CR.

This brings me to frigate. I really want this to work but am running into walls. I should also note that I know the Pi4 will be under powered for my camera needs, but I was hoping to try out frigate before upgrading hardware. I’m testing with a reolink RLC-520 camera. I have a coral on order.

First I tried using the HACS custom component. HACS install works fine, but it fails to configure after I add the integration. No logs are visible since it never starts.

I also tried the add-on and had some success. Frigate installs OK but crashes after startup

I’m asking for assistance but also hopeful that I can find someone with expertise to assist me for a fee. I know this is generally frowned upon, but I’m willing to learn. When I’m over my head I’d like to have an experienced mentor to help guide me.

Lastly, my home in CR is on a 17 acre parcel of land. I intend to install somewhere between 12 and 15 cameras which are integral to my security system. This will obviously take more horsepower than a Pi can supply. I need guidance on which platforms I should consider and also how I might segment the system so that simple automation runs on a Pi and the processor intensive stuff like cameras runs on a more powerful platform.

Here’s the log

* Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
frigate.app                    INFO    : Creating tmpfs of size 256m
mount: /tmp/cache: cannot mount tmpfs read-only.
frigate.app                    ERROR   : Failed to create tmpfs, error code: 8192
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 40
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Camera processor started for back: 43
frigate.app                    INFO    : Capture process started for back: 44
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...
frigate.events                 INFO    : Exiting event processor...
frigate.record                 INFO    : Exiting recording maintenance...
frigate.object_processing      INFO    : Exiting object processor...

frigate.yml

cameras:
  back:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h264Preview_01_main
          roles:
            - detect
            - rtmp
    width: 640
    height: 480
    fps: 4


mqtt:
  # Required: host name

  host: core-mosquitto
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate
  # Optional: user
  #user: mqtt_user
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  #password: password
  # Optional: interval in seconds for publishing stats (default: shown below)
  stats_interval: 60
  
clips:
  # Optional: Maximum length of time to retain video during long events. (default: shown below)
  # NOTE: If an object is being tracked for longer than this amount of time, the cache
  #       will begin to expire and the resulting clip will be the last x seconds of the event.
  max_seconds: 300
  # Optional: size of tmpfs mount to create for cache files (default: not set)
  # mount -t tmpfs -o size={tmpfs_cache_size} tmpfs /tmp/cache
  # NOTICE: Addon users must have Protection mode disabled for the addon when using this setting.
  # Also, if you have mounted a tmpfs volume through docker, this value should not be set in your config.
  tmpfs_cache_size: 256m
  # Optional: Retention settings for clips (default: shown below)
  retain:
    # Required: Default retention days (default: shown below)
    default: 10
    # Optional: Per object retention days
    objects:
      person: 15

Looks like you need to define your detector. Check the docs: https://blakeblackshear.github.io/frigate/configuration/detectors

1 Like

Is it possible to install the driver in HassOS running ā€œbare-metalā€ on the nuc?

detectors:
  cpu1:
    type: cpu

Is what I used.
It got away from me too, lower in the config doc.
In my mind the most important things would be at the top.

Thanks that worked. I now have basic functionality. Now on to the next hurdle!

1 Like

Thanks. I did miss that in the docs.

Thanks for the help. I set the decoder to cpu (until mouser gets coral back in stock).

I now have basic functionality. But I’ve got a new problem. I did find some bug reports on this which may have crept into a recent version.

  • Starting nginx nginx
    …done.
    frigate.app INFO : Creating directory: /tmp/cache
    frigate.app INFO : Creating tmpfs of size 256m
    mount: /tmp/cache: cannot mount tmpfs read-only.
    frigate.app ERROR : Failed to create tmpfs, error code: 8192
    I’m running HA supervised on a Pi4 and Debian 10 to support boot from SSD.

Supervisor log reports

21-02-26 22:09:17 WARNING (MainThread) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR Beta (outdated)

So it seems I can view cameras but can’t save clips.

Also still looking for advise to build a large system with 12+ 4K cameras and a ton of HA devices. I know rpi won’t cut it. Any advise on system architecture and approach would be helpful. It would be great if basic lighting, HVAC, etc ran on a pi; and processor intensive stuff like CCTV ran on something more powerful.

The system will be installed in my new home in Costa Rica, so I need a stable system that can be as remotely administered as possible. A failover system would be awesome too. I’m not worried about costs. I’m looking for bulletproof.

Anybody with experience in the above please reach out. I’m happy to pay for support, or extend an invite to Costa Rica.

Hi, i run frigate ā€œsuccessfullyā€ on my home assistant VM (ESXi) 4vCPU 8GB RAM NUC8 with Frigate as addon and the integration. But I would like to split the meat from the bone :-). I have installed an ubuntu server on another NUC on hyperv (don’t ask why, it’s a given).

I want to ā€œmoveā€ the addon because I have around 50-60% CPU load and want to get rid of that in my homeassitant VM. Reading the documentation and trying to create a separate VM.

I have a few questions about it, the documentation is somewhat thin.

I am following:

  1. I want to install using docker (is there any other option?) I have a fresh install of ubuntu server 20 and docker.
  2. What do I need to change in the compose file and how to handle it?
  • a. How to determine this: image: blakeblackshear/frigate:<specify_version_tag>? And what happens if a new version comes out?
  • b. Where should I put/edit/do the config file? How to get there?
  • c. Do I really need this if I don’t want to ā€œpass the stream furtherā€?
    environment:
      FRIGATE_RTSP_PASSWORD: 'password'

d. any other thoughts?

Thanks in advance!

Excellent progress on frigate! I’m just waiting for one more feature to help when viewing on mobile devices. When viewing a live camera is it possible to add the full screen icon to the video? The same as you have when viewing a past event video

I can’t seem to get Frigate images attached to notifications. I’ve tried pushover with the local Frigate address and notifications with my casa addy. Neither work, the result with the native notifications is a missing URL notification. I’ve followed the instructions and must be missing something somewhere.

Do note, I’ve tried a mixture of double and single quotes and in this latest batch, escaping the single quotes. I’m on IOS. Note also that any formatting mistakes are likely due to putting here, I use MSCode and the tabs are setup.

- id: '1614357593119'
  alias: 1 Frigate
  description: ''
  trigger:
  - platform: mqtt
    topic: frigate/events
  condition:
  - condition: template
    value_template: '{{ ( as_timestamp(now()) - as_timestamp(state_attr(''automation.1_frigate'', ''last_triggered'')) |int(0) ) > 3 }}'
  - condition: not
    conditions:
    - condition: state
      entity_id: REDACTED
      state: disarmed
  action:
  - service: notify.mobile_app_REDACTED
    data:
      message: "{{trigger.payload_json['after']['label']}} was detected."
      data:
        attachment: "https://REDACTED/api/frigate/notifications/{{trigger.payload_json['after']['id']}}/thumbnail.jpg"
        content-type: png
        hide-thumbnail: false
  - delay:
      hours: 0
      minutes: 0
      seconds: 5
      milliseconds: 0
  mode: single

You installed the integration (custom component) right?

Yes I did.

I’m using this path to attach images to my notifications.
http://IP:PORT/api/CAMERANAME/person/best.jpg

How does it know which event?

I’m doing it in node-red, but as a trigger I’m using the person sensor that’s created with the integration being > 0

Also noticed you have the content-type set to png even though the image is a jpg.

That’s a typo in this thread, I re-added the attachment stuff which I removed in the actual automation for the time being. Sorry.

The integration creates a public endpoint for that image. You can grab the event id from the url when viewing in frigate’s webui and try to visit the url for the notification.

IP:PORT the local Frigate addy?