Local realtime person detection for RTSP cameras

What this mean in HA log?

21-02-26 13:49:47 WARNING (MainThread) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR
21-02-26 13:49:47 WARNING (MainThread) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR Beta (outdated)

Hello and sorry for the noob questions. I have looked for answers but finally decided to ask for help.

A little background. I worked in Home Automation starting in the 1980ā€™s and retired a few years back. Iā€™ve designed products for multiple companies and have more than a casual knowledge of home systems and software. That said, my knowledge is a bit stale and Iā€™m entirely new to Home Assistant and frigate. My linux knowledge is basic but I can usually make things work.

I have HA currently deployed in my loft running Supervised HA on a Pi 4b with 4G ram & Debian 10. Debian was used because I wanted to eliminate the SD card and go with a more robust SSD.

My loft is being used as a testbed for a very large system for a home Iā€™m building in Costa Rica. The loft has 38 Hue Lights, 8 TPLink Light dimmers, Lutron Pro bridge to use Lutron Pico remotes, a tempest weather station, 1-wire environmental sensors, ecobee thermostats, and a variety of Audio/Video devices. The home in CR will be 3-4 times as many devices, but I intend to test any integrations here before shipping stuff to CR.

This brings me to frigate. I really want this to work but am running into walls. I should also note that I know the Pi4 will be under powered for my camera needs, but I was hoping to try out frigate before upgrading hardware. Iā€™m testing with a reolink RLC-520 camera. I have a coral on order.

First I tried using the HACS custom component. HACS install works fine, but it fails to configure after I add the integration. No logs are visible since it never starts.

I also tried the add-on and had some success. Frigate installs OK but crashes after startup

Iā€™m asking for assistance but also hopeful that I can find someone with expertise to assist me for a fee. I know this is generally frowned upon, but Iā€™m willing to learn. When Iā€™m over my head Iā€™d like to have an experienced mentor to help guide me.

Lastly, my home in CR is on a 17 acre parcel of land. I intend to install somewhere between 12 and 15 cameras which are integral to my security system. This will obviously take more horsepower than a Pi can supply. I need guidance on which platforms I should consider and also how I might segment the system so that simple automation runs on a Pi and the processor intensive stuff like cameras runs on a more powerful platform.

Hereā€™s the log

* Starting nginx nginx
   ...done.
frigate.app                    INFO    : Creating directory: /tmp/cache
frigate.app                    INFO    : Creating tmpfs of size 256m
mount: /tmp/cache: cannot mount tmpfs read-only.
frigate.app                    ERROR   : Failed to create tmpfs, error code: 8192
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
detector.coral                 INFO    : Starting detection process: 40
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.app                    INFO    : Camera processor started for back: 43
frigate.app                    INFO    : Capture process started for back: 44
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate.app                    INFO    : Stopping...
frigate.events                 INFO    : Exiting event processor...
frigate.record                 INFO    : Exiting recording maintenance...
frigate.object_processing      INFO    : Exiting object processor...

frigate.yml

cameras:
  back:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554//h264Preview_01_main
          roles:
            - detect
            - rtmp
    width: 640
    height: 480
    fps: 4


mqtt:
  # Required: host name

  host: core-mosquitto
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate
  # Optional: user
  #user: mqtt_user
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  #password: password
  # Optional: interval in seconds for publishing stats (default: shown below)
  stats_interval: 60
  
clips:
  # Optional: Maximum length of time to retain video during long events. (default: shown below)
  # NOTE: If an object is being tracked for longer than this amount of time, the cache
  #       will begin to expire and the resulting clip will be the last x seconds of the event.
  max_seconds: 300
  # Optional: size of tmpfs mount to create for cache files (default: not set)
  # mount -t tmpfs -o size={tmpfs_cache_size} tmpfs /tmp/cache
  # NOTICE: Addon users must have Protection mode disabled for the addon when using this setting.
  # Also, if you have mounted a tmpfs volume through docker, this value should not be set in your config.
  tmpfs_cache_size: 256m
  # Optional: Retention settings for clips (default: shown below)
  retain:
    # Required: Default retention days (default: shown below)
    default: 10
    # Optional: Per object retention days
    objects:
      person: 15

Looks like you need to define your detector. Check the docs: https://blakeblackshear.github.io/frigate/configuration/detectors

1 Like

Is it possible to install the driver in HassOS running ā€œbare-metalā€ on the nuc?

detectors:
  cpu1:
    type: cpu

Is what I used.
It got away from me too, lower in the config doc.
In my mind the most important things would be at the top.

Thanks that worked. I now have basic functionality. Now on to the next hurdle!

1 Like

Thanks. I did miss that in the docs.

Thanks for the help. I set the decoder to cpu (until mouser gets coral back in stock).

I now have basic functionality. But Iā€™ve got a new problem. I did find some bug reports on this which may have crept into a recent version.

  • Starting nginx nginx
    ā€¦done.
    frigate.app INFO : Creating directory: /tmp/cache
    frigate.app INFO : Creating tmpfs of size 256m
    mount: /tmp/cache: cannot mount tmpfs read-only.
    frigate.app ERROR : Failed to create tmpfs, error code: 8192
    Iā€™m running HA supervised on a Pi4 and Debian 10 to support boot from SSD.

Supervisor log reports

21-02-26 22:09:17 WARNING (MainThread) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR Beta (outdated)

So it seems I can view cameras but canā€™t save clips.

Also still looking for advise to build a large system with 12+ 4K cameras and a ton of HA devices. I know rpi wonā€™t cut it. Any advise on system architecture and approach would be helpful. It would be great if basic lighting, HVAC, etc ran on a pi; and processor intensive stuff like CCTV ran on something more powerful.

The system will be installed in my new home in Costa Rica, so I need a stable system that can be as remotely administered as possible. A failover system would be awesome too. Iā€™m not worried about costs. Iā€™m looking for bulletproof.

Anybody with experience in the above please reach out. Iā€™m happy to pay for support, or extend an invite to Costa Rica.

Hi, i run frigate ā€œsuccessfullyā€ on my home assistant VM (ESXi) 4vCPU 8GB RAM NUC8 with Frigate as addon and the integration. But I would like to split the meat from the bone :-). I have installed an ubuntu server on another NUC on hyperv (donā€™t ask why, itā€™s a given).

I want to ā€œmoveā€ the addon because I have around 50-60% CPU load and want to get rid of that in my homeassitant VM. Reading the documentation and trying to create a separate VM.

I have a few questions about it, the documentation is somewhat thin.

I am following:

  1. I want to install using docker (is there any other option?) I have a fresh install of ubuntu server 20 and docker.
  2. What do I need to change in the compose file and how to handle it?
  • a. How to determine this: image: blakeblackshear/frigate:<specify_version_tag>? And what happens if a new version comes out?
  • b. Where should I put/edit/do the config file? How to get there?
  • c. Do I really need this if I donā€™t want to ā€œpass the stream furtherā€?
    environment:
      FRIGATE_RTSP_PASSWORD: 'password'

d. any other thoughts?

Thanks in advance!

Excellent progress on frigate! Iā€™m just waiting for one more feature to help when viewing on mobile devices. When viewing a live camera is it possible to add the full screen icon to the video? The same as you have when viewing a past event video

I canā€™t seem to get Frigate images attached to notifications. Iā€™ve tried pushover with the local Frigate address and notifications with my casa addy. Neither work, the result with the native notifications is a missing URL notification. Iā€™ve followed the instructions and must be missing something somewhere.

Do note, Iā€™ve tried a mixture of double and single quotes and in this latest batch, escaping the single quotes. Iā€™m on IOS. Note also that any formatting mistakes are likely due to putting here, I use MSCode and the tabs are setup.

- id: '1614357593119'
  alias: 1 Frigate
  description: ''
  trigger:
  - platform: mqtt
    topic: frigate/events
  condition:
  - condition: template
    value_template: '{{ ( as_timestamp(now()) - as_timestamp(state_attr(''automation.1_frigate'', ''last_triggered'')) |int(0) ) > 3 }}'
  - condition: not
    conditions:
    - condition: state
      entity_id: REDACTED
      state: disarmed
  action:
  - service: notify.mobile_app_REDACTED
    data:
      message: "{{trigger.payload_json['after']['label']}} was detected."
      data:
        attachment: "https://REDACTED/api/frigate/notifications/{{trigger.payload_json['after']['id']}}/thumbnail.jpg"
        content-type: png
        hide-thumbnail: false
  - delay:
      hours: 0
      minutes: 0
      seconds: 5
      milliseconds: 0
  mode: single

You installed the integration (custom component) right?

Yes I did.

Iā€™m using this path to attach images to my notifications.
http://IP:PORT/api/CAMERANAME/person/best.jpg

How does it know which event?

Iā€™m doing it in node-red, but as a trigger Iā€™m using the person sensor thatā€™s created with the integration being > 0

Also noticed you have the content-type set to png even though the image is a jpg.

Thatā€™s a typo in this thread, I re-added the attachment stuff which I removed in the actual automation for the time being. Sorry.

The integration creates a public endpoint for that image. You can grab the event id from the url when viewing in frigateā€™s webui and try to visit the url for the notification.

IP:PORT the local Frigate addy?