Local realtime person detection for RTSP cameras

[thundergreen]

Not sure if you can get it running on libreelec as it is Limited and not sure for the raspberry if it runs on it ?

I mean on Which machine do I wanna run the coral pcie?

[Morphy]

It’s not on a Pi, it’s an Intel NUC type box. An MSI cubi. It has M.2 PCIe A+E (the small one) and B+M (the long one). I’ve got both to test so I put it in the small one as running the OS on a M.2 SSD

[thundergreen]

Then similar to my setup…what’s going wrong when installing the drivers ?

[Morphy]

Can’t install drivers on Libreelec as it’s a read only OS, pretty sure that’s why the PCIe doesn’t work as it’s missing the apex driver. The USB coral works fine though. I’ll whack Ubuntu on it tomorrow and see if I get more joy with that.

[thundergreen]

You can remove the read only in libreelec. But indeed I can’t iwnstall packages

[putch]

There is a custom integration you can install, which you can then point to your frigate docker running elsewhere.

https://github.com/blakeblackshear/frigate-hass-integration

[AussieByrd (Ben)]

Thanks - I did see that, but got it confused with the HassOS addon (which I had already installed for pre-docker testing)

[AussieByrd (Ben)]

Just checked intel_gpu_top with the container both running and then stopped - all metrics drops to almost zero when stops, so hardware acceleration is definitely working. Def need to get a Coral in play through–CPU idles around 70+% during the day (wind and shadows I’d guess)–order has been picked up in China (so some time between 2 weeks and 3 months to wait )

Thanks again!

[_dev_null]

No problem Ben,

Good luck ! I’m not quite ready for Frigate just yet, it’s on my to do
But as a new user to HA I’m still evaluating the wealth of options that this community has posted - For example In terms of TensorFlow it may be more efficient to do this through nodered instead of Frigate etc etc etc

I’m so tempted to order a coral but I have bigger fish to fry like getting a stable system running prior to playing with topics like this.

Good luck and kind regards

[thundergreen]

Hello… dumb quesstion… when i add the cameras generated from Frigate to my lovelace dashboard… the “fullscreen” (more info) of the camera is really small… I have NO stream activated. My cameras are configured as follows:

  garage:
    ffmpeg:
      inputs:
        - path: rtsp://40.40.40.220:554/media/stream1
          roles:
            - detect
            - rtmp
            - clips
            - record
    width: 1280
    height: 720
    fps: 23

BUT it is too small when opennng the entity:

Any idea why it is so small? and how can i make it bigger???

If i’m not wrong the video (camera) i see here is the rtmp stream? Possible to make the resolution bigger?

[Morphy]

Now I have my system running on a more powerful system I’ve been doing some testing with notifications and delays and noticed there is some considerable delay by using the event trigger. AFAIK the delay is because the clip can’t be sent until the event is over?

Using the binary sensor for motion seems to be a lot quicker (almost instant) so I’ve started using this on top of the one with the clip:

- id: Camera Back Garden Person Alert
  alias: Camera Back Garden Person Alert
  trigger:
    platform: state
    entity_id: binary_sensor.back_garden_person_motion
    to: 'on'
  action:
  - service: notify.mobile_app
    data_template:
      title: "Back Garden Camera Person Detected"
      message: "Full clip will follow"
      data:
        image: "/api/camera_proxy/camera.back_garden"
        actions:
          - action: URI
            title: View Cameras in HA
            uri: '/lovelace-security/cameras'

It’s not as refined as the other as I need one automation per camera and per object. Ideally I’d like to include a link to events from the first notification but that would mean getting Frigate events inside HA for external viewing or exposing frigate web gui to the web. Is there a way to turn of the clips api so this isn’t exposed to the web openly?

[walt]

Node-RED notification flow for IOS Users - Send notification when an object is detected, 3D/force touch into notification for live view of camera, use actionable notification to send the clip (intelligently waits for clip to be ready)

Example:

NodeRED Flow:

Requirements:

• Frigate HA Integration
• Home Assistant Companion App (IOS)
• Home Assistant accessible from the internet (or via Nabu Casa)

Installation:

Copy the flow to NodeRED (Hamburger Menu => Import):
https://paste.ubuntu.com/p/XD62VwNj7s/

Edit the first “Format notification” function node and add your external URL. Optionally, edit case statement to customize your camera to location mapping. (For example, I customize the camera.doorbell location to be “front door”)

Add following to your home assistant configuration for actionable notifications.

ios:
  push:
    categories:
      - name: camera
        identifier: 'camera'
        actions:
          - identifier: 'SENDSTILLANDCLIP'
            title: 'Send still and video clip'
            authenticationRequired: false

Restart HA and sync push categories to your phone (HA App => App Configuration => Notifications => Categories)

That’s it, you’re all done!

For my personal setup, I also add in history checking of my external doors and current state of doors to limit alerts. For example, don’t notify if someone is at the front door if the front door opened in the last five minutes. I also use a input_boolean.person_notification to control if alerts are enabled, and add a actionable notifcation above called “SNOOZEPERSON” that temporarily disables notifications for four hours (useful when the kids are playing in the yard).

I also push all Frigate clips to my Roku TV (if it’s idle) as well and announce alerts on Google Homes / Amazon Dots.

That full flow, for advanced users, is here:
https://paste.ubuntu.com/p/bPjDwDC9ms/

[thundergreen]

has anyone an idea how to overlay the time with output / input parameters to ffmpeg ?

[phairplay]

hi,
could some one please help

I’m running frigate on hassio, RPi 4 8gb
I have a feed showing can I can see it tracked me

yet when i click in the events nothing is there, also i don’t see anything sent via mqtt

here is my settings

mqtt:
  host: 10.0.0.9
  user: *****
  password: ****
  port: 1883
  topic_prefix: frigate
  client_id: frigate
cameras:
  front:
    ffmpeg:
      inputs:
        - path: rtsp://user:[email protected]:port/Streaming/channels/302/
          roles:
            - detect
            - rtmp

    width: 960
    height: 576
    fps: 12

detectors:
  cpu1:
    type: cpu
  cpu2:
    type: cpu

what am I missing or what am I doing wrong?

[danbutter]

Do you have the integration installed and setup?

[phairplay]

Yes I have MQTT setup and running

[danbutter]

I meant for frigate. I’m running HassOS so I don’t know if it looks different for other types of installs, but what I have is if in your HA you go to Configuration - Integrations.
Do you have one for Frigate?
Mine looks like this:

[phairplay]

I have frigate running on the same rpi as home assistant via the supervisor add-on, do I also need an integration?

[hawkeye217]

Frigate works best with Home Assistant when using the official integration. Check the docs.

[phairplay]

Thank you @danbutter and @hawkeye217 I completely missed the fact there is also an integration.

I thought the add on would just post to HA via MQTT

The integration is working