Local realtime person detection for RTSP cameras

Share your Projects!
2872

Who can help me fix my automation? I would like to get a notification if a person is detected, but as soon as there is someone in front of the camera, i get a notification every second.
I started off with the example from github

automation:
  - alias: Notify of events
    trigger:
      platform: mqtt
      topic: frigate/events
    action:
      - service: notify.mobile_app_pixel_3
        data_template:
          message: 'A {{trigger.payload_json["after"]["label"]}} was detected.'
          data:
            image: 'https://your.public.hass.address.com/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}.jpg?format=android'
            tag: '{{trigger.payload_json["after"]["id"]}}'

I wonder if it would be better to have a trigger based upon the binary sensor like it was in the example for 0.7.0

automation:
  - alias: Alert me if a person is detected while armed away
    trigger: 
      platform: state
      entity_id: binary_sensor.camera_person
      from: 'off'
      to: 'on'
2873

Only 1 day left to deliver v0.8 for Christmas :rofl: :rofl: :rofl:
Thanks for all the great comments and tutorials. I am still running v0.7.x, and will be updating to v0.8 once ready, also running Blueiris. Will check the great tutorial mentionned above.
For non gen10 processors, does QSV really run better than VAAPI?
Also thinking of moving my cameras to h265.
thx

2874

Are you referring to lag when viewing the mjpeg stream or the camera component frigate provides in homeassistant?

2875

Itā€™s not gonna happen by Christmas, but I think it will be feature complete with a release candidate by the end of the year.

H265 is not compatible with rtmp, so you would need to use a h264 substream for that role. I think you could use h264 for the other roles, but you may run into additional compatibility issues. It will probably take additional CPU to use it for the detect role as well. I would consider it for the clips and record roles only.

I havenā€™t compared QSV to VAAPI.

2876

Iā€™ve been trialling H264 and H265 coming out of my cameras for the clips and record roles. At night, a 1 minute recording on one camera went from 6.8 MB under H265 to 38 MB under H264. CPU usage went up around 1% for the H264 stream too even though I was not using it for detect. I expect the difference will be worse during the day.

So H265 has a massive advantage in storage space for me, it could be the difference between having to store 5-7 days of recordings under H264 or 30 days of recordings with H265.

2877

Iā€™m stuggeling to transfer the frigate/events data (<camera> and <id>) to the node-red-contrib-chatbot telegram video node.
trying to use payload.after.camera and payload.after.id in the filename without sucess.
can you please provide some guidance please?

2879

I need to know more about what you are trying to do, what you expect to happen, and what error you are seeing.

2880

Does H265 play correctly in the media browser?

2881

trying to change the filename (camera + id) in node-red depending on the frigate/events mqtt json message.

[{"id":"b2acbc61.f47a3","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"8b8f1fc1.cdee6","type":"chatbot-video","z":"b2acbc61.f47a3","name":"video","filename":"","video":"http://192.168.1.77:5000/clips/-1608658831.328959-1ynd2u.mp4","caption":"video","x":1210,"y":340,"wires":[["12ca5246.4711fe"]]},{"id":"3c83eb08.13ae84","type":"mqtt in","z":"b2acbc61.f47a3","name":"frigate/events","topic":"frigate/events","qos":"2","datatype":"auto","broker":"77a96594.e33e9c","x":330,"y":340,"wires":[["b16da60d.10a4d8"]]},{"id":"b16da60d.10a4d8","type":"json","z":"b2acbc61.f47a3","name":"test","property":"payload","action":"","pretty":false,"x":550,"y":340,"wires":[["2091db5b.df0f34"]]},{"id":"2091db5b.df0f34","type":"switch","z":"b2acbc61.f47a3","name":"start switch","property":"payload.after.end_time","propertyType":"msg","rules":[{"t":"nnull"}],"checkall":"true","repair":false,"outputs":1,"x":730,"y":340,"wires":[["10f77f03.059dc1"]]},{"id":"10f77f03.059dc1","type":"chatbot-conversation","z":"b2acbc61.f47a3","name":"Conversation Node","botDevelopment":"8fc57b69.702da8","botProduction":"","chatId":"1476560297","userId":"","transport":"telegram","store":"","x":990,"y":340,"wires":[["8b8f1fc1.cdee6"]]},{"id":"12ca5246.4711fe","type":"chatbot-telegram-send","z":"b2acbc61.f47a3","bot":"8fc57b69.702da8","botProduction":"","track":false,"passThrough":false,"errorOutput":false,"outputs":0,"x":1430,"y":340,"wires":[]},{"id":"77a96594.e33e9c","type":"mqtt-broker","name":"Eclips Mosquito","broker":"192.168.1.77","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"8fc57b69.702da8","type":"chatbot-telegram-node","botname":"steenbank_cam","usernames":"","providerToken":"","polling":"1000","store":"","log":"","debug":false,"webHook":"","connectMode":"polling"}]```
2882

It is too bad that H.265 doesnā€™t work. Out of my dozen or so cameras, the only one I still have set to H.264 is the only one I use for Frigate. The rest have been on H.265 for over a year running with Blue Iris.

It is the streaming aspect via RTMP that is the issue with H.265?

2883

Thatā€™s the main issue. You would have to use a h264 substream for that or disable it. RTMP doesnā€™t support h265 in the spec from what I can tell.

2884

So I have the substream set to H.264 with no issues and that is also being using to stream to HA. But the issue is pulling the clip from the H.265 main stream. I was getting ffmpeg errors when I tried to do that last time in the console. I had to change the main stream back to H.264 and then clip saving worked. Iā€™ll try it again today and post the errors if Iā€™m still getting them.

2885

Was just kidding. Take whatever time needed.
thanks for all the comments on h265. The goal indeed would be to use for recording, but if it is HW accelerated, should not have impact of using for motion detect also.

2886

Yes, the camera component frigate provides in HA. Strange thing, I donā€™t have any problem if I create the MQTT cameras manually and using the v0.7.x. With the 0.8.x, and the custom component, I get the previous detection image.

Would it be possible to get the events on the API without save_clips enabled, only the detection role in 0.8.x ? I would be able then to use the snapshot.jpg of the API (e.g. http://IP:5000/events/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg) for the notifications, and that would solve the delay issue.

2887

Steps to get a media browser for Frigate. I have Frigate running successfully, beta 2 in docker container. I have installed the HA integration via HACS. That populated a number of entities correctly in HA. However I do not see any media browser and am a loss for how to add it. Thx.

2888

Hi,

If running Frigate on SSD, is the tmpfs really needed?
Canā€™t get it to work using the YAML as proposed.
Can I replace this:

        - type: tmpfs # 1GB of memory, reduces SSD/SD Card wear
          target: /cache
          tmpfs:
            size: 100000000

by a simple volume that is mapped on docker host ram, like this:

      volumes:
        (.....)
        - /HOST_RAM:/cache

thanks

2889

So I left the substream set to H.264 and changed the main stream to H.265. Frigate didnā€™t throw any errors and proceeded to record the clip on the main H.265 stream via the conditions I have set. That all works wonderfully! The issue is playing it back in HA. The H.265 clip wonā€™t play back in HA which is a bummer.

2890

My trees are moving too much, I need to tell the wind to stop blowing :rofl:
Will add a no detect zone on this tree. Thanks.

2891

Great progress on this, the HA integration and clips are fantastic.

Is there any way of changing the time format to 24 hour rather than AM/PM on the clips in media browser?

Thanks

2892

No, in firefox on Linux I get this error
h265

In Chrome I just get a blank black video box but nothing plays.