Local realtime person detection for RTSP cameras

Hmm, so in that case I wonder what I should put there instead?

I was using the HA add-on previously but wanted to move Frigate to a more powerful machine as the RPi my HA is on started to struggle with any more than two camera feeds.

Perhaps something like this?

The documentation mentions config.yml must be a path-to-config-file:

      - <path_to_config_file>:/config/config.yml:ro
      - <path_to_directory_for_media>:/media/frigate

Yeah, I thought I tried that too, tbh. I even just moved the config.yml file into the root directory where my docker-compose.yml file is but no dice.

So, essentially, I have the docker-compose.yml file in c:\containers\frigate

And placed the config.yml file in the same directory.

So, should the line in the docker compose file be:

- c:/containers/frigate/config.yml:/config/config.yml:ro

I would assume with some changes it would be able to work off the HA HLS stream, since as you mentioned the input just gets passed to ffmpeg here: frigate/config.py at 03e756dd27082ea665be03638a21e5d51ab7f895 Ā· blakeblackshear/frigate Ā· GitHub
I guess to support this there would need to be some support for runtime addition/modification of inputs, as currently they are defined/loaded from the frigate.yml file when frigate is started.

So maybe my cameras dont work with frigate that well. Using an RTSP FFMPEG camera in HA directly i can see my stream with a few seconds delay.

Using the RMTP stream through frigate just seems to stop working on my cameras or the stream takes roughly 10-15 seconds to load in LoveLace UI and then will buffer every 5 seconds. They also dont appear to work via VLC where as the direct RTSP stream does.

Getting a lot of packet size mismatch errors on the stream integration (i still have FFMPEG integration enabled also - not sure if they interfere).

* Error demuxing stream: [Errno 1094995529] Invalid data found when processing input: 'rtmp://192.168.1.10/live/bedroom'; last error log: [flv] Unable to seek to the next packet
* Error demuxing stream: [Errno 5] I/O error: 'rtmp://192.168.1.10/live/cameraname'; last error log: [tcp] RTMP packet size mismatch 12929990 != 339804

Its causing me a quandry because i want to use the lower res stream for object detection which appears to work ok but then use the higher res stream for clips, snapshots and RTMP into HA but as it stands i cant.

I dont know the impact of running 2 high res RSTP connections to my cameras as i dont want my snapshots to be from the low res stream.

If you get this error using latest beta, a fix is already committed in the integration repo. We just need to wait for the release of the update. :slight_smile:

Setup failed for frigate: Unable to import component: cannot import name 'Message' from 'homeassistant.components.mqtt.models' (/usr/src/homeassistant/homeassistant/components/mqtt/models.py)

7:27:01 AM ā€“ (ERROR) setup.py

ok! so I managed to get Frigate up and running, sorta.

changed to this in my docker-compose.yml to get it working:

      - type: bind
        source: ./config.yml
        target: /config/config.yml

Now Frigate is booting up but not recognising my Google Coral and basically getting stuck in a bootloop. Itā€™s up for maybe 30-60 seconds and i can log into the UI and see the camera feeds but then it exits out as per the logs below.

Iā€™m not sure if this is related to it not being able to find the TPU, or something else iā€™ve messed up. for reference, i have this installed on an Alienware Alpha running Win 10 with onboard Nvidia chip so iā€™m using the amd64nvidia architecture.

If anyone could pls take a look at my logs and tell me what iā€™m doing wrong, I would really appreciate it. My config.yml i just copied over from the working version I had when i installed Frigate through the HA add-on so iā€™m assuming it should all transfer seamlessly, no?

Logs:


frigate    | Starting migrations
frigate    | peewee_migrate                 INFO    : Starting migrations
frigate    | There is nothing to migrate
frigate    | peewee_migrate                 INFO    : There is nothing to migrate
frigate    | frigate.mqtt                   INFO    : MQTT connected
frigate    | frigate.app                    INFO    : Camera processor started for backyard_cam: 38
frigate    | frigate.app                    INFO    : Camera processor started for back_porch_cam: 39
frigate    | detector.coral                 INFO    : Starting detection process: 35
frigate    | frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate    | frigate.edgetpu                INFO    : No EdgeTPU detected.
frigate    | Process detector:coral:
frigate    | Traceback (most recent call last):
frigate    |   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
frigate    |     delegate = Delegate(library, options)
frigate    |   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
frigate    |     raise ValueError(capture.message)
frigate    | ValueError
frigate    |
frigate    | During handling of the above exception, another exception occurred:
frigate    |
frigate    | Traceback (most recent call last):
frigate    |   File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
frigate    |     self.run()
frigate    |   File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
frigate    |     self._target(*self._args, **self._kwargs)
frigate    |   File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
frigate    |     object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
frigate    |   File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
frigate    |     edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
frigate    |   File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
frigate    |     raise ValueError('Failed to load delegate from {}\n{}'.format(
frigate    | ValueError: Failed to load delegate from libedgetpu.so.1.0
frigate    |
frigate    | frigate.app                    INFO    : Camera processor started for front_door_cam: 42
frigate    | frigate.app                    INFO    : Capture process started for backyard_cam: 43
frigate    | frigate.app                    INFO    : Capture process started for back_porch_cam: 46
frigate    | frigate.app                    INFO    : Capture process started for front_door_cam: 50
frigate    | frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
frigate    | frigate.app                    INFO    : Stopping...
frigate    | frigate.record                 INFO    : Exiting recording maintenance...
frigate    | frigate.events                 INFO    : Exiting event processor...
frigate    | frigate.object_processing      INFO    : Exiting object processor...
frigate    | frigate.watchdog               INFO    : Exiting watchdog...
frigate    | frigate.events                 INFO    : Exiting event cleanup...
frigate    | frigate.stats                  INFO    : Exiting watchdog...
frigate    | peewee.sqliteq                 INFO    : writer received shutdown request, exiting.
frigate    | root                           INFO    : Waiting for detection process to exit gracefully...
frigate    | frigate.video                  INFO    : back_porch_cam: exiting subprocess

Does container have permissions for coral?

Did you install coral driver into host as instructed at coral site? Honestly unclear if this needed or not but I would do. Would love someone else to clarify here.

Coral has good usb power?
This is an issue for many.

Thatā€™s whatā€™s so odd, I had HA installed in a VM on the same machine and I can pass the Coral onto it no problems. But the frigate container doesnā€™t seem to see it - how would I give it permission?

Make sure youā€™re not passing the one coral device to 2 ā€œclientsā€ at the same time. The first one running will take it.

I currently have Frigate running on in docker container on another machine separate from HA and I have HACS integration installed and its operating properly in object recognition etc. The thing that Iā€™m not sure is quite right is that I activated snapshots for a few cameras and they are showing up in the Frigate UI but not in the HA media frigate clips folder.

Should the snapshots also show up in the HA media frigate clips folder?

Next version of the integration will support snapshots.

1 Like

Will integration ever be official, opposed to custom?

Not sure if thatā€™s asked/answered previously

I hope so. It would make it possible to streamline setup like zwavejs.

1 Like

I got it up and running, very well documented. Thank you for a great add-on.

My setup and experience:

  • Run on a RPi 4, with SSD.
  • Got a Coral TPU attached and Frigate indicates in the logs that itā€™s found.
  • 2 cameraā€™s with 2 RTSP streams, low 352x288 and high 1920x1536.
  • When I assign the role of ā€œdetectā€ to the low stream, I get a CPU load of about 12-15%. But the images generated on the camera still (the last detected) with the bound box is pretty poor quality (due to res).
  • When I assign the role of ā€œdetectā€ to the high stream, I get a CPU load of roughly 97-99%. Which I tested as I wanted the image quality to be higher.

Questions:

  1. I thought the Coral was supposed to bear the load for the CPU, How come it crancks up to nearly 100% as soon as I pick the high stream?
  2. Is there any way I can remedy this?

My main aim would be to have the higher quality for the last photo entity as well as have higher quality snapshots.

Any help or suggestions?

As I understand itā€¦
For 1: the coral does the inference of objectsā€¦Not the decoding of the video stream.
For 2: is there a resolution in between those two you could try? Otherwise Iā€™m not personally aware of anything you can do except for buy better hardware. You could try searching this thread for other raspi users experiences.

Hi All - Ive tried to search the history of this thread but didnt have any luck with specifics for ffmpeg options - but it looks like there are a few wyze cam users that are having success with frigate?

I cant get the rtsp stream to connect - it complains about the audio codec

[2021-07-31 11:51:35] ffmpeg.back.detect ERROR : [mp3float @ 0x561c0df9c040] Header missing
[2021-07-31 11:51:35] ffmpeg.back.detect ERROR : [flv @ 0x561c0dfbcac0] FLV does not support sample rate 16000, choose from (44100, 22050, 11025)
[2021-07-31 11:51:35] ffmpeg.back.detect ERROR : [flv @ 0x561c0dfbcac0] Audio codec mp3 not compatible with flv
[2021-07-31 11:51:35] ffmpeg.back.detect ERROR : Could not write header for output file #0 (incorrect codec parameters ?): Function not implemented
[2021-07-31 11:51:35] ffmpeg.back.detect ERROR :
[2021-07-31 11:51:36] frigate.video INFO : back: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
[2021-07-31 11:51:36] frigate.video INFO : back: ffmpeg process is not running. exiting capture threadā€¦
[2021-07-31 11:51:39] ws4py INFO : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:55868]

I assume there would be ffmpeg options to transform the input to get it to work, but havent had any success. I saw some settings mentioned for the dafang hacks firmware on the camera itself, but I am looking for ffmpeg options rather than hardware specific tweaks - so its manageable from 1 spot with multiple inputs. I am using both dafang hacks and the wyze rtsp firmware - and the latter doesnt have any camera level config.

thanks!

urgency on this is low - fixed it my changing the audio codec to PCMU 8kbit on the dafang (which is what the wyze rtsp uses by default - and works).

would still like to know how to accomplish this via ffmpeg though.

Unfortunately no additional stream to refer to.

Try seeing if you can find something that helps you here: