Local realtime person detection for RTSP cameras

Make sure you include dog in your list of objects to detect. See the docs: https://blakeblackshear.github.io/frigate/configuration/cameras#objects

1 Like

@thundergreen is this true? as i am using nabucasa, will i be able to see that?
I am busy now, but will definetly let you know! :heart:

Tell me when u r ready :smirk:

Definitely works fine with the single-side M.2 PCIE (A+E), as thatā€™s what Iā€™m using (with a Docker container).

Hi Ben,

Thanks for your answer!
Could you please clarify what is a single-side M.2 PCIE (A+E)?
Do you have access to both TPUs in your system?

Reading This it looks like not all system supports both edge TPU.
Do you know how I can check on mine? (Manual and datasheet is not enough descriptive about that point.

2 Likes

Iā€™m using the:https://coral.ai/products/m2-accelerator-ae/. It works. No idea how to check for your motherboard.

I see, so I guess you have a single TPU model while I was referring to the double TPU one which seems (from the limited reading I had) trickier to make it work.

Thank you very much for your feedback anyway

Iā€™ve been looking into the dual TPU Coral. Is there any reason it couldnā€™t be plugged into an M.2 - PCIe adapter? Iā€™ve been testing that with the M.2 single Coral and itā€™s working quite well.

Iā€™m also testing ā€˜mixedā€™ Corals:

corals

The PCI Coral consistently outperforms its USB sibling.

Hi folks,

What is the best way to configure frigate when run multiple cameras? Single instance or a docker instance for every camera?
And what about the coral? A coral for every camera or many cameras with the same coral?

Regards

Have you added a heatsink or anything like that to your Coral? The documentation seems to suggest that they can overheat and throttle to a lower clock speed when you push them hard.

Iā€™ve been checking the thermals on mine, the maximum Iā€™ve seen is about 48c, but I havenā€™t been able to catch it to check the temps when running at ~130fps on its own.

Iā€™d definitely recommend a second or dual Coral as per mspinolo. The mixed Corals hit 223fps today :astonished:

Nope. Havenā€™t really dug into what my Coral is doing yet. Given its efficiency (apparently) compared to CPU, Iā€™m not too worried, as running CPU detection never really ran of ~65% utilisation. Iā€™m only running fours cameras, and detection is running on lower res streams at the moment.

Just had a check - my max fps over the last 24hrs is only 28fps (inference speed varies from 6.2 to 11.3ms). Not sure on temps at this point.

Does anyone have or know of examples of using Frigate with custom models as discussed here?

EDIT: For anyone with similar questions, see answers from u/blackbear85 on Reddit added at end of post.

Iā€™ve got a few questions in particular:

  1. Could the image classification models here such as MobileNet V2 (iNat birds) be used with Frigate with an appropriate width/height config or only object detection models?
  2. Is it possible to use two models concurrently, e.g., the built-on COCO model plus another?
  3. If all really want to do is to detect both people as with the COCO model and squirrels :wink: as with the MobileNet V2 (iNat birds), what is the simplest way to go about it?

Thanks!

EDIT: Answers from u/blackbear85 on Reddit:

  1. You canā€™t use classification models.
  2. Not without running something else in addition to frigate.
  3. Best way would be to train a new model.
1 Like

Question:
How to improve snapshot qualityā€¦
From MQTT or via api ā€œhttps://HA_URL/api/frigate/notifications//thumbnail.jpgā€?

I have the dual TPU Coral in a M.2 adapter; I have confirmed the adapter wires up both PCIe lanes. It is only detected as one device and I believe it should be detected as two devices.

The dual TPU exposes two x1 PCIe interfaces, so for this to work I think the motherboard will specifically need to support bifurcation on the M.2 interface, which as far as I know no motherboard does this. This would require custom UEFI firmware.

1 Like

Nice inference times. The lowest mine seems to go is 6.5ms up to 12.5ms.

It sounds like youā€™ll be pretty safe running at that framerate based on the readings Iā€™ve been getting. Today, I decided to get Home Assistant to keep an eye on the PCI Coral temp:

# configuration.yaml
sensor:
  - platform: command_line
    name: Coral PCI Temperature
    command: "cat /sys/class/apex/apex_0/temp"
    unit_of_measurement: "Ā°C"
    value_template: "{{ value | multiply(0.001) | round(1) }}"

coral_temp_12-04-2021

So when comparing temp to detection fps, my M.2 Coral peaked at 41.3c when running in mixed mode @ 153.1fps with an ambient temperature of 18c. So the M.2 Coral would have been processing something like 76.55fps at that point in time, which is a little more than half its maximum throughput.

coral_fps_12-04-2021

It isnā€™t a fair test, due to it being mixed and plugged into an M.2 adapter card. Tomorrow, Iā€™ll plug the Coral into the motherboard M.2 slot and max it out in single mode to see how hot it actually gets :grin:

1 Like

I donā€™t think there is a setting to increase/decrease the quality of the snapshot. I remember reading a release or two ago in the Frigate GH that there was a code change that permanently reduced the quality of the snapshot .jpg because I think they were larger than they needed to be when he first implemented them. I canā€™t find that code change now howeverā€¦

You can always run a higher resolution on your detect stream of course. The snapshots will be taken directly from that.

I do remember that there was a code change, but I dont recall where that change was made.
Running in HIGH stream mode increases my CPU usage to over 50% and Frigate add on bangs out with errors introduced in 0.8.2 far more often with HIGH streams. So im kinda stuck with LOW stream at the momentā€¦

1 Like

Has anyone managed to get 2x Coral PCI TPUs working simultaneously? I received a second M.2 Coral today and had been hoping to get it working alongside my other M.2 Coral (and perhaps even a USB Coral too) but it doesnā€™t seem to work.

Frigate fails to detect the second PCI Coral and crashes while loading. Itā€™s as though something specific needs to be specified in detectors, like with the ā€œMultiple USB Coralsā€:

detectors:
  coral1:
    type: edgetpu
    device: usb:0
  coral2:
    type: edgetpu
    device: usb:1

Thanks @mr6880ā€”might have to recycle your code to get the Coral temp.

image

image