Frigate - Detecting small objects - birds, cats etc

Hi everyone,
I have Frigate setup running the latest version. I have setup detection for person, dog, cat and bird. I am reliably getting notifications about person but I am unable to get and. bird or cat notifications. I have seen birds in the garden and I can see via camera feed but frigate is not detecting this. I presume this is the object being too small for frigate to detect as my camera is mounted high up. How can I ask the Frigate to detect these small objects? I am using a a substream with 720p for detection purpose. Would using the mainstream help? I am aware that this will take up more resources.

I would appreciate any help. My use case is the shoo the birds away with an automation as they feed on my vegetables in the garden.

Thank you

  1. Using a higher resolution would help and of course use more resources. If you are on the latest version you could also use the main stream and use the detect -> width / height settings to resize to a different resolution using your GPU to do most the work.
  2. You also may be okay with your current resolution and just need to make motion detection more sensitive.

Hi
Thanks for your reply.
I am using width/height in the config to adjust the detect area but I am not sure if I am doing it right. Do we have to give a small area for the smaller object to be detected reliably? My config is currenlty as below:

detect:
width: 1080
height: 720
fps: 5
objects:
track:
- person
- cat
- bird
- dog
filters:
bird:
min_area: 100
max_area: 5000
min_ratio: 0.5
max_ratio: 2.0
min_score: 0.5
threshold: 0.5

I have also attached 2 snapshots with a bird detected (to my surprise today, one even before I changed the resolution) As you can see they are quite small area. What does this mean? Do I have to make the area even smaller in the config?


I would not recommend setting these. These are only there to limit the detection. If you are having issues detecting birds you should remove all filters until you get a setup that detects the birds you want.

1 Like

I did not have any filters but nothing was detecting. Hence I was playing around with the settings after reading through frigate documents. Default threshold is .7 so I reduced to detect something. Likewise the area default is much higher, so I reduced this. I’m not sure if I’m doing the right thing here by reducing the area. As you can see a bird is detected with a bounding box of only 475 pixels. Also once my BBQ was detected as a bird and also a person was also detected as a bird. Hence the reason for setting these. Basically, asking frigate to only detect small objects as birds and ignore large ones. At least that’s what I understand and I could be completely wrong here.

Yeah these details weren’t included before so it wasn’t clear why they were being set, but that makes sense.

I need frigate to detect if a cat has a bird or chipmunk in its mouth before entering the cat door and to lock if true lol. My cats are bringing their kills back with them.

1 Like
    detect:
      width: 1270 # <- optional, by default Frigate tries to automatically detect resolution
      height: 720 # <- optional, by default Frigate tries to automatically detect resolution
      fps: 5
      stationary:
        interval: 20
        threshold: 50

    # object track ????
    objects:
      track:
        - person
        - dog
        - cat
        - bird
      filters:
        person:
          min_area: 5000
          min_score: 0.5
          threshold: 0.6
        dog:
          #min_area: 4000
          #min_score: 0.5
          #threshold: 0.6
          min_area: 5000
          min_score: 0.6 # Orig 0.5
          threshold: 0.85 # Orig 0.6
        cat:
          min_area: 5000
          min_score: 0.6 # Orig 0.5
          threshold: 0.85 # Orig 0.6
        bird:
          min_area: 100
          max_area: 5000
          min_ratio: 0.5
          max_ratio: 2.0
          min_score: 0.5
          threshold: 0.5

With the above parameters it detected a rock of about 12" square as a bird!

How would you guess, or measure the min_area or max_area pixels of a bird that appears in Frigate > Settings > Debug? My questions is, how would fix the values when a bird is detected and it’s a false detection, how would you measure the value of increase or decrease for min_area and, or max area?

Here is a picture for your reference:
Screenshot 2024-10-11 112817

the number to the right of the % is the area. Though a bird could easily be that area or close to it depending on how close it is to the camera

1 Like

My configuration does not capture any cats, dogs, birds or raccoons! It’s frustrating. I have read a lot the documentations, and I just get lost.

For the detection of person, cat, bird, and raccoon in Frigate, using a Lorex camera model E892AB, attached to the corner of a soffit of a house (about 10 feet from the ground, with a distance from the camera to the closest point on the ground of 11 feet, and the farthest point from the camera, being detected, of around 24 feet.

How do people figure out the resolution, width, height, min_area, max_area, based on the distance of a camera?

Is their a formula or a method that people use to figure out these values?

Furthermore, I am not getting not that many snapshots in the camera3 area to attempt to analyze, guess for values. The only ones I get are from object, person.

Is there a way to get more information about the object on the bounding_box?

This is my current yaml for one of the camera. I hope someone would help.

Yaml Code:

  #--------------------------------
  # <------ Name the camera 3 BEGIN
  #--------------------------------
  camera3:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=3&subtype=1
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=3&subtype=0
          roles:
            - record

    detect:
      width: 1920 # <- optional, by default Frigate tries to automatically detect resolution
      height: 1080 # <- optional, by default Frigate tries to automatically detect resolution
      fps: 5
      stationary:
        interval: 30
        threshold: 70

    # object track ????
    objects:
      track:
        - person
        #- dog
        - cat
        - bird
        - raccoon
      filters: # experimental from https://docs.frigate.video/configuration/object_filters
        #person:
        #  min_area: 5000
        #  min_score: 0.6 # Orig 0.5
        #  threshold: 0.8 # Orig 0.6
        #  min_ratio: 0.5
        #  max_ratio: 1.0
        person:
          min_area: 10000       # Persons are typically larger, so use a higher minimum area to filter smaller objects
          max_area: 100000      # Set a large max area to filter out anything too large
          min_score: 0.7        # Confidence score for detection, ensure relatively high accuracy
          threshold: 0.7        # Threshold for detection, increase if you're getting too many false positives
          min_ratio: 0.4        # Persons tend to be tall and narrow (standing position)
          max_ratio: 1.0        # Allow some flexibility in ratio for different body postures
        #dog:
        #  min_area: 5000
        #  min_score: 0.6 # Orig 0.5
        #  threshold: 0.85 # Orig 0.6
        cat:
          min_area: 2000       # Adjust depending on the size and distance of the object in view
          max_area: 10000      # Limit larger object detections
          min_score: 0.6       # Confidence score for detection
          threshold: 0.7       # Higher threshold to avoid false positives
          min_ratio: 0.4       # Cat's body tends to be longer than it is tall
          max_ratio: 2.0       # Allow for a longer body, so the width can be up to twice the height
        bird:
          min_area: 500        # Small area for small birds
          max_area: 2000       # Limit larger object detections
          min_score: 0.7       # Confidence score for detection
          threshold: 0.7       # Higher threshold to reduce false positives
          min_ratio: 0.5       # Birds have a relatively balanced height-to-width ratio
          max_ratio: 1.5       # Allows for slight variations when wings are spread or folded
        raccoon:
          min_area: 3000        # Raccoons are larger than small animals, so increase the minimum area
          max_area: 15000       # Maximum area to filter out larger objects (like people)
          min_score: 0.6        # Confidence score for detection
          threshold: 0.6        # Adjust based on detection accuracy (increase if false positives occur)
          min_ratio: 0.7        # Raccoons have a slightly longer body than their height, so adjust the ratio
          max_ratio: 1.8        # Account for flexibility in posture, allowing wider objects to be detected

    # Optional: Record configuration
    record:
      enabled: true
      retain:
        days: 3
        mode: active_objects #options: all motion active_objects
      events:
        pre_capture: 20
        post_capture: 20
        retain:
          default: 3
          mode: active_objects #options: All motion active_objects
          # Optional: Objects to save recordings for. (default: all tracked objects)
        objects:
          - person
          #- dog
          - cat
          - bird
          - raccoon
        #  Optional: Per object retention days
        # person: 7
        # cat: 3
        # bird: 2
        # raccoon: 3

    # Optional: Configuration for the jpg snapshots written to the clips directory for each event
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false
      retain:
        default: 3

    # Optional: Configuration clips
    #clips:
    #  enabled: true
    #  retain:
    #    default: 2

    zones:
      front3_zone1:
        coordinates: 
          0.004,0.988,0.006,0.139,0.141,0.102,0.284,0.091,0.453,0.115,0.661,0.155,0.677,0.093,0.742,0.092,0.742,0.184,0.792,0.227,0.919,0.392,0.911,0.478,0.979,0.234,0.995,0.252,0.995,0.423,0.901,0.68,0.771,0.991
        loitering_time: 0 # Orig 0
        objects:
          - person
          #- dog
          - cat
          - bird
          - raccoon
        inertia: 3
    motion:
      mask:
        - 0.124,0,0.127,0.027,0.184,0.026,0.372,0.034,0.584,0.062,0.682,0.077,0.744,0.079,0.75,0.17,0.922,0.325,0.92,0.373,0.929,0.386,0.978,0.213,1,0.236,1,0
        - 0.782,0.984,0.995,0.991,0.997,0.455
    review:
      alerts:
        required_zones: front3_zone1
      detections:
        required_zones: front3_zone1

It is not shown except on the debug view in Frigate 0.14 but these and many other debugging tools have been added in Frigate 0.15.

There is no need to have raccoon in your config because as I said this label does not exist in the model.

Your filters are incorrect and likely a big cause of the problem, these scores are way too high to accurately detect and track these objects. You should remove all filters until you are detecting the objects then start to dial things in

Before I start, I wanted to say that I sincerely appreciate your help.

The following is a set of pictures of what I have been fighting to deter (by activating the sprinkler in that zone) from last night. They were at it for 45 minutes.

I hear your suggestion to fine tune the detection process through the debug view, but raccoons show up at around 11:30pm – 2:00am, and not every night. It’s very hard to stay up every night to catch them on debug view.

To start, I will remove the person and raccoon objects and keep either dog and cat objects. I also removed all filters.

Here are some questions I have:

  • Which would you recommend keeping, dog or cat object?
  • What scores do you recommend starting with in back2_zone2?
  • Lastly, I have “Frigate (Full Access) Current version: 0.14.1”. Do you know, when the 0.15 version be released?

Thank you

I’d recommend leaving the both cat and dog, and leaving the filter as default (meaning, don’t set them in the config). If they are still not detected then you know you need to make the filters more lenient

1 Like

0.15 will be released in beta within a month or so

1 Like

Why is it that when there is a mask it detects the area as a bird?

Screenshot 2024-11-02 112716

I experienced similar on one of our CAMs where a tree stump in the garden were continuously detected as a bird. I then masked that area a bit more spacious and the false detections went away.

because you are using a motion mask, and motion masks don’t stop objects from being detected Frigate Tip: Motion Masks · blakeblackshear/frigate · Discussion #13040 · GitHub

1 Like

Oops. Thank to you and this forum, i am learning a lot.

I just added an object mask for object “bird”. Shown in gray below.

image

1 Like