Local realtime person detection for RTSP cameras

You could try the alexxit webRTC card

@blakeblackshear A very big thank you for your job. Well done…it’s an amazing realtime person detection.

I have a docker home assistant installation in a NAS (OMV) and I installed frigate in a different server than Home Assistant.
I installed the blakeblackshear/frigate-hass-integration and it created so many sensosrs, but most of them, all the objects sensors are always unanvailable.

what did i wrong? (no evident error in logs)

Thank You

my card is on the way

1 Like

Home assistant and frigate must be connected to the same mqtt server

Wow great thank you… it was just that.

i’m migrating my mqtt server and using the new one with proxy gtw to old.

Now i changed the frigate one and it works!

Thank You!

Where is more documentation available for this section:


    # Optional: Zone level object filters.
    # NOTE: The global and camera filters are applied upstream.
    filters:
      person:
        min_area: 5000
        max_area: 100000
        threshold: 0.7

in here:

What is the correct syntaxt in config.yml to apply a mask to an object?

e.g. I want to mask a parking place only for object car and not for persons etc…

Their is mask maker in Frigate UI>>cameras>>select camera live view>>show options>>mask and zone creator

You can create mask/zone and then copy/paste into your config. Just correct spacing

Sorry that is not an answer to my question. This part I already had… I asked how to get the mask in for a typical type of object and then ask for an example with correct spacing.

example for “person” in documentation for “objects”

Yes, I have seen that but there is no information I can find for the filter: part
(see my other question Local realtime person detection for RTSP cameras - #4686 by sender)

And this example is not clear for me regarding this part of my question:

So if this is my camera config:

  schuurachter:
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101/
          roles:
            - detect
            - clips
#            - rtmp
    width: 2048
    height: 1536
    fps: 4
    snapshots:
      enabled: true
      crop: false
      bounding_box: True
    objects:
      track:
        - person
        - car
        - dog
        - bicycle
        - cat
    motion:
        mask:    
            - '443,106,775,108,679,365,294,371'
            - '2048,371,2048,585,1136,401,1090,130'
    clips:
      # Required: enables clips for the camera (default: shown below)
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: Number of seconds before the event to include in the clips (default: shown below)
      pre_capture: 2
      # Optional: Number of seconds after the event to include in the clips (default: shown below)
      post_capture: 2
      # Optional: Objects to save clips for. (default: all tracked objects)
      objects:
        - person
        - car
        - dog
        - bicycle
        - cat
      # Optional: Restrict clips to objects that entered any of the listed zones (default: no required zones)
      required_zones: []
      # Optional: Camera override for retention settings (default: global values)
      retain:
        # Required: Default retention days (default: shown below)
        default: 10
        # Optional: Per object retention days
        objects:
          person: 100
          car: 50
          dog: 50
          cat: 10
          bicycle: 50

how would that look like to have cars not detected in the masked zone but do have person, dog, bicycle and cat in it?

filter is where object specific config parameter are placed
In this case I don’t think there is further detail to be provided since those items are documented in camera level documentation

objects:
  track:
    - person
    - car
  # Optional: mask to prevent all object types from being detected in certain areas (default: no mask)
  # Checks based on the bottom center of the bounding box of the object.
  # NOTE: This mask is COMBINED with the object type specific mask below
  mask: 0,0,1000,0,1000,200,0,200
  filters:
    car:
      min_area: 5000
      max_area: 100000
      min_score: 0.5
      threshold: 0.7
      # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
      # Checks based on the bottom center of the bounding box of the object
      mask: 0,0,1000,0,1000,200,0,200

I do not get that documentation. I can’t get it working because after reading it I do not know where in what order and indention to place. How do I split the

objects:
      track:
        - person
        - car
        - dog
        - bicycle
        - cat

to only have car match the mask?

A mask prevents either motion, or an object, being detected in a masked area. It is not a ‘match only this in the masked area’ mask.

There is no ‘inverted mask’, as far as I’m aware, you’d have to mask all other objects, all around the area you don’t want them to be detected in.

So, to NOT match a person in 95% of the image, you’d have to mask person in 95% of the image.

I understand. But what do I practically need to do to:
to prevent detection of cars in a parking space
to not prevent and other object detection on that parking space

Currently on every “motion”, wind, light, clouds I have lists full of cars that are parked.

And please help me with the yaml layout.

To prevent the car being detected in the parking space, create a car object mask so it covers the bottom-middle of the bounding box. Detection is based on the bottom middle point of the bounding box, so you need to mask this.

In my case, I mask about the bottom 10% of my garage, with a car mask… this means it can be detected as it approaches the garage, but not when it is parked (because when it is parked, the bottom middle of the bounding box is in the bottom 10% of the image)

Sorry I dont get that… do you have a picture of it?

You’re welcome to post pictures of you car, garage (with the bounding box), etc on a public forum and I’ll suggest where I’d mask - but I’m not going to be posting mine :wink:

Just a random google image:

this is where I would not like to detect cars but do want to have people, dogs, etc if there are no cars:

Not useful - to mask, you need to know where it is being detected. And that is the bounding box. Mask the bottom centre of wherever it is being detected, to prevent it being detected, without masking the entire area.

If red is the bounding box, green is where you could mask to prevent it being detected - because it covers the bottom centre/middle of the box.