Local realtime person detection for RTSP cameras

Sounds similar indeed ! Are you using the custom component or did you define the camera yourself in your HA config ?

Just installed HA OS on a NUC8 and am trying to use the M.2 Coral Accelerator M.2 PCI but so far itā€™s not being detected. I concluded that this is due to missing drivers. Since HA OS is different from Debian I canā€™t use the instructions on the coral.ai website.
Any help would be appreciated.

Update: I decided to install Home Assistant Supervised on Debian 10 in order to be able to install the required drivers

Custom component and the snapshot is of the camera that it creates. This is so I get the frame where the object was detected with the bounding box.

am I completely wrong or blind but I cannot fugure out the binary_sensors for motion detection (if such existā€¦) all other stuff works greatā€¦ when person detect (motion detection ??) then it records as expected. Now i wanted to use this as a trigger. Possibe? I use hassio Addon with custom component but no motion sensors are created

Its

sensor.<camera_name>_person

and it shows ā€˜0ā€™ changing to ā€˜1ā€™ when a person is detected.

binary sensors config

binary_sensor:
  - platform: template
    sensors:
      driveway_person_motion:
        delay_off: 5
        device_class: motion
        friendly_name: "Driveway Person Motion"
        availability_template: "{{ states('sensor.driveway_person') not in ('unknown', 'unavailable')  }}"
        value_template: "{{states('sensor.driveway_person')|float > 0}}"
      driveway_cat_motion:
        delay_off: 5
        device_class: motion
        friendly_name: "Driveway Cat Motion"
        availability_template: "{{ states('sensor.driveway_cat') not in ('unknown', 'unavailable')  }}"
        value_template: "{{states('sensor.driveway_cat')|float > 0}}"
      garden_person_motion:
        delay_off: 5
        device_class: motion
        friendly_name: "Garden Person Motion"
        availability_template: "{{ states('sensor.garden_person') not in ('unknown', 'unavailable')  }}"
        value_template: "{{states('sensor.garden_person')|float > 0}}"
      garden_cat_motion:
        delay_off: 5
        device_class: motion
        friendly_name: "Garden Cat Motion"
        availability_template: "{{ states('sensor.garden_cat') not in ('unknown', 'unavailable')  }}"
        value_template: "{{states('sensor.garden_cat')|float > 0}}"

Thanks a lot for your reply but my sensors sensor.xxx_person is always unavailable.

image

But I get Vdeos / Clips when motion detected.

This is my config:

detectors:
  coral:
    type: edgetpu
    device: 'usb:0'
mqtt:
  host: 192.168.1.41
ffmpeg:
  global_args:
    - '-hide_banner'
    - '-loglevel'
    - panic
  hwaccel_args:
    - '-hwaccel'
    - vaapi
    - '-hwaccel_device'
    - /dev/dri/renderD128
    - '-hwaccel_output_format'
    - yuv420p
  input_args:
    - '-avoid_negative_ts'
    - make_zero
    - '-fflags'
    - nobuffer
    - '-flags'
    - low_delay
    - '-strict'
    - experimental
    - '-fflags'
    - +genpts+discardcorrupt
    - '-rtsp_transport'
    - tcp
    - '-stimeout'
    - '5000000'
    - '-use_wallclock_as_timestamps'
    - '1'
cameras:
  front_door:
    save_clips:
      enabled: true
      pre_capture: 30
    ffmpeg:
      inputs:
        - path: 'rtsp://hass:[email protected]:88/videoMain'
          roles:
            - detect
            - rtmp
            - clips
            - record
    width: 1280
    height: 720
    fps: 5
    record:
      enabled: true
      retain_days: 1
  porch:
    save_clips:
      enabled: true
      pre_capture: 30
    ffmpeg:
      inputs:
        - path: 'rtsp://hassio:[email protected]:554/stream2'
          roles:
            - detect
            - rtmp
            - clips
            - record
    width: 1280
    height: 720
    fps: 5
    record:
      enabled: true
      retain_days: 1
  garage:
    save_clips:
      enabled: true
      pre_capture: 30
    ffmpeg:
      inputs:
        - path: 'rtsp://40.40.40.220:554/media/stream1'
          roles:
            - detect
            - rtmp
            - clips
            - record
    width: 1280
    height: 720
    fps: 5
    record:
      enabled: true
      retain_days: 1
  garden:
    save_clips:
      enabled: true
      pre_capture: 30
    ffmpeg:
      inputs:
        - path: 'rtsp://40.40.40.14:554/media/stream1'
          roles:
            - detect
            - rtmp
            - clips
            - record
    width: 1280
    height: 720
    fps: 5
    record:
      enabled: true
      retain_days: 1

any idea?

Make a username and password for mqtt would be my suggestion.

THAT! indeed worked!! great thanks! Another ā€œissueā€ is that I cannot make any modifications to the MQTT part for my cams like:

    # Optional: Configuration for the jpg snapshots published via MQTT
    mqtt:
      # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
      # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'. 
      # All other messages will still be published.
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: True
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: True
      # Optional: height to resize the snapshot to (default: shown below)
      height: 270

config tells me that mqtt is not valid. I am using beta and hassio addon and it seems that this config is not valid?

where did you got that example? i think those are configured under the snapshot section:

    # Optional: Configuration for the snapshots in the debug view and mqtt
    snapshots:
      # Optional: print a timestamp on the snapshots (default: shown below)
      show_timestamp: True
      # Optional: draw zones on the debug mjpeg feed (default: shown below)
      draw_zones: False
      # Optional: draw bounding boxes on the mqtt snapshots (default: shown below)
      draw_bounding_boxes: True

I had the same thought at first, but they are there in the readme for the beta.


I tried to add them in and just got: ERROR : Error parsing config: extra keys not allowed

I have also the full size frame using camera.snapshot on the cameras created by the custom component, however, only on the MQTT cameras created for each object (camera.<camera_name>_person, camera.<camera_name>_dog, etc.) but not on the RTMP cameras created by the custom component (camera.<camera_name>).

You can see the difference using HA Developers Tools -> Services:

camera.snapshot
  entity_id: camera.<camera_name>_person
  filename: '/config/www/test_person.jpg'

and then:

camera.snapshot
  entity_id: camera.<camera_name>
  filename: '/config/www/test_view.jpg'

You should get different frame size.

exactly! I used this docsā€¦ well then i would like to know where to configure the snapshotā€¦ my resolution is too small ā€¦ i donā€™t recognise anything on it :stuck_out_tongue:

antoher issue is that it will not create other sensors camera etc than personā€¦ thozght by standard it will have object for person vehicle and animals ? who has other than person up and running?

again my config:

http://pastebin.ddns.me/qobareduya.bash

if spmeone culd point me to the right docs / running config :slight_smile: ?

Now I see where we differ. I donā€™t use the other cameras generated by frigate. I only use the ones for object detection.
I donā€™t recall where I read it, but I think Blake mentioned having snapshots saved to the media folders as something on the roadmap so there could be changes to come.
You could probably make an issue on github for this to get his attention.

I added some zones on some of my cameras so i could use the sensors provided just to turn on lights. But the parent camera setting are tracking different objects (person, cat). Is it possible to exlude detenction for cat in a zone but keep traked for the parent camera?

Can you not just use a condition in your automation Iā€™d it detects a person @Melkor82

I can but i got saved pics for cats in that zone too, flooding my media folder

How did u get cats detection working? I only get person.

Cat and dog detection is highly dependent on your cameraā€™s position. The models used a trained from straight on, rather than typical security camera position of being above. This makes mine miss quite a few and often mis-categorizes people as dogs.

Hahaā€¦ well that explains why my wife (nearly) always is recognized as a dogā€¦ Couldnā€™t explain it to her.
:smile::smile:

1 Like