Viseron v3.0.0b1 - Self-hosted, local only NVR and AI Computer Vision software

Share your Projects!
21

Looks interesting. I’ll have to give it a try when I get time to sit down and play with this.

22

Has this been tested with a h265 camera?

I have tried adding codec: hevc_cuvid but if you look at the debug of it trying to load the stream it still says -c:v h264_cuvid is Viseron ignoring the hevc_cuvid command or do I need to add something to the hwaccel_args for it to decode h265?

The camera config


cameras:
  - name: Drive
    host: 192.168.1.10
    port: 554
    username: admin
    password: password
    path: /cam/realmonitor?channel=1&subtype=0
    width: 3840
    height: 2160
    fps: 20
    codec: hevc_cuvid
    motion_detection:
      interval: 1
      trigger: true
    object_detection:
      interval: 1
      labels:
        - label: person
          confidence: 0.9
23

What you have should be working! Looking at it right now.

24

This is what I could see with debug on in logging.


[2020-09-06 10:16:54] [lib.nvr.garage          ] [DEBUG   ] - NVR thread initialized
[2020-09-06 10:16:54] [lib.nvr.drive           ] [DEBUG   ] - Initializing NVR thread
[2020-09-06 10:16:54] [lib.camera.drive        ] [DEBUG   ] - Initializing ffmpeg RTSP pipe
[2020-09-06 10:16:54] [lib.camera.drive        ] [INFO    ] - Resolution: 3840x2160 @ 20 FPS
[2020-09-06 10:16:54] [lib.camera.drive        ] [DEBUG   ] - FFMPEG decoder command: ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -c:v h264_cuvid -rtsp_transport tcp -i rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0 -f rawvideo -pix_fmt nv12 pipe:1

So it looks like it’s still trying to use h264. For now I am using the low res sub-stream that’s encoded in h264. Are you planning to send the jpg screen shots to HA via mqtt like Frigate does?

25

Latest release now properly allows you to specify your own codec. Check it out and see if it works for you.

You mean images of detections?
I am currently working on zones, and with this i will add functionality to send images over MQTT with drawn zones/objects for easy tuning.

26

1.3.0 - Basic support for mjepg and bug fixes

Changes and new Features

  • Support for mjpeg streams. To use it, add stream_format: mjpeg to your camera configuration.
    This is not fully tested since i dont have access to an mjpeg camera.

Fixes

  • Fixed interval for object detector and motion detector. It now allows floats.
  • No longer uses infinite memory when lookback: 0.
  • Fixed user supplied codec being overwritten bu default value.
27

1.4.0.b1 - Initial support for zones
I just added the first beta for zones. Check out the release page on github to see how to configure!
Right now zones wont trigger recordings but that will come in the upcoming betas.
New binary sensors will be created in Home Assistant to signal if any objects are in any zone.

28

I am intrigued by this and I got it installed in Docker on unRAID. If I understand what this does, I could see this being useful for displaying low quality clips which I could use to determine if I need to look more closely at the 4k recordings.

Unfortunately the only streams I have access to are only 640x352@30fps. Is that too low to be useful for detection? I have no idea what any of these settings are. Do you have any advice on my settings?

[2020-09-09 16:26:49] [root ] [INFO ] - -------------------------------------------
[2020-09-09 16:26:49] [root ] [INFO ] - Initializing...
[2020-09-09 16:26:49] [root ] [DEBUG ] - Starting cleanup scheduler
[2020-09-09 16:26:49] [root ] [DEBUG ] - Running initial cleanup
[2020-09-09 16:26:49] [lib.cleanup ] [DEBUG ] - Running cleanup
[2020-09-09 16:26:49] [lib.mqtt ] [INFO ] - Initializing MQTT connection
[2020-09-09 16:26:49] [lib.detector ] [INFO ] - Initializing detection thread
[2020-09-09 16:26:49] [root ] [INFO ] - Initializing NVR threads
[2020-09-09 16:26:49] [lib.nvr.driveway_viseron] [DEBUG ] - Initializing NVR thread
[2020-09-09 16:26:49] [lib.camera.driveway_viseron] [DEBUG ] - Initializing ffmpeg RTSP pipe
[2020-09-09 16:26:49] [lib.camera.driveway_viseron] [INFO ] - Resolution: 640x352 @ 30 FPS
[2020-09-09 16:26:49] [lib.camera.driveway_viseron] [DEBUG ] - FFMPEG decoder command: ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -rtsp_transport tcp -i rtsp://hass:[email protected]:554/h264Preview_01_sub -f rawvideo -pix_fmt nv12 pipe:1
[2020-09-09 16:26:49] [lib.camera.driveway_viseron] [DEBUG ] - Starting decoder thread
[2020-09-09 16:26:49] [lib.camera.driveway_viseron] [DEBUG ] - Starting capture process
[2020-09-09 16:26:49] [lib.camera.driveway_viseron] [DEBUG ] - Starting decoder thread
[2020-09-09 16:26:49] [lib.recorder ] [DEBUG ] - Initializing ffmpeg recorder
[2020-09-09 16:26:49] [lib.recorder ] [DEBUG ] - FFMPEG encoder command: ffmpeg -hide_banner -loglevel panic -f rawvideo -pix_fmt nv12 -s:v <width>x<height> -r <fps> -i pipe:0 -y <file>
[2020-09-09 16:26:49] [lib.nvr.driveway_viseron] [DEBUG ] - NVR thread initialized
[2020-09-09 16:26:49] [lib.nvr.porch_viseron ] [DEBUG ] - Initializing NVR thread
[2020-09-09 16:26:49] [lib.camera.porch_viseron] [DEBUG ] - Initializing ffmpeg RTSP pipe
[2020-09-09 16:26:49] [lib.camera.porch_viseron] [INFO ] - Resolution: 640x352 @ 30 FPS
[2020-09-09 16:26:49] [lib.camera.porch_viseron] [DEBUG ] - FFMPEG decoder command: ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -rtsp_transport tcp -i rtsp://hass:[email protected]:554/h264Preview_03_sub -f rawvideo -pix_fmt nv12 pipe:1
[2020-09-09 16:26:49] [lib.camera.porch_viseron] [DEBUG ] - Starting decoder thread
[2020-09-09 16:26:49] [lib.camera.porch_viseron] [DEBUG ] - Starting capture process
[2020-09-09 16:26:49] [lib.camera.porch_viseron] [DEBUG ] - Starting decoder thread
[2020-09-09 16:26:49] [lib.recorder ] [DEBUG ] - Initializing ffmpeg recorder
[2020-09-09 16:26:49] [lib.recorder ] [DEBUG ] - FFMPEG encoder command: ffmpeg -hide_banner -loglevel panic -f rawvideo -pix_fmt nv12 -s:v <width>x<height> -r <fps> -i pipe:0 -y <file>
[2020-09-09 16:26:49] [lib.nvr.porch_viseron ] [DEBUG ] - NVR thread initialized
[2020-09-09 16:26:49] [lib.nvr.driveway_viseron] [DEBUG ] - Waiting for first frame
e[80De[1Ae[K[2020-09-09 16:26:49] [lib.nvr.porch_viseron] [DEBUG ] - Waiting for first frame, message repeated 2 times
[2020-09-09 16:26:49] [root ] [INFO ] - Initialization complete
[2020-09-09 16:26:55] [lib.camera.driveway_viseron] [DEBUG ] - Running object detection at 1s interval, every 30 frame(s)
[2020-09-09 16:26:55] [lib.camera.driveway_viseron] [DEBUG ] - Running motion detection at 1s interval, every 30 frame(s)
[2020-09-09 16:26:57] [lib.camera.porch_viseron] [DEBUG ] - Running object detection at 1s interval, every 30 frame(s)
[2020-09-09 16:26:57] [lib.camera.porch_viseron] [DEBUG ] - Running motion detection at 1s interval, every 30 frame(s)
[2020-09-09 16:26:57] [lib.camera.porch_viseron] [DEBUG ] - Running object detection at 1s interval, every 30 frame(s)
[2020-09-09 16:26:57] [lib.camera.porch_viseron] [DEBUG ] - Running motion detection at 1s interval, every 30 frame(s)
[2020-09-09 16:27:03] [lib.nvr.driveway_viseron] [DEBUG ] - First frame received
e[80De[1Ae[K[2020-09-09 16:27:04] [lib.nvr.porch_viseron] [DEBUG ] - First frame received, message repeated 2 times
[2020-09-09 16:28:43] [lib.motion ] [DEBUG ] - Motion frames: 1, area: 15820.5
[2020-09-09 16:28:47] [lib.motion ] [DEBUG ] - Motion frames: 2, area: 15669.0
[2020-09-09 16:28:52] [lib.motion ] [DEBUG ] - Motion frames: 3, area: 14192.5
[2020-09-09 16:28:52] [lib.nvr.porch_viseron ] [DEBUG ] - Motion detected! Starting object detector
[2020-09-09 16:28:56] [lib.motion ] [DEBUG ] - Motion frames: 4, area: 13317.0
[2020-09-09 16:29:00] [lib.motion ] [DEBUG ] - Motion frames: 5, area: 13142.5
[2020-09-09 16:29:04] [lib.motion ] [DEBUG ] - Motion has ended
[2020-09-09 16:29:05] [lib.nvr.porch_viseron ] [DEBUG ] - Not recording, pausing object detector
[2020-09-09 16:29:05] [lib.nvr.porch_viseron ] [DEBUG ] - Not recording, pausing object detector

logging:
  level: debug

recorder:
  lookback: 10
  timeout: 10
  retain: 7
  folder: /recordings

cameras:
  - name: Driveway Viseron
    host: 10.0.0.2
    port: 554
    username: <username>
    password: <password>
    path: /h264Preview_01_sub
    width: 640
    height: 352
    fps: 30
    #hwaccel_args: "-hwaccel vaapi -vaapi_device /dev/dri/renderD128"
    motion_detection:
      interval: 1
      trigger: true
    object_detection:
      interval: 1
      labels:
        - label: person
          confidence: 0.8
        - label: car
          confidence: 0.8
        - label: truck
          confidence: 0.8

  - name: Porch Viseron
    host: 10.0.0.2
    port: 554
    username: <username>
    password: <password>
    path: /h264Preview_03_sub
    width: 640
    height: 352
    fps: 30
    #hwaccel_args: "-hwaccel vaapi -vaapi_device /dev/dri/renderD128"
    motion_detection:
      interval: 1
      trigger: true
      timeout: true
    object_detection:
      interval: 1
      labels:
        - label: person
          confidence: 0.8
          height_min: 0.1481
          height_max: 0.7
          width_min: 0.0598
          width_max: 0.36
        - label: pottedplant
          confidence: 0.9


# MQTT is optional
mqtt:
  broker: 10.0.0.5
  port: 1883
  username: <username>
  password: <password>
29

A question about configuration:

Where does one get the models for object detection? It references a /detectors/models path, but none of the example docker invocations mention them, nor are they described in the README. I do see tensorflow model garden, model zoo, etc., but haven’t used tf at all before this project.

30

Generally a higher quality image will result in better detections, but the image is resized anyway to fit the model, and they are usually around 300x300, so it should work fine for you.

31

Models are included by default in the container.
If you are using edgetpu you can read more about it here.

darknet is using OpenCVs implementation of YOLO, and here there are a few different models to choose from. This repository has a few to choose from.
Genrally YOLOv4 is better than YOLOv3, and the -tiny versions are smaller in size and needs less CPU/GPU to run, but with less accuracy.

32

1.4.0 - Zones and more MQTT sensors
Zones are now released together with a lot of fixes and performance enhancements.
This version also improves the MQTT integration with Home Assistant, still a little way to go tho until its 100%
Check out the release notes

1 Like
33

1.5.0 - Mask and improvements to motion detector
Masks can now be configured to limit motion detection to not run in certain noisy areas.
They are configured the same way as zones. Check the README for an example.

There are lots of other changes in this one, as well as a breaking one so i suggest you check out the release notes on GitHub

1 Like
34

Hello, i like your project. Can you give example how can i use notify with android app and motion detection from viseron? I’ve got notification automated, but what about picture or video attached ?

35

I use iOS myself so im not sure how Android works, but i just use attachemnt: camera feature.

See here. In an upcoming release Viseron will store the triggering image in a file and/or a camera entity in Home Assistant for easy attachment.

36

Thank you. Sadly this is for ios only, android notification cant use camera directly… Notification need to be with saved picture attachment. I will keep using nodered for now

37

do you think it is possible to use Viseron in a proxmox container without GPU ( only virtual gpu ) but with a Google Coral EdgeTPU?
Could it handle the load of around 8 cameras?

38

Thank you. Sadly this is for ios only, android notification cant use camera directly… Notification need to be with saved picture attachment. I will keep using nodered for now

@TUISTERa Ahh too bad. Viseron will serve you an image in the future which you can use in your notification.

do you think it is possible to use Viseron in a proxmox container without GPU ( only virtual gpu ) but with a Google Coral EdgeTPU?
Could it handle the load of around 8 cameras?

Should work fine, but it really depends on your cameras FPS and resolution.
If you use lower FPS to 6 for example, and then fine tune the interval for the motion and object detector to not run so agressively, i think it would be okay.

39

1.6.0b1 - Face recognition and MQTT changes
New beta is here with support for face recognition!
Check out the release notes for how to set this up.

There are a few breaking changes if you previously set your own discovery prefix, or manually subscribed to MQTT topics, so be sure to read those thoroughly.

40

I tried setting this up but I get an error message. I have the following configuration:

# See the README for the full list of configuration options.
cameras:
  - name: Voordeur
    mqtt_name: viseron_voordeur
    host: deurbel.lan
    port: 8081
    path: / 
    width: 800
    height: 600
    stream_format: mjpeg
    fps: 6
    motion_detection:
      interval: 1
      trigger_detector: false
    object_detection:
      interval: 1
      labels:
        - label: person
          confidence: 0.9
        - label: pottedplant
          confidence: 0.9

# MQTT is optional
mqtt:
  broker: 10.0.1.21
  port: 1883
#  username: <if auth is enabled>
#  password: <if auth is enabled>

recorder:
  lookback: 10
  timeout: 10
  retain: 7
  folder: /recordings

When starting the container I get the following in the log:

[2020-10-05 12:10:41] [root ] [INFO ] - -------------------------------------------
[2020-10-05 12:10:41] [root ] [INFO ] - Initializing...
[2020-10-05 12:10:41] [root ] [DEBUG ] - Starting cleanup scheduler
[2020-10-05 12:10:41] [root ] [DEBUG ] - Running initial cleanup
[2020-10-05 12:10:41] [lib.cleanup ] [DEBUG ] - Running cleanup
[2020-10-05 12:10:41] [lib.mqtt ] [INFO ] - Initializing MQTT connection
[2020-10-05 12:10:41] [lib.detector ] [DEBUG ] - Initializing object detector darknet
[2020-10-05 12:10:41] [lib.detector ] [DEBUG ] - OpenCL activated
[2020-10-05 12:10:42] [lib.detector ] [DEBUG ] - Object detector initialized
[2020-10-05 12:10:42] [root ] [INFO ] - Initializing NVR threads
[2020-10-05 12:10:42] [lib.nvr.voordeur ] [DEBUG ] - Initializing NVR thread
[2020-10-05 12:10:42] [lib.camera.voordeur ] [DEBUG ] - Initializing ffmpeg RTSP pipe
[2020-10-05 12:10:42] [lib.camera.voordeur ] [DEBUG ] - Resolution: 800x600 @ 6 FPS
[2020-10-05 12:10:42] [lib.camera.voordeur ] [DEBUG ] - FFMPEG decoder command: ffmpeg -hide_banner -loglevel fatal -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i http://deurbel.lan:8081/ -f rawvideo -pix_fmt nv12 pipe:1
[2020-10-05 12:10:42] [lib.motion.voordeur ] [DEBUG ] - Initializing motion detector
[2020-10-05 12:10:42] [lib.motion.voordeur ] [DEBUG ] - Motion detector initialized
[2020-10-05 12:10:42] [lib.camera.voordeur ] [DEBUG ] - Starting decoder thread
e[80De[1Ae[K[2020-10-05 12:10:42] [lib.camera.voordeur] [DEBUG ] - Starting decoder thread, message repeated 2 times
[2020-10-05 12:10:42] [lib.nvr.voordeur ] [DEBUG ] - Starting camera
[2020-10-05 12:10:42] [lib.camera.voordeur ] [DEBUG ] - Starting capture thread
[2020-10-05 12:10:42] [lib.recorder.voordeur ] [DEBUG ] - Initializing ffmpeg recorder
[2020-10-05 12:10:42] [lib.camera.voordeur ] [DEBUG ] - Performing a sanity check on the ffmpeg command
[2020-10-05 12:10:42] [lib.recorder.voordeur ] [DEBUG ] - FFMPEG encoder command: ffmpeg -hide_banner -loglevel error -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo -pix_fmt nv12 -s:v <width>x<height> -r <fps> -i pipe:0 -y -c:v h264_vaapi -vf format=nv12|vaapi,hwupload <file>

[2020-10-05 12:10:42] [lib.nvr.voordeur ] [DEBUG ] - NVR thread initialized
[2020-10-05 12:10:42] [lib.mqtt ] [DEBUG ] - MQTT connected with result code 0
[2020-10-05 12:10:42] [lib.nvr.voordeur ] [DEBUG ] - Waiting for first frame
[2020-10-05 12:10:42] [root ] [INFO ] - Initialization complete
[2020-10-05 12:10:43] [lib.camera.voordeur ] [ERROR ] - Error starting decoder pipe! Option stimeout not found.

Retrying in 5 seconds
e[80De[1Ae[K[2020-10-05 12:10:49] [lib.camera.voordeur] [ERROR ] - Error starting decoder pipe! Option stimeout not found.

Retrying in 5 seconds, message repeated 2 times
e[80De[1Ae[K[2020-10-05 12:10:55] [lib.camera.voordeur] [ERROR ] - Error starting decoder pipe! Option stimeout not found.

Retrying in 5 seconds, message repeated 3 times
e[80De[1Ae[K[2020-10-05 12:11:01] [lib.camera.voordeur] [ERROR ] - Error starting decoder pipe! Option stimeout not found.

can you help me to get it running?