Hi asmirnou,
Thanks for your reply and suggestions I have enabled hardware acceleration but this results in an error and no GPU acceleration or decoding takes place
watchdog WatchDog WARNING : Thread def (FFmpegDecoder) is not alive, restarting...
watchdog WatchDog WARNING : Thread def1 (FFmpegDecoder) is not alive, restarting...
unifi FFmpegDecoder INFO : [h264 @ 0x557d9cd58940] error while decoding MB 60 35, bytestream -22
Thread-36 werkzeug INFO : 192.168.1.125 - - [20/Jul/2020 19:33:59] "GET /metrics HTTP/1.1" 200 -
unifi FFmpegDecoder INFO : [AVHWDeviceContext @ 0x557d9cd9ee40] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
unifi FFmpegDecoder INFO : [AVHWDeviceContext @ 0x557d9cd9ee40] Failed to initialise VAAPI connection: -1 (unknown libva error).
unifi FFmpegDecoder INFO : Device creation failed: -5.
unifi FFmpegDecoder INFO : Device setup failed for decoder on input stream #0:1 : Input/output error
def FFmpegDecoder INFO : [AVHWDeviceContext @ 0x556908349a20] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
def FFmpegDecoder INFO : [AVHWDeviceContext @ 0x556908349a20] Failed to initialise VAAPI connection: -1 (unknown libva error).
def FFmpegDecoder INFO : Device creation failed: -5.
def FFmpegDecoder INFO : Device setup failed for decoder on input stream #0:0 : Input/output error
def1 FFmpegDecoder INFO : [AVHWDeviceContext @ 0x56132f6e8180] libva: va_getDriverName() failed with unknown libva error,driver_name=(null)
def1 FFmpegDecoder INFO : [AVHWDeviceContext @ 0x56132f6e8180] Failed to initialise VAAPI connection: -1 (unknown libva error).
def1 FFmpegDecoder INFO : Device creation failed: -5.
This is the only thing that gives me something but still through errors and no hardware acceleration,
So I am paying the price of living on the bleeding edge with cuda… I can’t launch it and I am getting an cuda initialization failure running cuda11.0, cuDNN 8.0.1 and TensorRT 7.1.3.4. I was able to compile and run opencv but not watsor…
If VAAPI doesn’t work for you, use hardware acceleration on Tesla K80:
...
- -c:v
- h264_cuvid
- -i
...
or
...
- -hwaccel
- cuvid
- -i
...
GPU will be responsible for decoding and detecting, which is less effective than using dedicated HW accelerated decoder, but better than no acceleration at all.
Managed to got mine up and running but am struggling with an rtsp udp stream. I keep on getting tcp connection refusals in spite of having added the rtsp_tranport udp. Seems to be more of a problem with ffmpeg but… I have no problem running ffmpeg on that stream…
edit: streaming directly from the camera instead of a proxy apparently fixed it.
Under metrics, I am seeing inference time of 5.0. Is it in seconds?
Just installed the add-on in my supervised install of HA.
I’m wondering how to run the command to deteck what hardware accels I have available. In the comments of the sample config you said # ffmpeg -hwaccels.
Thing is that as I understand it the supervised HA has its own ffmpeg (maybe I misunderstood that), but anyway I don’t know where to run that command.
Any advice?
Thanks asmirnou, had some time today to get back into this, when trying to refer to any form of hardware configuration the system stops responding, removing all hardware references in the config:
ive installed Watsor via Hass`s Addon . My hass is running on Ubuntu Intel NUC i3 8gb ram
http:
port: 8080
ffmpeg:
decoder:
- -hide_banner
- -loglevel
- error
- -nostdin
- -hwaccel # These options enable hardware acceleration of
- vaapi # video de/encoding. You need to check what methods
- -hwaccel_device # (if any) are supported by running the command:
- /dev/dri/renderD128 # ffmpeg -hwaccels
- -hwaccel_output_format # Then refer to the documentation of the method
- yuv420p # to enable it in ffmpeg. Remove if not sure.
- -fflags
- nobuffer
- -flags
- low_delay
- -fflags
- +genpts+discardcorrupt
- -i # camera input field will follow '-i' ffmpeg argument automatically
- -f
- rawvideo
- -pix_fmt
- rgb24
encoder: # Encoder is optional, remove the entire list to disable.
- -hide_banner
- -loglevel
- error
- -hwaccel
- vaapi
- -hwaccel_device
- /dev/dri/renderD128
- -hwaccel_output_format
- yuv420p
- -f
- rawvideo
- -pix_fmt
- rgb24
- -i # detection output stream will follow '-i' ffmpeg argument automatically
- -an
- -f
- mpegts
- -vcodec
- libx264
- -pix_fmt
- yuv420p
- -vf
- "drawtext='text=%{localtime\\:%c}': x=w-tw-lh: y=h-2*lh: fontcolor=white: box=1: [email protected]"
detect:
- person:
area: 10 # Minimum area of the bounding box an object should have in
# order to be detected. Defaults to 10% of entire video resolution.
confidence: 60 # Confidence threshold that a detection is what it's guessed to be,
# otherwise it's ruled out. 50% if not set.
- car:
area: 15 # Minimum area of the bounding box an object should have in
# order to be detected. Defaults to 10% of entire video resolution.
confidence: 60
cameras:
- porch: # Camera name
width: 640 #
height: 480 # Video feed resolution in pixels
input: !ENV "rtsp://admin:[email protected]:554/Streaming/Channels/102"
detect: # The values below override
- person: # detection defaults for just
- car:
I did not see any error on Watsors Log . Ive setup my Hikvisions IPCAM Substream to 640x480 , 6 fps . ive played with multiple fps ranging from 4 - 12 . Here is the Metric output
@asmirnou, I am also finding some oddities with ffmpeg and I am pretty sure it is more related to ffmpeg itself than watsor. Have you thought about using opencv (which also use ffmpeg) to manage the video streams? It seems a little more efficient and has python API. That’s what I ended up using on my facial recognition component and made my python code very simple, only a dozen of lines and also somehow significantly reduced my cpu utilization: For one video stream this is what I am observing:
ffmpeg with cpu: 30%
ffmpeg with GPU: 40%
opencv with GPU: 6%
This is my camera code to maintain the video stream on:
That paragraph in documentation describes a different type of deployment, where the command line is available on the machine. In Hassio that’s difficult cause a lot is hidden behind the supervisor, the only way I know of to get inside through the SSH add-on. Then one needs to “exec” into docker container where Watsor is running in order to run ffmpeg -hwaccels.
What machine are you running HA on? Maybe someone already set it up.
Getting the camera to work efficiently in FFmpeg is the hardest part of the whole setup. Unfortunately, due to the variety of devices, formats and hardware, it is difficult to advise without seeing…
A good practice is to get FFmpeg working with the camera outside of Docker container by running FFmpeg from the command line and then move the options to the application config.
Right. The only thing I can think of is your camera is mounted high and a person walking in front of it occupies less than 10% of the image. Set the area threshold to zero, will it show something up?
I hardly believe that OpenCV can be so much more effective than FFmpeg. Decoding is not performed by any of them anyway, but by the avcodec library. Most probably the hardware acceleration wasn’t enabled in FFmpeg or the input conditions were different.
I am scratching my head too to be honest but running FFmpeg as a stand-alone with the same command gets me the same result. I can’t explain it either and indeed openCV uses FFmpeg and the avcodec library and I did verify that hardware acceleration is enabled from the nvidia-smi output. The CPU usage is significantly higher and even higher than using CPU decoding alone… it is strange. I see the exact opposite using openCV on the same camera:
openCV CPU: +20%
openCV GPU: +6%
FFMpeg CPU: +30%
FFMpeg GPU: +40%
The output is a bit different with FFMPEG the RGB24 it appears…
ive reduce both person and car area to 0 and even reduce confidence to 10. that car in the picture wasnt detected. few min ago ive seen car pass the camera but non detected. hermm.
Due to the absence of any hardware accelerator the detection falls over to CPU. To minimize the load (as far as CPU is not suitable for such kind of mathematical task) the lightest possible model is used trained on a smaller dataset. That model doesn’t recognize the yellow van in your example and nothing else on that image. The bigger models I tried were managed to detect several objects of very low confidence ~10%. The van was detected 17% certain. This is quite unusual as most often any model even the lightest one provides decent accuracy.
Try rotating the camera slightly to get better angle.
any ideas thanks. Andy.
