WTH does a streaming camera display live video 15 seconds delayed by default

[lmamakos (Louis Mamakos)]

I understand why there’s a 15 second buffer, so you an start recording before an event fired. But often it’d be nice to bring up a lovelace card and display the more recent/current video coming from the camera. You can grab the scrubber knob and drag it forward to remove much of the delay, but I’d like to be able to configure a card with as little delay as possible.

When the doorbell rings, I’d like to see who’s there now, rather than wait and watch them walk up to the door.

[jschwalbe (Justin S)]

Vote for your own too

[thundergreen]

I totally agree it’s odd having a delay of 15 seconds

[hunterjm]

As @uvjustin points out in more detail on the post below, it’s due to the underlying technology that is used to support live streaming across the internet with robust browser compatibility (HLS). You will also notice a similar delay in any live streaming platform (Facebook Live/YouTube Live/Twitch/etc).

I tried all the camera platforms so you don't have to

As @HeyImAlex points out, HLS will have latency by its nature. We are taking a stream and packaging it up into segments which get served over http. Since segments are discrete there will be a latency of at least one segment length. Then the player itself might buffer another few segments. Our implementation of HLS already uses a non-standard fragment duration (Apple recommends 6 seconds, we use 2 seconds). It will probably be hard for us to get to less than ~4-5 seconds of latency using HLS. Tha…

[uvjustin (Justin Wong)]

As @hunterjm mentioned, there is inherently going to be some delay in HLS. There are perhaps some things which we can tweak, but as there are tradeoffs with respect to segment duration/bandwidth/lag/stalling/CPU usage there’s no magic formula.
@lmamakos you mention that there’s a 15 second buffer which you can drag forward to remove much of the delay. This brings up 2 things:

1. As there are probably 3 segments buffered, this means your segment lengths are around 5 seconds, which is longer than our target durations of 1 second for HA version<=0.114 and 2 seconds for HA version 0.115. Do you know what your camera keyframe rate is set to? Do you have an option like H264+ or H265+ enabled? Those will increase segment durations since they reduce the number of keyframes sent.
2. Since HLS started as an Apple standard and was only later adopted elsewhere, we actually use 3 different ways to play it depending on the platform/browser. Safari and iOS devices can play it natively, while most everything else uses hls.js (https://github.com/video-dev/hls.js/). We also recently added ExoPlayer to play it through the Android HA app (https://github.com/google/ExoPlayer). Each of these players has its own buffering scheme. For example, hls.js by default starts 3 segments back from the live edge: https://github.com/video-dev/hls.js/blob/70fbbe9f77c37115e5112e1eb4a76a2ac7119903/docs/API.md#livesyncdurationcount . We could consider lowering this to 2 which might be a sweet spot (we can also set something similar on ExoPlayer, but I’m not sure if there’s an option to do it for Safari/iOS), but I think 1 will probably induce sporadic stalling.