I tried all the camera platforms so you don't have to

As @HeyImAlex points out, HLS will have latency by its nature. We are taking a stream and packaging it up into segments which get served over http. Since segments are discrete there will be a latency of at least one segment length. Then the player itself might buffer another few segments. Our implementation of HLS already uses a non-standard fragment duration (Apple recommends 6 seconds, we use 2 seconds). It will probably be hard for us to get to less than ~4-5 seconds of latency using HLS. That might be reasonable if you’re viewing the stream remotely. If you’re local and you want to reduce latency, you can probably use other methods which may have less latency but use more bandwidth (MJPEG) or require other TCP/UDP ports (just use the original RTSP stream).

The Low Latency HLS link you’ve shared is not just about tweaking/tuning. It’s an extension to HLS which involves a different encoding/http pipeline (it needs to produce encoded chunks before the whole segment is done muxing/encoding and it uses http/2 to send these chunks as soon as they are available). Unfortunately the pyav library we use to access libav won’t make it easy to do this. Edit: The PyAV package we use to access the ffmpeg libraries doesn’t make it easy to do the chunked encoding part, but we may be able to work around that. A bigger issue is actually http/2, since aiohttp does not look like it will support that anytime soon, but it looks like the http/2 requirement may have been relaxed. Anyway, as @hunterjm pointed out, ll-hls player/browser support is not really there either. We may revisit this sometime in the future, but probably not anytime soon.

Note - one significant source of lag can come from the use of H264+/H265+. These non-standard codecs reduce bandwidth by significantly reducing the number of keyframes sent. This ends up increasing the segment duration which increases latency. Also since they will cause the segment durations to vary and differ from the target segment durations, this will affect things like the lookback period in stream recordings. We should probably recommend against using these if latency is of any concern.

4 Likes