Just a quick sidenote here. It might be worthwhile to take a look at TileBoard, which is a third party dashboard for HA written entirely in JS. I use it everywhere, because it’s so much more flexible than Lovelace (and nicer, but that’s subjective 
).
The important thing about Tileboard with respect to this thread is the way cameras are handled. While streams still run over the HA stream component to get them to HLS (and all caveats mentioned in this thread apply), it will entirely bypass the Lovelace camera rendering, which is pretty inefficient for some reason. Instead it will directly use hls.js (a very popular Javascript HLS decoder, also used by HA internally), which you can tweak to your hearts content - including changing buffer sizes. That’s how I managed to get my stream lag down to less than 5 seconds.
And if you used camera still images, Tileboard gives you entirely flexibility to refresh your camera images with whatever frequency you like. You could, for example, update your camera shots every 5 seconds by default, but switch to 1 second updates when motion is detected. Or change the update frequency at night, according to your presence status, etc.