As I said above, my 640x480 Android IP Webcam stream from a cheap old phone over wifi to HA on a low-power Banana Pi M1 is immediate (I can hear traffic directly and watch it in HA with sub-second delay), and as smooth as the IP webcam frontend in the card itself, no need for the pop-up window.
The phone is also recording the stream and ftping it to the Pi every 15 minutes with no apparent slowdown.
Perhaps it’s the video format that’s the issue? I’m using MPEG4 as the phone is too old to cope with H.264/5.
ok, interesting topic here!
I have some Foscam c1 as well, they work well in lovelace, the stream is quite OK, alo have the 10 sec lag behind
But i have exposed them to my google HUB , its horrible to watch the camera streams there… always buffering…
i also have a Synology, (its virtualized on a ESXI server), same for ma HA…
i am running it on a HP Xeon server with more then enough memory, cpu …
i have tried the foscam , generic, onvif component, always lag…
now i am using the rtsp stream paths;, it gives the best results, but still horrible
what can i do to improve this? do i need to load that custom .py file? can it be loaded as custom component? do i need to change something also in that file?
I just installed a fresh homeassistant installation on a Pi3b+ and added only 1 camera… I got exactly the same result, without camera-view set to live I get 1 frame every 10 seconds or so and it when it refreshes, it’s maybe only 1 second behind, but when I switch to live view, I get 15 seconds of lag (which is actually 2-3 seconds behind my intel box I’m running now), even if I open and preload stream.
So it’s got to be something on the camera side. I’d love to see someone’s hikvision camera (not DVR) configuration that doesn’t have this issue. I’m running h.264 so to answer @Troon, that doesn’t seem to be any better than MPEG4.
To be clear, I’m running MPEG4 on old hardware and I do not have any delay issues.
EDIT: Just tried holding a phone showing the stream which has a clock overlaid on it in front of the camera whilst watching the stream on my laptop, and the delay is slightly over a second whilst running two simultaneous streams. Tried the same thing with the IP Webcam web front end, and the delay is about 3s, so HA is faster.
This always seems to have been an issue. I ended up setting up ffserver on a linux server and streaming my rtsp streams using ffserver/ffmpeg as mjpeg format. I had to seriously lower the size and resolution of the streams as well as lower the fps to 10 or less. This works great, however, it means that I’m always playing the video stream from each camera 24/7/365. I have over 20 cameras. This is a waste of resources as it’s highly CPU intensive an it causes a lot of additional I/O on the dvr disks as well as the Linux disk drive. I really wish my dvr and cameras supported mjpeg natively. This would make life so much easier.
Okay, holy crap, I finally figured it out and it was so damned simple.
I was playing with MJPG again based on your latest suggestions but only getting 1 frame every 10 seconds in my lovelace card and it wouldn’t open the detail view due to the fact that the streaming component was enabled and apparently that only works with h.264… So I commented out
stream:
in my lovelace config and when I restarted, all my h.264 cameras were near real time… The framerate in the frontend is only maybe .5-1 FPS, but there’s almost no lag (maybe 1-2 seconds), which is much more useful than the higher framerate… That’s all it was! Just turn of stream… Combine that with
camera_view: live
in the lovelace card, and everything works well.
Thanks @Troon , @rafale77, @Coolie1101 or bearing with me, you’ve been a great help in giving me hope that this was actually solvable. Now I feel confident to move forward with using hass for video doorbells and other things. @pergola.fabio, give it a try.
Argh, okay now I just realized why I stopped doing things this way… On my fire tablets and iOS, the cameras don’t really show up properly… They will show a few frames, then disappear and then come back… Damnit… I think I went through this before and settled on the stream component because they would at least display that way… What a bummer. Well I guess it was worth a try to see if that had gotten fixed.
For my case it would be very useful if I could have the stream component only apply to certain cameras in my config while having others not use it (to get rid of the delay). If you feel the same way, please vote for my feature request here
The stream component is really one of the parts in HA that has always been very problematic for me, due to the way it works. It converts the RTSP stream into HLS on the fly. Because it needs to support low power embedded systems, it can’t transcode the stream. So it has to cut at I-Frame boundaries. It will always buffer 3 HLS segments, this is hardcoded. If you set the camera to encode an I-Frame every second, you will get a minimum delay of 3 seconds (3 segments at I-Frame boundary, with an I-Frame every second). The HLS decoder used for the frontend will buffer again, which will likely add more delay.
I don’t use Lovelace, but an external UI (Tileboard), so I get the streams over the HA websocket API and decode them directly with HLS.js, where I can manually change the buffer size in the code. With all that I get around 4 second delays on my Hikvisions. That’s pretty much the minimum latency you’ll be able to realistically get with the unmodified HA stream component. That is OK for me, as realtime recording is done by an external NVR anyway.
I haven’t tried to change the hardcoded 3 segment buffer in the stream component. I vaguely remember from the HLS specs that 3 segments minimum are required by the standard, but possibly it will work with less. Native camera apps don’t have this problem as they don’t go the RTSP->HLS->display route, but decode and display the RTSP stream directly (or use a proprietary streaming format).
Hi all, as a thank you to all of you who inspired me to keep dicking around with my camera config, I have done a long write up about everything I learned about the strengths and limitations of ~10 different camera configs I ran. I think at this point I have tried pretty much every combination of camera components with and without stream. You can see my findings here:
RTSP is not suitable for direct viewing in a web browser while HLS is. That’s why it’s converted first.
Yes and no. The problem is not the conversion. There is no actual transcoding of the video stream done. The RTSP h264 packets are simply extracted and repackaged into a different container. The CPU load to do that is insignificant, even in a slow language like Python. The lag problem is inherent to the HLS stream format itself and the buffering that it incurs. So yes, if you create the HLS faster than HA can do it (say for example using libav in C++ to do it), then the time saved will reduce the lag. But don’t expect much of an improvement, because that’s not the bottleneck. You will also lose the advantage of having the data proxied over HA, unless you write your own integration.
Native camera events are completely separate and not part of the stream integration.
Just updated my repo above, jumping from 0.112.5 all the way to 118.3. Apparently had no breaking changes so it is pretty amazing. It appears that as part of all the speed optimizations/improvements of 0.113 have changed the way HA calculates scan_intervals for image processing and either the timing is more accurate now or HA is now sucking up a lot more resources with cameras as I am getting warnings that my inferences are taking longer than the intervals with the same previous settings. I used to get these when I had much shorter intervals and have optimized the different camera streams and intervals to prevent this. I think it is rather the former as I am seeing my CPU and GPU utilization both double.
