Realtime camera streaming without any delay - WebRTC

Share your Projects!
516

Do you connect to go2rtc via RTSP with external access?

Sorry I misread, no I dont, from WAN I connect to HA wich connect to go2rtc.

I did a lot of tests and I’m in doubt, the GenericCamera + go2rtc works well (not perfect… as I mentionned some delay that grow sometimes for an unknow reason, but I think the main concern is TCP is not made to stream video or anything else…)

So I am in doubt because I prefer your HA Card GUI but the GenericCamra+go2rtc is more reliable. But none are perfect yet.

Note that, I monitoredCPU and network IO and they was far to be overloaded.

517

Because RTSP does not work like every other soft in the world. In the case of UDP, it works the reverse. Server (ex Camera) send data to Client’s (ex VLC) UDP port.

518

But when I sniff packet with Wireshark when accessing from WAN , using your component WebRTC I see a flow of UDP Packet and it is very reesponsive les than 1 secoond of delay, when I do the same with generic cam using gotortc I see a flow of TCP packet and it get more and more delay (with CPU and LAN far to be overloaded), really your component WebRTC Cards is the way to go, but sorry to say that for now it is not enought robust, lot of JS Exception / Disconnection, need refresh etc… but when it works Wow, this is as fast than video chat program

519

Check my answer here: Security cameras doubt- go2rtc , RTSPtoWeb and what is between - #4 by AlexxIT

GenericCamera + go2rtc gives you absolutely nothing to reduce latency. You continue to use the HLS protocol. Everything changes when you turn on WebRTC mode. You can do this in a bunch of different ways: RTSPtoWebRTC Integration, WebRTC Camera Integration, go2rtc Addon Web UI…

520

I have RTSPtoWebRTC connected to a go2rtc instance, it seems to give a more reliable stream but still TCP, when I use WebRTC card I got a perfect video (wich use UDP and work with no latency even on WAN), thaat’s the point, the only problem is that WebRTC card fail half of the time, that’s the only problem…

Do you talk about a new version of this player when ou say WebRTC proto" ?

This player is perfect when it works.

521

Yes: Imou Life - cloud integration - #109 by mhoogenbosch

522

Perhaps go2rtc WebRTC port setting will support both TCP and UDP in the future.

Right now it only supports TCP because it can be easily tunneled. But UDP can only be proxied by users with a public IP address.

But if WebRTC succeeds in doing UDP hole punching for your network setup, it will use UDP. But you need to setup STUN server in the RTSPtoWebRTC Integration options.

523

Waht do you mean, every one have a public addresses ?

But if WebRTC succeeds in doing UDP hole punching for your network setup, it will use UDP

It already does, It use UDP and works nearly perfect from WAN (except the JS part, that often crash), wich is not the case for generic camera that use TCP and are laggy. As far as I understood, WebRTC (yours compoent) get ride of RTSPToWebRTC config and do its stuff itself ? at least it use UDP itself

524

This is the traffic I got when using WebRTC custom component

image
image1144×350 44.3 KB

Note that there are two gateway between the public IP and the local computer, I just need forward both TCP/UDP twice, one for each gateway. So sure it works and video have no lantency except some time it hang and restart (but always that JS part crash is a pain… when moving window or changing page)

I think that if client receive too much packets and that it cant handle all of them (because of a client pause or anything else), after some time it should skip a bunch of them, it will make artifacts visible for seconds, as it can be see sometime on TV/Youtube, but it would maybe make it more reliable, but you may already do that ?

525

Note:
To make it easily usable by browser & phone app with the same URL, I have configured on my router a local DNS that resolve my HA house.domain.tld with its local IP (forward anything else to 8.8.8.8) and use a server with DDNS that resolve my WAN IP when I am not connected on my Wifi router.

So connected to my Wifi (LAN) I got :
ha.domain.tld => local IP of HA
Outside on WAN :
ha.domain.tld => my Internet public Box IP (updated every ten minutes by a request from a device in my LAN)

For Chrome it require to desactivate DNS protection, because chrome dont care about your router/DHCP or computer DNS config…

EDIT :
Note about the UDP exchange stream screenshot :

It IMHO would work a lot better if packets was fullfilled, packet payload is really too small, 70 bytes for some , it least payload than all the network layer header to carry them!!, it could take 2 or 5 times less packet by making them largers. UDP can go up to nearly 65000 bytes (this would be too much but at least 2048/4096 would make more sense) but I may missed something .

EDIT2:
I have tried different configuration you mentioned from different internet access point and the best by far of all is :
3 - Generic Camera > RTSPtoWebRTC Inegration > go2rtc Addon > WebRTC

Once the vidéo is started, if I dont touch anything (just go on/off fullscreen), it works perfectly and I can see UDP packet scrolling under Wireshark, zero latency on video, if it was reliable it would be a perfect solution, problems arrive as soon as there is a disconnection (focus lost, change page, minimisation, etc…) or any DOM modification.

What I dont understand is that when it is not working/breaked after one of the things I mentioned above, there are tons of STUN call, severals per seconds, and if I do not reload or do something it never succeed to reconnect, I usually have to refresh 2 or 3 times

image
image961×378 188 KB

EDIT 3 : some more measures
I monitored CPU usage and Network locally and on the client, this is pretty interresting.

On client side it took only 550Kb/ on avergae for streaming 20 fps HD1080p + sound (I suppose there is a keyframe each 5s)

image
image1046×424 39.1 KB

On server side rtsptowebrtc took 5% cpu but nearly 2Mb of network (RX from the camera) and twice less on TX

526

Which repository did you use paid to get a version compatible with Yi Dome Guard

527

Hi there for any reader,

as I have made a lot of tests and to give an other answer at the OP subject here is some informations about HA configuration to get reliable WebRTC with very small delay.

This is the setup I use at the moment and it may change in the futur depending on the progress of the differents projects, mainly RTSPToWeb and go2rtc.

In short :
RTSPToWeb as a server/service (easiest way is Docker)
RTSPToWebRTC intégration in HA configured with the RTSPToWeb ip/port and a public STUN server if you dont have one or dont want care.
Cards in HA : picture-glance or picture-entity

I have heavily tested this configuration and it give me reliable and nearly “no-delay” video even from a WAN access (test from mobile device/computer), but there are some settings that should be done if you dont want the Video player to fall back to HLS

1 - configure RTSPToWeb (config.json) :
a - You must set a range of UDP port to use, It seems to not use WebRTC and fallback to HLS if you do not set this two configuration keys.
ex config.json :

....
    "webrtc_port_max": 41000,
    "webrtc_port_min": 40000
....

b - Forward this range of UDP port (40000-41000) from your internet router/WAN access to your HA/RTSPToWeb device IP (if you have more than one router between your device and WAN access you may have to do it on each router)

2 - configure RTSPToWebRTC intégration in HA :
a - set the IP:port of your RTSPToWeb server
b - configure it with a STUN server, there are some public one :
image

You dont need to configure RTSPToWeb (config.json) streams, once the integration is working they will be automatically created on the fly.

You can verify that it is working by using the browser debugger (F12) and look at the video element, you should see that its parent is a “ha-web-rtc-player” :
image

If you see “ha-web-hls-player”, this mean that you miss something in the configuration somewhere.

On a computer client you can also use Wireshark to verify the Network traffic, you should see a lot of UPD packets using the range of UDP port you have configured while watching a video cam :
image

As this configuration works so well that I would recommend to :

  • uncheck preload video (on HA camera entity)
  • set “on_demand” to true in config.json for RTSPToWeb server

Doing that make all resources are released after few seconds when no one watch the cameras (CPU == 0/Network == 0) and camera still start in 4/5 seconds and display video with very low latency (often < 1s)

The only downside I found with RTSPtoWeb is that for some camera (not all) the sound is delayed :confused:

Note : you can easily access to RTSPtoWeb web interface with a browser by using the same URL you have configured in the RTSPToWebRTC HA integration, there you can see all auto registered streams, test them, and set “on_demand” setting on or off.

Hope it will help !

528

And you write all of this in the thread not related to this add-on and this integration…

529

hoho, keep cool,

Well, it is related to WebRTC, to realtime video, to streaming, to video delay etc… and the topic is Realtime camera streaming without any delay - WebRTC

I think those informations can be usefull for someone searching for Realtime camera streaming without any delay - WebRTC this is how I find this topic first time…

No offense but you may need to be more precise in the title if you dont want to talk about WebRTC but only about your projects… as my last post is about WebRTC wtih HA without delay

EDIT: the misunderstanding come come from the fact that you have named your project with a “common name”… , it is like naming a project “Google” or “HTTP” or anything elses… my two cents try to avoid that, common name, brand, etc… HAWebRTCPlayer maybe ?

530

Just for your information. You wanted to help users, but you can confuse them even more with your post.

PS. I’ve been in contact with the author of RTSPtoWebRTC for a long time. And I was the first to make it an Hass addon. And made some improvements to its code.

PS2. Check history of first post. It was an RTSP2WebRTC addon more than 2 years ago.

531

Nice, and your projects are nice too, that absolutly dont make my post “unrelated to the topic” unless your change it, you began this post two years ago, thing have changed and as I said in my post, "my solution, maybe true today but it may change in the futur depending on projects progress (including yours)

PS: back in 2009 I was often exchanging with Markus, and also about a game prototype called Minecraft, long time before it become public, the thread is still open and you know what ? no one told about the original prototype anymore, time have passed

532

I’d recommend you move this to a different thread to avoid confusion.

533

This topic about custom integration. Hass built-in integration has another topic. You wanted to do better, but in my opinion, you only do worse by confusing users.

534

I’d recommend you move this to a different thread to avoid confusion.

Well, ok, I will stop posting in this thread about WebRTC unless it is related to Alexxiit/WebRTC, for the existing posts, mainly the last one where I give a full explanation that works well for me using WebRTC I ll will move it if I find how, or I will do a copy/past in a new thread…

EDIT: @AlexxIT you are right, I did not notice the section of the thread “share your projects”

535

I appreciate the spirit of being helpful. i’m contributing to the confusion as well myself :slight_smile: which is why i chimed in.