Realtime camera streaming without any delay - WebRTC

[vingerha]

Hi all, this post has grown soo huge and over such a long time that I am not sure anymore which info sticks with the most recent options.
My story…

I had issues connecitng multiple clients to my camera
So I have implemented go2rtc on my SynNas which ‘collects’ 2 streams form the camera, a Tapo C310
The clients of go2rtc are

• Syno Surveillance Station on the same machine
• HA on a NUC, same network segment
• Frigate on the same NUC

I’d like to focus on HA for now.
I setup a generic camera in HA that connects to go2rtc, the corresponding entity I then use in my dashboard via Picture Glance card in live streaming.
I however still see about 2s delay between camera (me waving my hand) and the stream in HA. That delay does not change if I stop Frigate.
As this post’s header stated “without any delay”…any idea what I am doing wrong or where to look?

[AlexxIT (Alexey)]

You setup generic camera and using built-in Hass capabilities. So nothing has changed for you about delay.

[vingerha]

Sorry to sound a bit thick…what is diferent with your feedback to what I implemented? A generic camera utility in a generic card?
EDIT: I the delay coming from HA or from go2rtc…or the camera-feed to go2rtc or how to verify this?

[AlexxIT (Alexey)]

The delay is almost always after Hass. Because of HLS technology and built-in FFmpeg.
You’ve added go2rtc before Hass. To a place where there was no delay at all.
You need to stop using Hass built-in streaming capabilities. And the delay will be gone.

[vingerha]

@AlexxIT

stop using Hass built-in streaming capabilities

Euh…and how? just bypass HA for the use I have?

[fajrstartr]

I like to know that too. I’m using reolink outdoor and i get 2 sec delay at best. Over time it accumulate to 15 sec delay. (Using webRTC integration)
This is my dashboard entry:

type: custom:webrtc-camera
url: rtsp://admin:*********@192.168.50.253:554/h264Preview_01_sub

[vingerha]

Posisbly this is what you meant, i.e. to use the webrtc card? This is indeed a LOT closer to realtime than the camera entity + picture glance.??
Any other suggestions welcome

[vingerha]

EDIT: just found your git ‘issue’ on it… no solution for wallpanel so it seems

Another thing on HA…webrtc may be more accurate on the stream, but sadly it takes way more time to ‘start’ on my tablet with ‘wallpanel’.
The option “background: true” works fine on my laptop but has no noticeable effect on the tablet…any ideas?

[AlexxIT (Alexey)]

Probably you have AAC audio, so MSE technology selected. Also you have high bitrate and bad connection from Hass to browser, so you can’t receive MSE data in real time.

[NeoID]

I have the same tablet and while webrtc is working with all other devices, it won’t work with the Samsung Tab A7. Did you ever find a solution?

Strangely it seems to be able to play the sub stream, but not the main stream of my Reolink E1 Outdoor camera. However, it’s unable to play any of the recodings of the main stream either which is really annoying. Haven’t yet been able to convert the live stream or the recording to a supported format.

[vingerha]

I had similar challenges and believe this to be related to other request(s) to the same rtsp address (other machne/app). I installed go2rtc as a single streaming recipient and connect all my cards and other machines with go2rtc, this works fine for me

[Se7enair]

I use the jsmpeg stream for my live preview. It is enough in my case.
When motion is detected it opens a substream in a separate window. Mainstream lags to much.

With recordings I learned to live without it.

So not a real solution, but fine for me. So never dipped deeper.

[NeoID]

I also used jsmpeg for now. What do you mean by the substream opening in a separate window. Is that using jsmpeg too? It’s baffling that there are not more mentions about this being an issue on the web. I mean the tablet is from 2020, it’s insane webrtc isn’t better supported. I mean it works in Firefox so must be Chrome just blocking it. Some people mention modding webview with libwebrtc.aar, but nobody is going into details.

I’m really disappointed by this piece of junk. I have two of them and both of them had their batteries expand twice too. Seems the whole thing was build out of the cheapest components available. The last posts around support for it in Chrome are all from 2018, so I guess there’s no hope then

[Se7enair]

I have a dashboard on a wall mounted tablet. If there is motion, eg. some is at the door, there is a popup opened on the dashboard. It shows the substream with jsmpeg.
This was the only way to get short loading times and a “fluent” stream. I learned that most of live streams are substreams. Main stream is only used for recording, not only in my setup but also on other setups.

  action:
    - service: browser_mod.popup
      data:
        size: wide
        style:
          .: |
            :host .content {
              width: 1150px;
            }
        content:
          type: custom:frigate-card
          cameras:
            - go2rtc:
                stream: gartentuere_sub
                modes:
                  - mse
              live_provider: jsmpeg
              frigate: {}
              camera_entity: camera.gartentuere
          performance:
            profile: low
          menu:
            style: none

[fajrstartr]

I have reolink csmera. I would try jsmpeg for live preview to, but i don’t know how to set it up.
Can you post your lovelace card code?

[uspino]

I use an iPad with a full screen browser. I have a button to start a camera full screen with WebRTC, but I can’t find a way to navigate back to the main page. I can add overlay buttons (shortcuts) to open the door, lights, etc., is there a way to add a button to navigate to a url (Home Assistant main page)?

[tmjpugh]

Creation of Button URL

Hello, i am looking for a solution to create a booton linked with an URL adress. Can you give me the good ways please, In lovelaca it’s shouldn’t possible, then i try to creat it in configuration yaml. please read my code: type: button tap_action: action: url url_path: 'http://192.168.XX.XX/config/motion.cgi?enable=yes' entity: motion.camera_salon_on I have this error : Setup failed for tap_action: Integration not found. Setup failed for type: Integration not found. thanks for you…

[Se7enair]

      - type: custom:frigate-card
        view_layout:
            grid-area: camera
        cameras:
          - camera_entity: camera.hof
            live_provider: jsmpeg
            icon: mdi:garage
            go2rtc:
              stream: hof_sub
          - camera_entity: camera.gartentuere
            live_provider: jsmpeg
            triggers:
              entities: []
              motion: true
            go2rtc:
              stream: gartentuere_sub
            icon: mdi:door
        view:
          default: live
          update_seconds: 300
          timeout_seconds: 300
          update_force: false
          scan:
            enabled: true
            untrigger_seconds: 240
        menu:
          style: none
        live:
          preload: true
          draggable: true
          actions:
            tap_action:
              action: fire-dom-event
              browser_mod:
                service: browser_mod.popup
                data:
                  card_mod:
                    style:
                      ha-dialog$: |
                        div.mdc-dialog__scrim {
                          backdrop-filter: blur(15px) !important;
                          -webkit-backdrop-filter: blur(50px) !important;
                          background-color: rgba(0,0,0,0.2) !important;
                        }
                  style: >
                    --popup-max-width: 1150px;
                  content:
                    type: custom:frigate-card
                    cameras:
                      - camera_entity: camera.hof
                        live_provider: go2rtc
                        icon: mdi:garage
                        go2rtc:
                          stream: hof
                          modes:
                            - webrtc
                      - camera_entity: camera.gartentuere
                        live_provider: go2rtc
                        triggers:
                          entities: []
                          motion: true
                        go2rtc:
                          stream: gartentuere
                          modes:
                            - webrtc
                        icon: mdi:door
                    view:
                      default: live
                      update_seconds: 300
                      timeout_seconds: 300
                      update_force: false
                      scan:
                        enabled: false
                        untrigger_seconds: 240
                        show_trigger_status: false
                        untrigger_reset: false
                    menu:
                      style: overlay
                      buttons:
                        frigate:
                          enabled: true
                        cameras:
                          enabled: true
                        timeline:
                          enabled: true
                        expand:
                          enabled: false
                        fullscreen:
                          enabled: false
                        live:
                          enabled: false
                        snapshots:
                          enabled: false
                    live:
                      preload: true
                      draggable: true
                      actions:
                        tap_action:
                          action: fire-dom-event
                          browser_mod:
                            service: browser_mod.popup
                            data:
                              content:
                                type: custom:frigate-card
                                cameras:
                                  - camera_entity: camera.gartentuere
                      controls:
                        next_previous:
                          style: none
                      auto_play: all
                    image:
                      mode: camera
                    performance:
                      profile: low
                    media_viewer:
                      auto_play: all
                      draggable: true
                      layout: {}
                      controls:
                        timeline:
                          mode: below
                          style: stack
                          media: clips
                        title: {}
                        thumbnails: {}
                      transition_effect: slide
                    timeline:
                      style: ribbon
                      media: clips
                      show_recordings: true
                      controls:
                        thumbnails:
                          mode: below
                          show_details: false
                    media_gallery:
                      controls:
                        thumbnails:
                          size: 175
                          show_details: false
                          show_timeline_control: false
                    view_layout:
                      grid-area: camera

[smile]

All cameras show in HomeAssistant addon RTSPtoWeb - WebRTC (ver: 1.4.0), but not in Homeassistant. No devices or entities shows in Home Assistant integration. I have deleted and reinstalled both addon and integration and restart HomeAssistant, No success.

Home Assistant 2023.8.1
Supervisor 2023.08.1
Operating System 10.4
Frontend 20230802.0 - latest

2023-08-10 15:07:52.692 DEBUG (MainThread) [rtsp_to_webrtc.web_client] Not updating stream url since already set
2023-08-10 15:07:52.692 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[post] http://homeassistant.local:8083/stream/camera.garage/channel/0/webrtc
2023-08-10 15:07:52.697 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/streams) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 10 Aug 2023 13:07:52 GMT', 'Content-Length': '1559')>

2023-08-10 15:07:52.698 DEBUG (MainThread) [rtsp_to_webrtc.web_client] Not updating stream url since already set
2023-08-10 15:07:52.698 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[post] http://homeassistant.local:8083/stream/camera.zoom/channel/0/webrtc
2023-08-10 15:07:52.708 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/streams) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 10 Aug 2023 13:07:52 GMT', 'Content-Length': '1559')>

2023-08-10 15:07:52.708 DEBUG (MainThread) [rtsp_to_webrtc.web_client] Not updating stream url since already set
2023-08-10 15:07:52.709 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[post] http://homeassistant.local:8083/stream/camera.porch_door/channel/0/webrtc
2023-08-10 15:07:52.802 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/stream/camera.garage/channel/0/webrtc) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Date': 'Thu, 10 Aug 2023 13:07:52 GMT', 'Content-Type': 'text/plain; charset=utf-8', 'Transfer-Encoding': 'chunked')>

2023-08-10 15:07:52.838 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/stream/camera.zoom/channel/0/webrtc) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Date': 'Thu, 10 Aug 2023 13:07:52 GMT', 'Content-Type': 'text/plain; charset=utf-8', 'Transfer-Encoding': 'chunked')>

2023-08-10 15:07:52.849 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/stream/camera.porch_door/channel/0/webrtc) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Date': 'Thu, 10 Aug 2023 13:07:52 GMT', 'Content-Type': 'text/plain; charset=utf-8', 'Transfer-Encoding': 'chunked')>

2023-08-10 15:07:55.284 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[get] http://homeassistant.local:8083/streams
2023-08-10 15:07:55.304 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [139938016557760] Error handling message: Camera not found (unknown_error) sony from 192.168.1.1 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36)
2023-08-10 15:07:55.323 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[get] http://homeassistant.local:8083/streams
2023-08-10 15:07:55.345 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[get] http://homeassistant.local:8083/streams
2023-08-10 15:07:55.346 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/streams) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 10 Aug 2023 13:07:55 GMT', 'Content-Length': '1559')>

2023-08-10 15:07:55.347 DEBUG (MainThread) [rtsp_to_webrtc.web_client] Not updating stream url since already set
2023-08-10 15:07:55.347 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[post] http://homeassistant.local:8083/stream/camera.garage/channel/0/webrtc
2023-08-10 15:07:55.370 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/streams) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 10 Aug 2023 13:07:55 GMT', 'Content-Length': '1559')>

2023-08-10 15:07:55.370 DEBUG (MainThread) [rtsp_to_webrtc.web_client] Not updating stream url since already set
2023-08-10 15:07:55.371 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[post] http://homeassistant.local:8083/stream/camera.zoom/channel/0/webrtc
2023-08-10 15:07:55.403 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/streams) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Thu, 10 Aug 2023 13:07:55 GMT', 'Content-Length': '1559')>

2023-08-10 15:07:55.403 DEBUG (MainThread) [rtsp_to_webrtc.web_client] Not updating stream url since already set
2023-08-10 15:07:55.404 DEBUG (MainThread) [rtsp_to_webrtc.web_client] request[post] http://homeassistant.local:8083/stream/camera.porch_door/channel/0/webrtc
2023-08-10 15:07:55.587 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/stream/camera.zoom/channel/0/webrtc) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Date': 'Thu, 10 Aug 2023 13:07:55 GMT', 'Content-Type': 'text/plain; charset=utf-8', 'Transfer-Encoding': 'chunked')>

2023-08-10 15:07:55.598 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/stream/camera.garage/channel/0/webrtc) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Date': 'Thu, 10 Aug 2023 13:07:55 GMT', 'Content-Type': 'text/plain; charset=utf-8', 'Transfer-Encoding': 'chunked')>

2023-08-10 15:07:55.647 DEBUG (MainThread) [rtsp_to_webrtc.web_client] response <ClientResponse(http://homeassistant.local:8083/stream/camera.porch_door/channel/0/webrtc) [200 OK]>
<CIMultiDictProxy('Access-Control-Allow-Credentials': 'true', 'Access-Control-Allow-Headers': 'Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization, accept, origin, Cache-Control, X-Requested-With', 'Access-Control-Allow-Methods': 'POST, OPTIONS, GET, PUT, DELETE', 'Access-Control-Allow-Origin': '*', 'Date': 'Thu, 10 Aug 2023 13:07:55 GMT', 'Content-Type': 'text/plain; charset=utf-8', 'Transfer-Encoding': 'chunked')>

[mhoogenbosch (Martijn Hoogenbosch)]

You shouldn’t be expecting cameras there.