But I do see the stream zeroconfig occuring in my nest cam on the google hub without any configuration setup for that camera in go2rtc. It works (im using RTSPtoWebRTC with go2rtc). Its just really slow to come up.
Thanks for all the replies everyone. I did some investigating to see if the problem with my Nest audio was at the go2rtc level or with the RTSPtoWebRTC integration. Seems like itās go2rtc. To test it, I reinstalled the RTSPtoWeb addon by @allenporter, and paired it with the RTSPtoWebRTC integration, and all the audio works well, for the Foscams and the Nest cam. The main reason I donāt use the RTSPtoWeb addon is because I need remote access to my cameras, which go2rtc does beautifully.
So it seems the issue with Nest cam audio is with go2rtc. I tried viewing the camera using the addon web interface and audio was also not available. See below the info of the stream when viewed in my Android in case that helps. I also tested it from my desktop using chrome and same behavior.
[{"media:0":"audio, sendonly, 96 MPEG4-GENERIC/16000/2","media:1":"video, sendonly, 97 H264/90000","receive":16995,"remote_addr":"##.##.##.###:###","send":0,"track:0":"97 H264/90000, sinks=1","type":"RTSP client producer","url":"rtsps://stream-uc2-charlie.dropcam.com:443/sdm_live_stream/XXXXXXXXXXXXXXXX"},{"remote_addr":"udp4 host ##.##.##.###:51441","send":16841,"type":"WebRTC server consumer","user_agent":"Mozilla/5.0 (Linux; Android 13; Pixel 6 Build/TP1A.220624.021; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/105.0.5195.79 Mobile Safari/537.36 Home Assistant/2022.8.0-2624 (Android 13; Pixel 6)"}]
One thing to add is that this may become a mute point once @AlexxIT incorporates the addon as the back bone of the WebRTC custom component, at which point Iāll be able to use the custom component for my Foscams and leave the Nest Cam alone.
As of now, the WebRTC custom component is very unstable for me, and gives me a āERROR: NotSupportedError: Failed to execute āaddSourceBufferā on āMediaSourceā: The type provided (āvideo/mp4; codecs=āavc1.000000āā) is unsupported.ā error very often, regardless of whether I set up my Foscams using the camera entity or the RTSP url itself.
WebRTC technology doesnāt support AAC audio (MPEG4-GENERIC). You can hear sound only with HLS technology (default for Hass) or MSE technology (WebRTC Camera custom component).
go2rtc also support MSE, but audio doesnāt work well yet.
You can get audio with WebRTC technology with using on the fly transcoding feature.
Unfortunately though thereās really no way to reference the nest integration camera in the go2trc to do this though because the url changes with the authorization. So we canāt reference it by rtsp url, hass camera, etc.
Might there be a possibility for another way to reference cameras /streams like this by entity_id or something else?
Interesting, I wonder how the RTSPtoWeb addon is doing it. Because the sound for my Nest Cam does work when I use that with the RTSPtoWebRTC integration.
If you can hear AAC sound, this means only one - WebRTC donāt works and you viewing built-in HLS stream. If you stop the addon - nothing will change.
Interesting, just to make sure weāre all on the same page, let me summarize the behavior Iām seeing:
No integrations/addons: Nest cam (yes audio); Foscams (no audio)
RTSPtoWeb addon + RTSPtoWebRTC integration: Nest (yes audio); Foscams (yes audio); only works locally.
go2rtc addon + RTSPtoWebRTC integration: Nest (no audio); Foscams (yes audio); works remotely.
So my questions are:
Does the RTSPtoWeb addon use the built-in HLS stream of the Nest cams instead of WebRTC? If not, Iām not sure how it is that the audio works for both the Foscams and the Nest cam when Iām using it.
If the only fix for the Nest cam audio to work with go2rtc is to exclude it from the using WebRTC, any chance we could add a setting to either enable the on-the-fly setup of streams? This way, I could set up only my Foscams to work via go2rtc, and let the Nest cam use the built-in integration.
Thanks again to all of you for making these great addons happen, really appreciate it.
You can add all your foscam cameras to the go2rtc.yaml
You can then add each of those cameras as generic cameras in HA using the go2trc URL (rtsp://127.0.0.1:8554/-go2trc_entry-).
remove the RTSPtoWebRTC integration
you now have all your cameras that you want webrtc for managed and the ones you donāt want excluded.
You still do not have webrtc for the nest camera using the above. That will still use HLS because of issue #1 above. THereās no way in the go2rtc to reference that camera to transcode audio to something like opus/pcm.
IMO, Home Assistant should be giving us rtsp URLs for cameras like they do for its still images in the cameras properties.
I donāt know why RTSPtoWeb addon may donāt work for your Nest cameras.
For now you may use go2rtc for transcodings audio to AAC format for your Foscams. And donāt use WebRTC at all for all your cameras. So you get audio for all of them.
Iāll think how to add transcoding for zero-setup cameras.
Thanks for your help @calisro. I tried this, but I must be doing something wrong, because I still canāt get the audio to work with my Foscams in lovelace. Hereās what Iāve done. Iāve set up the streams for my 3 Foscams in the go2rtc.yaml file. The sound works well when I open the webrtc option via the go2rtc addon webui. I then created the 3 generic camera entries as you described, for example one of them is rtsp://127.0.0.1:8554/baby_cam_1. Iāve removed the RTSPtoWebRTC integration altogether.
Unfortunately, when I add the generic camera to my lovelace the audio doesnāt work. Should I be specifying the audio codec in the rtsp url?
Iāve tried the following:
rtsp://127.0.0.1:8554/baby_cam_1?video&audio=opus
rtsp://127.0.0.1:8554/baby_cam_1?video&audio=pcmu
rtsp://127.0.0.1:8554/baby_cam_1?video&audio=pcma
I just canāt seem to get the sound to work in lovelace with either google chrome or the home assistant Android companion app. Again, sound works when the Foscamās viewed using the go2rtc addon webui in both chrome and the companion app.
Thatās not how it works. When you get RTSP from go2rtc, you can filter tracks with codecs. As you do. But this tracks should be already in stream. So you need to config streams with ffmpeg source type and transcoding options.
That will pass the existing stream AND a new stream with JUST audio in opus codec to teh client. The client will pick from the 2 streams any codecs that match. That will give you video from the first entry and audio from the second on browsers/clients that donāt support the audio stream from the first one.
Then I added that cameras to generic with URL:
snapshot: existing snapshot URL from the reolink camera. This is used by the glance cards, for example, to see intermittant image changes
rtsp: rtsp://127.0.0.1:8554/backyard_main
EDIT: didnāt see Alexās examples. now you have two sets.
Alright guys, at this point I feel Iām just being plain dumb, so I deeply apologize but I still canāt get the audio to work. Hereās my go2rtc.yaml config:
I feel like somethingās going on with my ffmpeg setup up, just not sure what it is. Iām running hass os in an RPI 4 btw, and have go2rtc installed as an addon.
now you have two sets.