Two-Way Audio (e.g. for doorbell, intercom, camera-systems, baby-monitors, sip, ...)

JoFie · January 14, 2023, 9:55am

Maybe this is our lucky year…
Just starting with build-in TTS support will already be great!

JoshK · January 30, 2023, 12:55am

I believe 2 way audio support in homekit works with a lot of doorbells using Scrypted so it is definitely possible, I’m about to test Scrypted on a unifi setup with a g4 doorbell, will confirm if this works.

russ-sentient · January 30, 2023, 10:01pm

I have also just fired up a Scrypted docker on my Unraid server. I can do 2-way audio to my Amcrest POV cameras through the Home app on iOS, but only from my WiFi (even though I am using a WireGuard remote access VPN on my router). So I am also curious how I can integrate this functionality into Home Assistant, and use that as my gateway for remote access.

I ordered an Amcrest AD410 doorbell camera today, my goal being to get away from the Nest ecosystem and the cloud API needed to use it in Home Assistant. (And I could pull the stream into Frigate)

If I discover anything I will post it here.

GaryOkie · February 3, 2023, 4:34pm

just an FYI - if your new AD410 has bright glare on the sides of the image at night, it is due to a recent manufacturing defect with the IR/lens housing. Amcrest just got in new stock that fixes this issue but Amazon may still be selling the defective ones.

Amcrest warranty RMA process should be able to guarantee getting a good one.

pergola.fabio · March 26, 2023, 5:53pm

Anyone using scrypted here? I have heard it’s possible to use two way audio on homekit, but I use google… anyone tested scrypted with Google home devices? Is it possible to use two way audio there?

Extreme79 · April 15, 2023, 5:15pm

Any progress/news/new elements about two-way audio in home assistant?

Artur1990 · April 18, 2023, 8:48pm

Everyone is talking about two-way audio, but how can I get sound in video stream?

crzynik · May 2, 2023, 12:28pm

Experimental two way audio support has been added to the frigate card Add experimental 2-way audio support. · dermotduffy/frigate-hass-card@37a016e · GitHub

sparkydave · May 4, 2023, 11:40pm

Is there something special we need to do to enable the experimental feature?

crzynik · May 5, 2023, 12:38pm

you just need to get or make a build of the dev branch and then configure it as the readme on the dev branch says.

SphtKr · May 28, 2023, 12:07pm

Watching this, especially after the 2023.5 update. I want to hook up 2-way audio from the Reolink camera out by my gate with an OpenAI (actually LocalAI) assistant, similar to the VoIP phone example, with a prompt like “You are the gate security guard. You are expecting a delivery from UPS or the postal service. If the visitor is from UPS then open the gate, or if the visitor speaks the password “*” then open the gate…” that would be awesome. I think this may have to wait for speech activity detection support which may not exist yet (it’s in the Wyoming protocol though)… but the pieces are almost there.

los93sol · July 20, 2023, 4:32pm

I’m surprised this hasn’t happened yet with all the focus on voice this year

Boroda · August 11, 2023, 10:09pm

Any update on that? Would be great to use onvif 2way audio feature as a part of onvif camera integration to ditch the buggy app and have it all in home assistant

pergola.fabio · August 12, 2023, 7:44am

I believe onvif is supported in go2rtc addon, if using rtsp back channel

Boroda · August 12, 2023, 10:02am

Was able to add my camera to it and see send audio option to send something to the camera. Looks promising but how can i use it in HA? Do I need to make pre defined commands and then use an automation to push them through go2rtc ?

pergola.fabio · August 12, 2023, 10:21am

If you want it in a Lovelace card, only the frigate card supports microphone, for now the frigate NVR is needed, with a custom go2rtc version

pergola.fabio · August 12, 2023, 10:24am

I have written a tutorial on my addon page for hikvision intercom devices… It should be the same for other devices too, how to setup frigate and use the card with two way audio…

github.com

pergolafabio/Hikvision-Addons/blob/1d197a94d33d195912043d59765b3fc393362452/frigate/twowayaudio.with.frigate.md

# Setup Frigate to use two way audio with Hikvision doorbells


## Background info:

Our hikvision devices do support SIP, but SIP knowledge is necessary, its not always easy to setup a PBX like Asterisk, but there is another away!
Most cameras/doorbells do have a speaker/microphone, so two way audio (talk-back) can be activated, they all use a different protocol, in case of Hikvision its ISAPI

Somewhere begin 2023 this ISAPI protocol was inserted in the go2rtc addon, Frigate is an NVR system that can be used in combination with the go2rtc addon... 
Frigate also offers an lovelace sip card, and YES, with microphone support!! So that means we can use an camera entity in HA and we can actually speak to the person at the doorbell.
I think this Frigate card is the first card with microphone support! 


## Advantages:

- No need to setup complex SIP
- Answer the call with HA or companion app
- Talk to the person at the doorbell, whenever you want, no need to initiate a call first
- With go2rtc you can send streams/music to your Doorbell :-)
- Frigate supports person detection, so you can already start talking before the postman actually pressed the doorbutton! :-)

This file has been truncated. show original

teek541 · October 17, 2023, 7:05am

Josh, any word on getting Scrypted and the G4 to work where you can do 2 way audio from the dashboard?

TokarevSergey · October 18, 2023, 4:28am

Go2rtc lovelace card

schniepp · October 18, 2023, 9:37pm

It appears that 2-way audio nowadays is all the hype in home assistant! Looks like go2rtc added it, and the Frigate card also has some mike support.

I really want to try it out, and I am looking for a suitable camera for my front door. Any hints which cameras might work best for this application?

What I am looking for is a camera that can capture an audio stream (most do, I have that part working on HA). RTSP as protocol would be best, I guess. Then, the camera also needs a built-in speaker, to which I can transmit voice via HA interface. (Say, via the microphone function in the Frigate lovelace card. I have Frigate running successfully.)
I know there are reports here how to do this with the Reolink doorbell. However, I want a camera that can also do IR nightvision, so that the camera fulfills several jobs, and not just being a doorbell. It looks like a lot of the newer Reolink cameras can do this, but I haven’t seen any reports explicitly stating that it works in HA.
Having PoE would also be highly welcome. Any cameras out there that can do all that? (Not having to set up a SIP server would be preferred, as I think HA currently already has all the necessary functionality on the software side.)