Go2rtc: send audio to cam?

prankousky · July 3, 2023, 8:21am

Hi everybody,

I have a Reolink Duo PoE running firmware v3.0.0.1388_22100600. It is integrated via Reolink integration. I stumbled upon a blog entry somewhere (unfortunately, cannot find the link atm) mentioning you can send audio and/or TTS to 2-way cameras utilizing go2rtc.

So I tried, but am stuck.

Here is what I did

on Camera

create user rtc, password 12345 (not really, just to have a value to work with here)

Home Assistant

Add-Ons > Repositories > add https://github.com/AlexxIT/hassio-addons
Install go2rtc Add-On
Start go2rtc Add-On

At first, this didn’t work. It kept telling me that port 1984 was already in use; I couldn’t really troubleshoot this, but I had also installed and configured the webrtc integration, which I then deactivated and restarted Home Assistant.

Now I could successfully run go2rtc, the port had been free to use.

go2rtc

open go2rtc from lovelace sidebar
click config; contents:

streams:
  einfahrt: onvif://rtc:[email protected]:8000
  rtc_einfahrt: rtsp://rtc:[email protected]:554/h264Preview_01_main

Save & Restart

The einfahrt entry (onvif) did not work. It would point at http://192.168.100.3:8000/onvif, which would only show this

<SOAP-ENV:Envelope>
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>SOAP-ENV:Client</faultcode>
<faultstring>HTTP GET method not implemented</faultstring>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

I did not find any setting on the Reolink camera to implement HTTP GET method; when I’d try to open the stream in go2rtc, I would get an error message pointing to that link from above. So I removed that camera.

So we’re working with rtc_einfahrt only now. The webUI looks like this

grafik

Here are the contents:

`0 / info`

{
  "producers": [
    {
      "url": "rtsp://rtc:[email protected]:554/h264Preview_01_main"
    }
  ],
  "consumers": []
}

`1 / info`

sometimes, I believe after actually loading the camera stream through this UI, 0 / info would change to 1 / info and show this content

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.100.3/h264Preview_01_main/",
      "remote_addr": "192.168.100.3:554",
      "user_agent": "go2rtc/1.5.0",
      "medias": [
        "video, recvonly, H.264 High 5.1",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        "96 H264, bytes=1022948, senders=1",
        "97 MPEG4-GENERIC/16000, bytes=16512, senders=0"
      ],
      "recv": 1048688
    }
  ],
  "consumers": [
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "udp4 prflx 10.0.0.25:59602 related :0",
      "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, TELEPHONE-EVENT/8000, L16"
      ],
      "senders": [
        "126 H264, bytes=1022948, receivers=1"
      ],
      "send": 1033721
    }
  ]
}

`stream`

Would actually show the camera stream in the browser, including camera audio.

`links`

A bunch of different links to streams provided by this Add-On, as well as this part

So I entered ffmpeg:http://<hass>:8123/local/audio/tom-green_ding-dong.mp3#audio=pcma#input=file. This file exists, and is available to the camera.

The camera does not play this audio after pressing send.

I also added this to my configuration

media_player:
  - platform: webrtc
    name: RTC/Einfahrt
    stream: rtc_einfahrt
    audio: pcmu/8000

Which creates this entity

(translates to something like `error calling service media_player/play_media.‘webrtc’)

Does anybody know what I need to do in order to get this working? Thanks for your ideas

AlexxIT · July 4, 2023, 11:00am

Reolink cameras has awful RTSP realisation. Also without RTSP backchannel.
Only Reolink Doorbell has buggy support of RTSP backchannel.

prankousky · July 4, 2023, 12:27pm

Thank you. You have created this Add-On, right? Which camera or doorbell would you recommend to buy for best compatibility?

Other than the awful RTSP (read this on the github as well), I am quite content with my Duo PoE. But if there was some not-too-expensive camera or doorbell that I could additionally use, I might do so.

gurkburk76 · July 11, 2023, 11:46am

I’ve been looking for a say to play sounds or via tts on my reolink doorbell as well so happy to have found something about it
I’m currently using frigate and home assistant and would like to make my doorbell “speak” somehow. @prankousky there is a newer firmware on reolinks site, see if that somewhat helps out.
I’ll try to make it work but without adding web2rtc addon in ha, frigat uses it so there should be a way to use that without introducing more complexability, i’de like to keep things simple when possible @.@

gurkburk76 · July 11, 2023, 12:30pm

Small progress, from go2rtc 1.6 (just released) from within frigate i tried to run a small sample and that seemed to work just fine, not sure if it actaully plays the whole thing or cuts off randomly but it’s something
ffmpeg:https://file-examples.com/storage/fe8ec1a8f464ac695986bbb/2017/11/file_example_MP3_700KB.mp3#audio=pcma#input=file

Now let’s see if i can make it work from ha without setting up a go2rtc thing there, seems like i should’t have to.

gurkburk76 · July 11, 2023, 1:30pm

I sort of added the stream from frigate to the go2rtc addon in ha and it sort of works. Now to another problem. i tried to create a media_player thingy like you in the example above but i seem to be missing the webrtc platform for some reason, perhaps it renamed ?
This is the error: Platform error media_player.webrtc - Integration ‘webrtc’ not found.
I also tried to add RTSPtoWebRTC addon but that did nothing.

EDIT:
Argh… so i had to install the webrtc via hacs even though i’m unlikely to use it in the gui, oh well…

So yeah about the thing that it outputs sound by going thru go2rtc like my example above, it seems to be very hit and miss when it feels like working. A better solution is imo doing it from ha but then i run into those same problems with media_player not working.
Hopefully someone with more skills will fix it

prankousky · July 15, 2023, 5:01am

Thank you for all your input. You are using the Doorbell though, correct?

According to AlexxIT, only that device works - I am using the Reolink Duo PoE. Have you by any chance tested other devices than the doorbell one?

gurkburk76 · July 15, 2023, 8:20am

Not sure i understand the question fully, if i’m using the doorbell? Yes, it’s on my wall fully functional
It’s this one to be precise: Reolink Video Doorbell PoE | Smart Wired Doorbell with Chime
I have not tried any other devices.

prankousky · July 15, 2023, 10:46am

Sorry, I just meant you have the doorbell… which is, according to the developer, the only Reolink product that supports sending audio to it. That means that your workflow likely won’t work for me. I didn’t see any relevant firmware updates for my devices (I don’t have the doorbell one).

gurkburk76 · July 15, 2023, 6:29pm

Ok, i thought you also have one that you tested with, my bad

Twig_Bark · October 12, 2023, 4:16pm

But when using https stream then is it possible senda audio to cam?

calisro · October 20, 2023, 5:36pm

Depends which camera specifically? Works on the doorbell.

Twig_Bark · October 20, 2023, 6:02pm

Reolink 823A

calisro · October 23, 2023, 2:07pm

This is how i’ve done it on the Reolink Doorbell which supports 2-way audio. Not sure about the 823A but I can see it has a speaker and I believe it is profile T. Results will depend on the camera actually supporting 2-way. I believe the only one that does currently is the door bell but not 100% sure.

go2rtc

frontdoor_main:
    - rtsp://admin:[email protected]/h264Preview_01_main
    - ffmpeg:frontdoor_main#audio=opus

.

shell script with post:


$FILE=/some/file

curl -v \
  -X POST \
  "http://localhost:1984/api/streams?dst=frontdoor_main&src=ffmpeg:$FILE%23audio=pcma%23input=file"

Twig_Bark · October 24, 2023, 5:26am

Thanks.
How to execute this shell script?

calisro · October 24, 2023, 10:15am

It’s just a post command. You can run it directly from ha with the ha shell command service or on any os directly. Really anywhere. Depends on what you’re trying to accomplish. I feed a tts mp3 into it from Google cloud service so it speaks.

Twig_Bark · October 25, 2023, 6:28pm

My goal is too send tts to camera. But can you give a hint or tutorial how to get this post command work.
I don’t have any idea where to start.
I looked some youtube videos but wont found anything about how to post shell command in HA.
Is there any tutorial or something?

calisro · October 27, 2023, 6:02pm

I’m not sure how you intend on initiating the tts to the bell but you could do this. You’ll need long lived token for the api.

a shell script in /config directory called ttsplay

This shell script will take an input of some text and pass it to the home assistant google_cloud integration to generate a mp3 file speaking that text. It stores in in HA and can be played with ‘URL’.

Then it takes that URL and with some ffmpeg/go2rtc magic it pads the beginning and end a few seconds and plays that mp3 to GO2RTC camera.

#!/bin/bash   
TEXT="$1"
LOG=/config/ttslog
echo "TEXT: $TEXT" > $LOG
TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxx
GO2RTC=frontdoor_camera

URL=`curl -v -X POST \
 -H "Authorization: Bearer $TOKEN" \
 -H "Content-Type: application/json" \
 -d "{\"message\": \"$TEXT\", \"platform\": \"google_cloud\"}" \
 https://ha.someurl.org/api/tts_get_url | jq -r .url` >> $LOG 2>&1
echo CURLRC=$? >> $LOG

curl -v \                                                                                                                                                                                                                                                                                                   
  -X POST \                                                                                                                                                                                                                                                                                                 
  "http://localhost:1984/api/streams?dst=$GO2RTC&src=exec:ffmpeg+-hide_banner+-v+error+-re+-i+$URL+-af+\"adelay=1s,apad=pad_dur=5\"+-c:a+pcm_alaw+-ar:a+8000+-ac:a+1+-vn+-user_agent+ffmpeg/go2rtc+-rtsp_transport+tcp+-f+rtsp+%7Boutput%7D"  >> $LOG 2>&1                                           
echo DOORRC=$? >> $LOG

exit 0

You can test it but dropping into the homeassistant container and executing it:

# docker exec -ti homeassistant /config/ttsplay "Hello there.  Please leave the package by the door"

If/when it works, you can then execute this via home assistant by adding the shell_command to configuration.yaml and restarting HA. It will take the input of a input_text when it is called. See my automation after.

shell_command:
  doorbelltts: >-
    /config/ttsplay "{{ states('input_text.tts_door_text') }}"

and create a helper called ‘input_text.tts_door_text’ to hold whatever text you wanted to send.

and then create a simple automation so when you add text to it, it speaks to the doorbell and clears the input for the next time… You could also use a input_select and store canned responses and push those into the text field.

alias: Door Bell TTS
description: ""
trigger:
  - platform: state
    entity_id: input_text.tts_door_text
    not_to:
      - ""
      - unknown
    id: textset
    for: "00:00:02"
condition: []
action:
  - service: shell_command.doorbelltts
    data: {}
  - service: input_text.set_value
    data:
      value: ""
    target:
      entity_id: input_text.tts_door_text

mode: queued

An expanded example automation using a input_text and some locks and stuff.

alias: Door Bell TTS
description: ""
trigger:
  - platform: state
    entity_id: input_select.tts_door_text_selection
    not_to: Select
    id: selection
  - platform: state
    entity_id: input_text.tts_door_text
    not_to:
      - ""
      - unknown
    id: textset
    for: "00:00:02"
condition: []
action:
  - if:
      - condition: trigger
        id: selection
    then:
      - choose:
          - conditions:
              - condition: state
                state: Leave Package
                entity_id: input_select.tts_door_text_selection
            sequence:
              - service: input_text.set_value
                data:
                  value: Hi. Please leave the package at the door.  Have a great day.
                target:
                  entity_id: input_text.tts_door_text
          - conditions:
              - condition: state
                state: Visitor Unlock
                entity_id: input_select.tts_door_text_selection
            sequence:
              - service: input_text.set_value
                data:
                  value: Hi. I unlocked the door. Please come in.
                target:
                  entity_id: input_text.tts_door_text
              - service: lock.unlock
                data: {}
                target:
                  entity_id: lock.front_door
          - conditions:
              - condition: state
                state: Solicitor
                entity_id: input_select.tts_door_text_selection
            sequence:
              - service: input_text.set_value
                data:
                  value: >-
                    Hi. Im sorry but we arent able to come to the door and we
                    are not interested.  But thanks.
                target:
                  entity_id: input_text.tts_door_text
              - service: lock.lock
                data: {}
                target:
                  entity_id: lock.front_door
        default:
          - service: input_text.set_value
            data:
              value: "{{ states('input_select.tts_door_text_selection') }}"
            target:
              entity_id: input_text.tts_door_text
  - if:
      - condition: trigger
        id: textset
    then:
      - service: shell_command.doorbelltts
        data: {}
      - service: input_text.set_value
        data:
          value: ""
        target:
          entity_id: input_text.tts_door_text
  - service: input_select.select_option
    target:
      entity_id: input_select.tts_door_text_selection
    data:
      option: Select
mode: queued

calisro · October 27, 2023, 7:53pm

This works too. Much easier.

configuration.yaml

media_player:
  - platform: webrtc
    name: Frontdoor
    stream: frontdoor_send
    audio: '-af "volume=10dB,adelay=2s,apad=pad_dur=6" -c:a pcm_alaw -ar:a 8000 -ac:a 1'

service call:

service: tts.google_cloud_say
data:
  entity_id: media_player.frontdoor
  message: One two buckle your shoe.  three four close the door.

service: media_player.play_media
data:
  media_content_type: music
  media_content_id: http://10.100.1.183:8123/local/halloween-doorbell-ringtone.mp3
target:
  entity_id: media_player.frontdoor

funbobby · December 6, 2023, 6:58pm

In your last example media_player configuration, what is the frontdoor_send stream?