Tts.cloud_say and amazon Alexa

[solomos (Gerasimos Solomos)]

hello,
I was wondering if is possible to use tts.cloud_say with Alexa echos as the media player?

ofcourse I have already get the answer, but I was wondering if anyone has find a way to use alexa devices as media players?

2021-02-10 11:52:34 WARNING (MainThread) [custom_components.alexa_media.media_player] Sorry, text to speech can only be called with the notify.alexa_media service. Please see the alexa_media wiki for details.https://github.com/custom-components/alexa_media_player/wiki/Configuration%3A-Notification-Component#use-the-notifyalexa_media-service

[abogdal (Adam)]

This topic is quite old but still valid.

The short answer is YES, you can use Nabu Casa TTS service with your echos.

My idea was to use the tts_get_url REST endpoint to get a direct URL to the generated TTS file and then ask Alexa to play it as an mp3 file.

1. Get the URL

Request POST /api/tts_get_url

{
    "platform": "cloud",
    "message": "To jest mój test",
    "language": "pl-PL",
    "options": {
        "gender": "female"
    }
}

2. Play the TTS file as an mp3 on Alexa

service: notify.alexa_media
data:
  message: "<audio src='https://....nabu.casa/api/tts_proxy/..._cloud.mp3'/>"
  target: media_player.<your_echo_name>
  data:
    type: tts

Since mostly I use Node-Red for my automations I have used it for this case as well.

Import the following nodes and try them on your end:

[{"id":"4d325b8a56405a8c","type":"ha-api","z":"2a0a9d47cb5cc092","name":"Get TTS URL","server":"8147d85a.d4ab48","version":1,"debugenabled":false,"protocol":"http","method":"post","path":"/api/tts_get_url","data":"{\"platform\":\"cloud\",\"message\":\"{{payload}}\",\"language\":\"pl-PL\",\"options\":{\"gender\":\"female\"}}","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":370,"y":260,"wires":[["e3f537014143b116"]]},{"id":"e3f537014143b116","type":"api-call-service","z":"2a0a9d47cb5cc092","name":"Play TTS","server":"8147d85a.d4ab48","version":3,"debugenabled":false,"service_domain":"notify","service":"alexa_media","entityId":"","data":"{\"target\":[\"media_player.echo_dot\"],\"data\":{\"type\":\"tts\"},\"message\":\"<audio src='{{payload.url}}'/>\"}","dataType":"json","mergecontext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":560,"y":260,"wires":[[]]},{"id":"58a143b6c9ca034d","type":"inject","z":"2a0a9d47cb5cc092","name":"Message","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"To jest mój test","payloadType":"str","x":180,"y":260,"wires":[["4d325b8a56405a8c"]]}]

The above solution works like a charm with my echo dot 4 but it should work with other echos as well.

[butako123]

This works for me too!
The important thing to note is to use your Nabu Casa Remote UI URL in the one sent to Amazon, not your local installation one.

[solomos (Gerasimos Solomos)]

so far, I never tried your solution as I was not familiar with Node-Red.
After some nights without sleep now I know some things about it and I can implement quiet a few things.

I imported the code to Node-Red, is been executed OK as you can see

But alexa never executes it! Instead it says “Sorry. I am having trouble to acess Simon skill right now.”

Any ideas please?

[solomos (Gerasimos Solomos)]

@butako123

Nabu Casa Remote UI URL

pls give me an example. Used the following code at Node-Red

[{"id":"4d325b8a56405a8c","type":"ha-api","z":"2a0a9d47cb5cc092","name":"Get TTS URL","server":"8147d85a.d4ab48","version":1,"debugenabled":false,"protocol":"http","method":"post","path":"/api/tts_get_url","data":"{\"platform\":\"cloud\",\"message\":\"{{payload}}\",\"language\":\"pl-PL\",\"options\":{\"gender\":\"female\"}}","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":370,"y":260,"wires":[["e3f537014143b116"]]},{"id":"e3f537014143b116","type":"api-call-service","z":"2a0a9d47cb5cc092","name":"Play TTS","server":"8147d85a.d4ab48","version":3,"debugenabled":false,"service_domain":"notify","service":"alexa_media","entityId":"","data":"{\"target\":[\"media_player.echo_dot\"],\"data\":{\"type\":\"tts\"},\"message\":\"<audio src='{{payload.url}}'/>\"}","dataType":"json","mergecontext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":560,"y":260,"wires":[[]]},{"id":"58a143b6c9ca034d","type":"inject","z":"2a0a9d47cb5cc092","name":"Message","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"To jest mój test","payloadType":"str","x":180,"y":260,"wires":[["4d325b8a56405a8c"]]}]

Where exactly do I insert the Nabu Casa Remote UI URL ?

[solomos (Gerasimos Solomos)]

Nobody find this intersting ? Any help please ???
It would be great if I could have Alexa speak my language for Christmas …

[solomos (Gerasimos Solomos)]

I really need to solve this.
For anyone reading this pls drop me a line

Come on HA community! Where are all these nice folks always willing to help??

[butako123]

Hi @solomos
What I mean by Nabu Casa Remote URL is described on this page: Remote UI
It is a paid-for service. Once enabled in Settings → Home Assistant Cloud you will get a remote address to your HA instance, an address ending in ui.nabu.casa. It is this address that must be used in the message to alexa to play media.
HTH.

[solomos (Gerasimos Solomos)]

Thank you for your time and the reply to my msg.
I made some progress. Now it gives me the following error

30/12/2022, 10:09:40 μ.μ.node: Get TTS URL
msg : string[50]
"Error Message: Request failed with status code 401"

While Alexa says “Sorry, am having problem accessing Simon says smart skill right now”…

Any further help will be much apreciated

[solomos (Gerasimos Solomos)]

I also noticed that POST /api/tts_get_url never returns URL and path!

Instead it returns:

30/12/2022, 10:50:43 μ.μ.Get TTS URL
sent data : msg :
object
method: "post"
path: "tts_get_url"
data: "{"platform":"cloud","message":"hello how are you","language":"el-GR","options":{"gender":"male"}}"

My Node-Red node is as follows:

[butako123]

Hi,

So this took me a while but finally have a solution to share. Unfortunately, it is not straightforward in the slightest due to various constraints but I am documenting it here in case it helps someone. Finally, my Alexa speaks English with different accents (my favourite is the French, oh’la’la! or the Japanese-English pronunciation is also funny).

First up, you need a little web service (I run this on an Ubuntu server outside of Hassio).
Given a message on the URL, it will call the Hassio API tts_get_url and download the MP3. It then passes it through FFMPEG to convert into a format that Amazon Alexa likes. Finally return the MP3 to the callee. You need to generate a long lived token as indicated.

#!/usr/bin/env python3 
# 
import json
import requests
import shutil
import os
import time
import subprocess

from flask import Flask, send_file, abort, request
app = Flask(__name__)

# Define the API endpoint
URL = "http://hassio:8123/api/tts_get_url"

AUTH_TOKEN = "YOUR TOKEN HERE"

@app.route('/process')
def fetch_and_convert_tts_mp3():
    message = request.args['message'].strip()
    if not message or len(message) == 0:
        abort(500)
    # Define the payload
    payload = {"platform": "cloud", "message": message}
    # Convert the payload to JSON
    headers = {"Content-Type": "application/json", "Authorization": f"Bearer {AUTH_TOKEN}"}

    data = json.dumps(payload)

    # Send the POST request
    response = requests.post(URL, data=data, headers=headers)

    # Check the status code of the response
    if response.status_code != 200:
        print(f"An error occurred fetching MP3: {response}")
        abort(500)

    mp3_file = response.json()['url']

    response=requests.get(mp3_file, headers=headers)
    if response.status_code != 200:
        print(f"An error occurred fetching MP3: {response}")
        abort(500)

    raw_file = "tts.mp3"
    converted_file="tts2.mp3"
    if os.path.exists(raw_file):
        os.remove(raw_file)
    if os.path.exists(converted_file):
        os.remove(converted_file)

    with open(raw_file, "wb") as f:
        f.write(response.content)

    # ffmpeg -i <input-file> -ac 2 -codec:a libmp3lame -b:a 48k -ar 24000 <output-file.mp3>
    cmd = subprocess.run(['ffmpeg', '-i', raw_file, '-ac', '2', '-codec:a', 
                    'libmp3lame', '-b:a', '48k', '-ar', '24000', converted_file], 
                    stderr=subprocess.PIPE, stdout=subprocess.PIPE)
    print(cmd.stdout)
    print(cmd.stderr)

    return send_file(converted_file, mimetype='audio/mpeg')

if __name__ == "__main__":
    app.run(debug=True, port=5050, host="0.0.0.0")

Next, create a command_line sensor within your Hassio configuration like below. I also created an input_text field for typing messages into the Lovelace UI, mostly for debug purpose but you can substitute this for whatever message or text you want.

sensor:
  - platform: command_line
    name: cloud_to_alexa
    command: 'python3 ./cloud2alexa.py "{{ states(''input_text.alexa_announce'') }}"'
    scan_interval: 605800

The third step is the script cloud2alexa.py which should be inside your Hassio config directory. This script calls the earlier mentioned flask service, and saves the MP3 file into your /config/www/local which is accessible to Alexa.

import json
import requests
import shutil
import os
import time
import urllib.parse
import sys

www_local_dir = '/config/www/tts'
os.makedirs(www_local_dir, exist_ok=True)

def age_files(target_dir, age):
    # Get the current time in seconds
    now = time.time()

    # Iterate through the files in the directory
    for file in os.listdir(target_dir):
        if not file.endswith('mp3'):
            continue
        # Get the file's path
        file_path = os.path.join(target_dir, file)
        # Get the file's modification time in seconds
        mod_time = os.path.getmtime(file_path)
        # Calculate the age of the file in days
        age_in_days = (now - mod_time) / 86400
        # If the file is older than 5 days, delete it
        if age_in_days > age:
            os.remove(file_path)

id=int(time.time())
message = urllib.parse.quote(sys.argv[1])
response = requests.get(f'http://YOUR SERVER:5050/process?message={message}')
if response.status_code == 200:

    # Move the MP3 file to externally readable location
    with open(f'{www_local_dir}/tts_{id}.mp3', "wb") as f:
        f.write(response.content)
    
    age_files(www_local_dir, 1)
    print (f'tts_{id}.mp3')
else:
    print("False")
   

And finally, to glue it all together is the automation. This updates the sensor, which calls the script, which fetches the MP3, and then it invokes the Alexa Media Notify service with the fully baked URL.

Note the important thing is the Nabu Casa URL is sent to Amazon. As the op stated, Alexa can only play MP3 files from certain Internet locations (I assume AWS).

- alias: Announce Over Alexa
  initial_state: true
  trigger:
    - platform: state
      entity_id: input_text.alexa_announce
  action:
    - service: homeassistant.update_entity
      target:
        entity_id: sensor.cloud_to_alexa
    - service: notify.alexa_media
      data:
        message: '<audio src="https://YOUR NABU CASA HOSTNAME.ui.nabu.casa/local/tts/{{ states(''sensor.cloud_to_alexa'') }}"/>'
        target:
          - media_player.living_room
        data:
          type: tts

OK, so this is insanely complicated and I bet most people think it bonkers but this was the only way I could find to make Alex speak other languages, or English with accents and intonations etc. The Nabu Casa Cloud TTS has many more options. A nice fun way to personalize your Alexa announcements!

[sla004 (Simo Lääveri)]

You managed to solve this? Have you set access token for server?

[solomos (Gerasimos Solomos)]

I gave up because the instructions provided at the first place were wrong.
POST /api/tts_get_url never returns URL and path!

[sla004 (Simo Lääveri)]

I managed to get Node-Red version working.
I just had to change code a bit for Play TTS part:
{"target":["media_player.my_echo"],"data":{"type":"tts"},"message":"<audio src='https://my.ui.nabu.casa{{{payload.path}}}'/>"}
I added server nabu casa URL but for some reason it will answer with local address.

[solomos (Gerasimos Solomos)]

Dear Simo,
Would you please also share your Get TTS URL code part as I have been struggling to get tts_get_url to work ??

[sla004 (Simo Lääveri)]

Below is my full code:

[{"id":"9d4af7e4e53bdfec","type":"tab","label":"Alexa talks","disabled":false,"info":"","env":[]},{"id":"e9bf10a4e2a14899","type":"ha-api","z":"9d4af7e4e53bdfec","name":"Get TTS URL","server":"68035ac8f84e1597","version":1,"debugenabled":false,"protocol":"http","method":"post","path":"/api/tts_get_url","data":"{\"platform\":\"cloud\",\"message\":\"{{payload}}\",\"language\":\"fi-FI\",\"options\":{\"gender\":\"female\"}}","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":540,"y":160,"wires":[["06da753bb5060143"]]},{"id":"06da753bb5060143","type":"api-call-service","z":"9d4af7e4e53bdfec","name":"Play TTS","server":"68035ac8f84e1597","version":5,"debugenabled":false,"domain":"notify","service":"alexa_media_simos_2_echo","areaId":[],"deviceId":[],"entityId":[],"data":"{\"target\":[\"media_player.simo_echo\"],\"data\":{\"type\":\"tts\"},\"message\":\"<audio src='https://my.ui.nabu.casa{{{payload.path}}}'/>\"}","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":730,"y":160,"wires":[[]]},{"id":"94f9d2e6717d02f9","type":"server-state-changed","z":"9d4af7e4e53bdfec","name":"Message","server":"1ba751f1.76a74e","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"binary_sensor.etuovi_motion","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"on","halt_if_type":"str","halt_if_compare":"is","outputs":2,"output_only_on_state_change":true,"for":"3","forType":"num","forUnits":"seconds","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"Huomio! Joku on etuovella","valueType":"str"}],"x":320,"y":160,"wires":[["e9bf10a4e2a14899"],[]]},{"id":"68035ac8f84e1597","type":"server","name":"Nabu Casa","version":5,"addon":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":"30","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":": ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"default","statusTimeFormat":"h:m","enableGlobalContextStore":false},{"id":"1ba751f1.76a74e","type":"server","name":"Palvelin","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"","connectionDelay":false,"cacheJson":false,"heartbeat":false,"heartbeatInterval":"","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]

Get TTS URL:
[{"id":"e9bf10a4e2a14899","type":"ha-api","z":"9d4af7e4e53bdfec","name":"Get TTS URL","server":"68035ac8f84e","version":1,"debugenabled":false,"protocol":"http","method":"post","path":"/api/tts_get_url","data":"{\"platform\":\"cloud\",\"message\":\"{{payload}}\",\"language\":\"fi-FI\",\"options\":{\"gender\":\"female\"}}","dataType":"json","responseType":"json","outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"results"}],"x":540,"y":160,"wires":[["06da753bb5060143"]]},{"id":"68035ac8f84e","type":"server","name":"Nabu Casa","version":5,"addon":false,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":"30","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":": ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"default","statusTimeFormat":"h:m","enableGlobalContextStore":false,"credentials":{}}]

[sla004 (Simo Lääveri)]

have you made token for server access?

[solomos (Gerasimos Solomos)]

Dear Simo,
Thank you very much for youe kind reply.

I have made token for server access but untill now I am not able to play the mp3 file
Which eventually I see that is been created properly…

Instead I get Alexa saying that it can’t access Simon skill right now …

What I am missing ??? It drives me mad

[sla004 (Simo Lääveri)]

sorry late response…audio src is incorrect…first part should your own server nabu casa URL, I replaced it with https://my.ui… in my example