Home Assistant Voice Preview Edition cannot connect to local server and fails to download .flac voice response

Configuration Voice Assistant
1

During onboarding HA Voice Preview Edition the workflow says that the device cannot connect to HA server and directs me to a troubleshooting page to ensure that Settings → System → Network → Local Network URL is correct. (it is correct and it is using a https:// url with a lets encrypt certificate that is working for other devices on the network).

I pressed on anyway and added the device to ESP home and the adoption worked, latest code compiled and installed. I could issue voice commands successfully, but no voice response was given. Checking the device logs shows that it is getting an SSL error and trying to download it using the remote HA cloud URL instead of the local URL. I following is the log file (with my HA cloud URL changed - it is correct in the original log)

Couple of questions:

  1. Why is it trying to use the HA cloud URL instead of the local network URL configured in HA
  2. Why can it not download the flac file with a HA Cloud URL that works fine on my laptop if I put it into a browser?
  • Core2025.1.2
  • Supervisor2024.12.3
  • Operating System14.1
  • Frontend20250109.0

The specific SSL error:

[10:26:28][D][esp-idf:000][ann_read]: E (1881717) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x7280

The conversation log:

[10:26:23][D][micro_wake_word:355]: Detected 'Okay Nabu' with sliding average probability is 0.89 and max probability is 0.98
[10:26:23][D][media_player:080]: 'Media Player' - Setting
[10:26:23][D][media_player:084]:   Command: STOP
[10:26:23][D][media_player:093]:  Announcement: yes
[10:26:23][D][media_player:080]: 'Media Player' - Setting
[10:26:23][D][media_player:093]:  Announcement: yes
[10:26:23][D][nabu_media_player.pipeline:173]: Reading FLAC file type
[10:26:23][D][nabu_media_player.pipeline:184]: Decoded audio has 1 channels, 48000 Hz sample rate, and 16 bits per sample
[10:26:23][D][nabu_media_player.pipeline:211]: Converting mono channel audio to stereo channel audio
[10:26:24][D][voice_assistant:515]: State changed from IDLE to START_MICROPHONE
[10:26:24][D][voice_assistant:522]: Desired state set to START_PIPELINE
[10:26:24][D][voice_assistant:225]: Starting Microphone
[10:26:24][D][voice_assistant:515]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[10:26:24][D][voice_assistant:515]: State changed from STARTING_MICROPHONE to START_PIPELINE
[10:26:24][D][voice_assistant:280]: Requesting start...
[10:26:24][D][voice_assistant:515]: State changed from START_PIPELINE to STARTING_PIPELINE
[10:26:24][D][voice_assistant:537]: Client started, streaming microphone
[10:26:24][D][voice_assistant:515]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[10:26:24][D][voice_assistant:522]: Desired state set to STREAMING_MICROPHONE
[10:26:24][D][voice_assistant:641]: Event Type: 1
[10:26:24][D][voice_assistant:644]: Assist Pipeline running
[10:26:24][D][voice_assistant:641]: Event Type: 3
[10:26:24][D][voice_assistant:655]: STT started
[10:26:24][D][light:036]: 'voice_assistant_leds' Setting:
[10:26:24][D][light:047]:   State: ON
[10:26:24][D][light:051]:   Brightness: 66%
[10:26:24][D][light:109]:   Effect: 'Waiting for Command'
[10:26:24][D][power_supply:033]: Enabling power supply.
[10:26:25][D][voice_assistant:641]: Event Type: 11
[10:26:25][D][voice_assistant:804]: Starting STT by VAD
[10:26:25][D][light:036]: 'voice_assistant_leds' Setting:
[10:26:25][D][light:051]:   Brightness: 66%
[10:26:25][D][light:109]:   Effect: 'Listening For Command'
[10:26:27][D][voice_assistant:641]: Event Type: 12
[10:26:27][D][voice_assistant:808]: STT by VAD end
[10:26:27][D][voice_assistant:515]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[10:26:27][D][voice_assistant:522]: Desired state set to AWAITING_RESPONSE
[10:26:27][D][voice_assistant:515]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[10:26:27][D][light:036]: 'voice_assistant_leds' Setting:
[10:26:27][D][light:051]:   Brightness: 66%
[10:26:27][D][light:109]:   Effect: 'Thinking'
[10:26:27][D][voice_assistant:515]: State changed from STOPPING_MICROPHONE to AWAITING_RESPONSE
[10:26:27][D][voice_assistant:515]: State changed from AWAITING_RESPONSE to AWAITING_RESPONSE
[10:26:27][D][voice_assistant:641]: Event Type: 4
[10:26:27][D][voice_assistant:669]: Speech recognised as: "Turn off laundry lights."
[10:26:27][D][voice_assistant:641]: Event Type: 5
[10:26:27][D][voice_assistant:674]: Intent started
[10:26:27][D][power_supply:033]: Enabling power supply.
[10:26:28][D][voice_assistant:641]: Event Type: 6
[10:26:28][D][voice_assistant:641]: Event Type: 7
[10:26:28][D][voice_assistant:697]: Response: "Turned off the light"
[10:26:28][D][light:036]: 'voice_assistant_leds' Setting:
[10:26:28][D][light:051]:   Brightness: 66%
[10:26:28][D][light:109]:   Effect: 'Replying'
[10:26:28][D][voice_assistant:641]: Event Type: 8
[10:26:28][D][voice_assistant:719]: Response URL: "https://myhacloudurl.ui.nabu.casa/api/tts_proxy/jlEuNKyISsY4CXrf5ecRyA.flac"
[10:26:28][D][voice_assistant:515]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[10:26:28][D][voice_assistant:522]: Desired state set to STREAMING_RESPONSE
[10:26:28][D][media_player:080]: 'Media Player' - Setting
[10:26:28][D][media_player:087]:   Media URL: https://myhacloudurl.ui.nabu.casa/api/tts_proxy/jlEuNKyISsY4CXrf5ecRyA.flac
[10:26:28][D][media_player:093]:  Announcement: yes
[10:26:28][D][power_supply:033]: Enabling power supply.
[10:26:28][D][voice_assistant:641]: Event Type: 2
[10:26:28][D][voice_assistant:733]: Assist Pipeline ended
[10:26:28][D][esp-idf:000][ann_read]: E (1881717) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x7280

[10:26:28][D][esp-idf:000][ann_read]: E (1881719) esp-tls: Failed to open new connection

[10:26:28][D][esp-idf:000][ann_read]: E (1881720) transport_base: Failed to open a new connection

[10:26:28][D][esp-idf:000][ann_read]: E (1881723) HTTP_CLIENT: Connection failed, sock < 0

[10:26:28][E][nabu_media_player.pipeline:171]: Media reader encountered an error: ESP_ERR_HTTP_CONNECT
[10:26:28][E][nabu_media_player:305]: The announcement pipeline's file reader encountered an error.
[10:26:28][D][voice_assistant:515]: State changed from STREAMING_RESPONSE to IDLE
[10:26:28][D][voice_assistant:522]: Desired state set to IDLE
[10:26:28][D][light:036]: 'voice_assistant_leds' Setting:
[10:26:28][D][light:047]:   State: OFF
[10:26:28][D][light:109]:   Effect: 'None'
[10:26:38][D][power_supply:048]: Disabling power supply.
2

I was able to get HA Voice PE replying by removing TLS security from configuration.yaml and updating the local URL to HTTP instead of HTTPS

Not a great solution as now everything is in plain text.

Has anybody else got Voice PE working with a local HTTPS connection using a valid Let’s Encrypt certificate?

3

I’ve seen a few others who have encountered this problem where “local URL” using https does not work but have not seen a solution other than what you did or use nginx proxy. I would suggest writing up an issue on this here. I am still waiting for my VPE, but I think I’m going to run into the same problem.

4

Thanks for the link to submit a ticket, was not sure where of actually submit an issue beyond community forum. Please message with how you go when you receive yours as I think that this is not a great outcome if the device is forcing a security downgrade and would like to help solve it.

5

I’m also having this issue. I have also tried taking over the firmware and adding the following lines:

http_request:
  verify_ssl: false

That didn’t fix the issue, so I’m thinking it must be something other than certificate validation. Like you, also not sure why it’s using the HA cloud URL instead of the local HA URL.

Did you ever figure out a solution to the issue?

6

Did some more testing. I was able to get it to work on HTTPS with the Nabu Casa remote connection turned off and the settings in my previous post (verify_ssl: false) on the firmware (I need this since I’m running a local CA with the built-in HA HTTPS). Not sure why it’s defaulting to the remote instance for this local device. Will investigate more.

2 Likes
7

Wanted to chime in to say I’m having a similar issue. I don’t have my HA setup for remote access, just local. But have HTTPS setup with a local certificate. I’m able to issue commands through the Preview Edition but can’t connect any voice assistant through it. I am troubleshooting too but haven’t has any luck.

8

Similar problem here. While setting up, I get a message saying “cannot connect to local server”, but I can see it in my devices, and I have enough control to turn the LED ring on and off, and change the wake word.
As with all things Home Assistant, there must be something very basic I am not doing.

9

Think i have a similar issue, however i have this on ESP side:
“ESP_ERR_HTTP_CONNECT”

also ge these warnings when using “take control” and install:

image
image2101×333 26.6 KB

image
image1665×396 17.7 KB

and many other warnings.

[UPDATE]:

I thought first was issue with my home assistant being in HTTPS, but no!

Actually i think it was a misconfiguration on the local home assistant URL.

So I went to Settings > System > Network and the make sure the “local network url” is http://[your ip address]:8123

then tried the voice assistant setup again and it worked.

10

I have been working through this and have found a fix, would be great if someone from the Home Assistant side could get this addressed in the firmware build.

I provided an example of how to recompile the firmware for Voice PE to get it to work 100% of the time and have no mbedtls issues by disabling an area of the software that has been programmatic. I’ve also linked to the mbedtls issues where it calls out all of these issues after people started enabling TLS 1.3 support incorrectly.

If someone could test my fix on their end that’s having the same problem I think we can better drive this to resolution. Thanks!

EDIT:
The final, and reliable, workaround is here: PE device has no TTS responses (server SSL) · Issue #315 · esphome/home-assistant-voice-pe · GitHub

1 Like
11

This worked for me (I only added the esp32: section). The odd thing is that VPE has been working fine until this morning.

12

From what I can tell the mbedTLS library does not handle TLS1.3 very well at all. And, how it’s currently implemented sends, incorrectly, TLS1.3 extensions on TLS1.2 connections.

My hypothesis is that TLS1.3 may slowly cropping up for folks (both on the HA Cloud as well as in home networks) and causing the breakage. I could be wrong. The other thing that was found (see the Github thread) was that the logic in how the VPE connects looks to be broken which is causing onboarding for a lot of folks to fail if they have TLS deployed in their HA instance.

Apparently the VPE does not officially support TLS on the local network side of things according to Nabu Casa. Although, it really doesn’t say that anywhere on the product page. I mentioned to them (in a support ticket) they should really be more up front about this as this has caused a lot of headaches for folks in the forums. Anyway, for folks that have the issue of onboarding (which is pretty easy to work around) or having issues with connections not returning TTS playback - feel free to check out the “workaround” I provided by forcing the VPE to only leverage TLS1.2 connections with all TLS1.3 extensions disabled.

13

I just got mine.
I use nginx split dns with the same name internal and external url
And let’s encrypt certificate.

Is there now any fix for this issue? Can tell me how to fix this without changing the internal url to an insecure http url