Http timeout for voice assistant pe, even though the response is recieved

ESPHome
1

Quite often my VA fails to respond with the error HTTP_CLIENT: Connection timed out before data was ready!. I see the issue was posted on git a week ago but there has been no response.

I am using the default sketch, no changes have been made. I can’t find where this timeout is defined. I assume that if it’s increased, it will correct this.

[09:41:25][D][voice_assistant:669]: Speech recognised as: " Who is Michael Jackson?"
[09:41:25][D][voice_assistant:641]: Event Type: 5
[09:41:25][D][voice_assistant:674]: Intent started
[09:41:25][D][power_supply:033]: Enabling power supply.
[09:41:25][D][power_supply:033]: Enabling power supply.
[09:41:26][D][power_supply:033]: Enabling power supply.
[09:41:26][D][voice_assistant:641]: Event Type: 6
[09:41:26][D][voice_assistant:641]: Event Type: 7
[09:41:26][D][voice_assistant:697]: Response: "Michael Jackson was a highly influential and popular American singer, songwriter, and dancer.  He's widely considered one of the most significant musical artists of all time, known for his distinctive voice, innovative dance moves (like the moonwalk), and groundbreaking music videos.  His career spanned several decades, and he achieved immense global success with albums like *Thriller*, *Off the Wall*, and *Bad*."
[09:41:26][D][light:036]: 'voice_assistant_leds' Setting:
[09:41:26][D][light:051]:   Brightness: 66%
[09:41:26][D][light:109]:   Effect: 'Replying'
[09:41:26][D][voice_assistant:641]: Event Type: 8
[09:41:26][D][voice_assistant:719]: Response URL: "http://192.168.0.101:8123/api/tts_proxy/TDshWjXeeXpMfBGMl63n_g.flac"
[09:41:27][D][voice_assistant:515]: State changed from AWAITING_RESPONSE to STREAMING_RESPONSE
[09:41:27][D][voice_assistant:522]: Desired state set to STREAMING_RESPONSE
[09:41:27][D][media_player:080]: 'Media Player' - Setting
[09:41:27][D][media_player:087]:   Media URL: http://192.168.0.101:8123/api/tts_proxy/TDshWjXeeXpMfBGMl63n_g.flac
[09:41:27][D][media_player:093]:  Announcement: yes
[09:41:27][D][voice_assistant:641]: Event Type: 2
[09:41:27][D][voice_assistant:733]: Assist Pipeline ended
[09:41:31][D][esp-idf:000][ann_read]: W (444982918) HTTP_CLIENT: Connection timed out before data was ready!
2

Looks like your HA instance is ignoring connections from your ESP. Firewall rules? Try putting a laptop up with the same IP (turn off your PE) in the same network and attempt connection to port 8123 of HA.

3

It looks like the problem lies with piper. If I switch the tts to google or pico, this doesn’t happen.

I see this at the same time VA throws the error, in the piper logs.

DEBUG:wyoming_piper.handler:Synthesize(text='Michael Jackson was an American singer, songwriter, and dancer.  He is one of the most popular and influential musical artists of all time.', voice=SynthesizeVoice(name='en_US-lessac-high', language=None, speaker=None))
DEBUG:wyoming_piper.handler:synthesize: raw_text=Michael Jackson was an American singer, songwriter, and dancer.  He is one of the most popular and influential musical artists of all time., text='Michael Jackson was an American singer, songwriter, and dancer.  He is one of the most popular and influential musical artists of all time.'
DEBUG:wyoming_piper.handler:input: {'text': 'Michael Jackson was an American singer, songwriter, and dancer.  He is one of the most popular and influential musical artists of all time.'}
DEBUG:wyoming_piper.handler:Sent info
DEBUG:wyoming_piper.handler:/tmp/tmphc4nasso/1737598455088302143.wav
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='wyoming event handler' coro=<AsyncEventHandler.run() done, defined at /usr/local/lib/python3.11/dist-packages/wyoming/server.py:31> exception=BrokenPipeError(32, 'Broken pipe')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/wyoming_piper/handler.py", line 48, in handle_event
    return await self._handle_event(event)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/wyoming_piper/handler.py", line 114, in _handle_event
    await self.write_event(
  File "/usr/local/lib/python3.11/dist-packages/wyoming/server.py", line 29, in write_event
    await async_write_event(event, self.writer)
  File "/usr/local/lib/python3.11/dist-packages/wyoming/event.py", line 131, in async_write_event
    await writer.drain()
  File "/usr/lib/python3.11/asyncio/streams.py", line 378, in drain
    await self._protocol._drain_helper()
  File "/usr/lib/python3.11/asyncio/streams.py", line 167, in _drain_helper
    raise ConnectionResetError('Connection lost')
ConnectionResetError: Connection lost
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/wyoming/server.py", line 41, in run
    if not (await self.handle_event(event)):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/wyoming_piper/handler.py", line 50, in handle_event
    await self.write_event(
  File "/usr/local/lib/python3.11/dist-packages/wyoming/server.py", line 29, in write_event
    await async_write_event(event, self.writer)
  File "/usr/local/lib/python3.11/dist-packages/wyoming/event.py", line 131, in async_write_event
    await writer.drain()
  File "/usr/lib/python3.11/asyncio/streams.py", line 366, in drain
    raise exc
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 1057, in write
    n = self._sock.send(data)
        ^^^^^^^^^^^^^^^^^^^^^
BrokenPipeError: [Errno 32] Broken pipe
4

I can’t find where this timeout is defined. I assume that if it’s increased, it will correct this.

I had the same issue (“HTTP_CLIENT: Connection timed out before data was ready!”) when the TTS took too long to respond. I traced the timeout to this line in the “nabu” component: client_config.timeout_ms = 5000;.

What I’ve done is copied the nabu component to a folder with custom components, increased the timeout on that line in the copied component and added the following to my esphome config to load my custom components folder (change the path to reflect your setup):

external_components:
  - source:
      type: local
      path: /home/homeassistant/esphome_config/.esphome/custom_components

Now it waits for TTS to finish producing the response and announces it correctly, even for long responses!

4 Likes
5

For long TTS responses I am now testing this alternative approach based on real-time streaming from LLM to TTS directly: Streaming LLM's responses into TTS for near-instant responses (works with HAVPE!)

1 Like
6

Hi I’m running into this timeout issue. I’m using GEMINI LLM. Will this fix work with gemini? Also I’m pretty new to HA and I’m trying to follow your steps to get the timeout increased. I can’t quite get there. I can’t seem to find where esphome config lives.
Could you provide more detailed steps please? Thank you for any help you can give me!

7

I have used this (I’m running ollama with a LLM, and it does give lengthy replies from time to time). I did also try the adf integration, but that gave me some CM error and that’s when I got to the Nabu solution. To edit the nabu mediaplayer timeout, follow the instructions above (thanks gyrga!). I’ll just share my steps:

  1. Download the components:
git clone https://github.com/esphome/home-assistant-voice-pe.git

  1. Copy them in your config/esphome/ folder (using the samba addon (I’m on a RPi4))

  2. Change this in your ESP32 code:

external_components:
  - source:
      type: local
      path: /config/esphome/home-assistant-voice-pe/esphome/components
1 Like
8

I’ve updated my solution to create a near-native real-time streaming, see the new version here: GitHub - eslavnov/ttmg_server: Talk To Me Goose Server

9

I have the exact same issue. Longer responses are not working with either Piper or Google Cloud TTS.

Can you please elaborate, how do I edit my ESP32 code? I have Home Assistant Voice PE up and running and don’t have ESP32 addon.

Thank you!