then you need to try to find out what you do different in the moments that it isnt working.
but it is very well possible that a slow connection, makes that the call is timed out.
first step that i would take is look into the alexa app history, to see if your commands all come through.
there are several things around.
plain (which is i think nothing more then the original) which is just a linux executable.
something that is adapted for connecting to HA, which i believe has special output
the alexa_mediaplayer component.
i just use the plain. because i like to tinker
if you only want tts and not things like starting the weatherreport i suggest you start using the media_player component. if installed correct it seems a bit more reliable and more easy to use.