I have been able to take the yaml from JLo above and apply it to the most recent version of the home-assistant-voice-pe repo on a fork which can be found here.
If you have the ESP Home Builder add-on installed, you have the option of taking control of your Voice Assistant device with that and loading my fork onto it instead. It is working fine with openWakeWord here; not tested for hours yet, but looking good so far.