Sonoff TX Ultimate and Voice Assistant

[PxPert (Pietro)]

Hi to all,
I’m new to this forum, but not new at all with home assistant…
I’ve converted (or better… tasmotized ) all of my home components since 2020 with plenty of mqtt (sonoff and esp8266 based built by myself) and zigbee devices.

I’ve recently bought some Sonoff TX Ultimate to test them (well, firstly with tasmota as I’ve never tried esphome before) and their features.

These seem really cool devices:

• Touch panel with 10 horizontal points (as I live in Italy i’ve got the 120 version of these devices) and gesture support
• up to 4 relays
• Led controlled with standard library
• Haptic support
• Even a speaker!

What i’ve missed was only microphone support! So why not just add one?
After flashed with esphome I decided to pick up and solder a mems microphone.
I used an INMP441 (one of those with circular breakout board).
The microphone was soldered using these pins available on the TX Ultimate:
GND → GND Hole (the one used to power the board when flashing the firmware)
VDD → 3V Hole (the one used to power the board when flashing the firmware)
SD → GPIO 0 (The boot pin)
WS → RX Pin
SCK → TX Pin
L/R → Floating (not soldered)

The good news is that it works perfectly! I’ve tried it with my voice assistant (so with wake word, STT, TTS Feedback Etc.)

For those interested:
For better sound capture quality I’ve placed the microphone between the mainboard and the Touch PCB. The pcb itself has a small hole used for positioning it on the cover. So I’ve drilled it with a small point to remove the plastic support of the cover, and aligned the microphone with it.
I’ve also removed with a Dremel some plastic to fit the microphone. Then I’ve remounted the main motherboard and soldered the cables to it.
You could also reduce the microphone PCB board size to fit it in the cover, under the speaker without having to adapt it (except for the microphone hole), but it’s more complicated as you could damage the microphone itself
Or you could also simply put the microphone over the mainboard: No modifications needed, but probably the microphone will be less sensitive.

This is a photo of the microphone placed between the mainboard and the touch interface

And here you can find my ESPHome repository with the TX and Voice Assistant functionality: GitHub - PxPert/esphome: esphome configuration
Please note that this is still a work in progress!

Anyway the yaml configuration is more modular as possible. Right now it support:

• Voice Assistant with led notification if wake word is active / Speech is detected / Response feedback
• Support for 1 or 2 virtual buttons (more to come)
• Light status feedback via LED (only the leds corresponding to the virtual button position will light up) with the possibility to turn them off
• Gestures status feedback via different LED colors / transitions
• Night light status

Next steps will be:

• Enable/Disable night light status by long press (in a specific position)
• Enable/Disable light feedback status by long press (in a specific position)
• Enable/Disable Wake word light (If the VA Component is included) status by long press (in a specific position)
• Start voice assistant via multitouch (If the VA Component is included)
• Develop a sort of “Remote MQTT Relay” to emulate the standard deviators functionality

You can enable / disable some feature simply by including/excluding or selecting the corresponding yaml file (Ex. to use only 1 or 2 virtual buttons)

All of this would not be possible without Esphome, Home Assistant itself and blakadder who developed the touch component. So thank you all!

PS I still haven’t created a readme (With the references for blakadder hard work) for my repo and the touch component is roughly imported. As I’ve said: Work in progress

Now some questions:

• I’d like to implement a way to let the user set the specific gestures led colors / brightness via Home assistant. Is there any virtual light (or sort of) component I can use to implement this feature?
• I’ve seen that by enabling wake word, the microphone is stopped and restarted every 5 second (without errors, I’ll post the log soon). Is this a normal behaviour?
• Is there a way to post process the TTS sound? What I need is simply to amplify it because the speaker is not so loud.

Thanks for your support, and in case you’ve got questions about my project, just ask!

[Arnaud-Albrecht]

Very cool project,

Have you carried on ? i just bought a house and bought the TX5 ultimate to see if i switch everything to it.

I will flash mine this week and start playing with it

[PxPert (Pietro)]

Hi!
I’m really sorry I’m very busy and unfortunately didn’t receive any notification for your reply… I’ve already filled my house with TX Ultimates (With microphones where I needed a Voice Assistant) Anyway I’ll try do document the yaml as soon as possible to let you use the available features. In the meantime I’ll upload a small video showing what this little device can do

[PxPert (Pietro)]

Hi, this is a video of one of my devices with a brief live description of the features supported: https://www.youtube.com/watch?v=AVS_tiUuSgQ
I’ve also updated my GitHub - PxPert/esphome: esphome configuration repository by adding some documentation to the tx_ultimate_00_base.yaml . Start from this file to write your own configuration

[the-mentor]

This mod seems to be really cool.
I was thinking of trying it out but I have the square version of the tx ultimate switch.
I wonder if that is a downside since it probably has less space inside.

Also the speaker in the tx ultimate switches is very low and I’m wondering what I can do about it.

[PxPert (Pietro)]

I don’t know, the microphone is really small and with a bit of patience you could even reduce its size using a dremel.

From the firmware side: maybe you’ll need to do some adjustments to make it work correctly on the square version (The number of leds and the touch zones are very probably different) but that should not be an issue at all.

Talking about the speaker: You are right, it’s a bit low on volume, that’s why i’m looking for software side solutions like:

• Doing some sort of audio post processing to amplify the output
• Replacing the TTS with locally stored and prerendered sound effects

[DDock]

This is a project a bit out of my league, but I’m willing to learn. I’ve attempted to get this running, but I think I’m attacking this all wrong. I’m even willing to write up a guide for others once I can get mine done.

Some things on my side:

• Setting up a TX Ultimate 4-Switch Variant
• I’ve copied the components, fonts, and includes folder into where my device’s YAML file had been moved to in ESPHome.
• Modified the code to only include only the switch components, no MQTT stuff
• I’ve attempted to validate, but have errors

Here are some questions:

• I’ve added the Wi-Fi secrets to my Secrets list, but had gotten errors for api_key, ota_pwd, and ap-pwd in the tx_ult_base.yaml file and don’t see any documentations on what those are supposed to be. Ended up commenting those out in the yaml file to continue on.
• Now I am getting this error
  esp_adf: [source config/includes/tx_ultimate_voice_assistant.yaml:148]
  This feature is only available with frameworks [‘esp-idf’].
• There are also warnings about strapping pins, but I’m guessing this is normal.

Any help would be appreciated. Thank you again for the cool project and look forward to getting this working.

[PxPert (Pietro)]

Hi,
The firmware needs to be compiled with esp-idf framework. Did you change the framework in the tx_ultimate_base.yaml?
Also, you need to set the above secrets:
api_key : That’s the encrypted key used to communicate with the device
ota_pwd: That’s the password used for OTA Updates.
ap_pwd : The Password to use if the device goes into AP (Access Point) mode

PS Tell me if you have bad sound coming from the speaker (for example when it’s playing the TTS response): it can be a speaker library problem and not an hardware issue

[DDock]

Sorry for the late reply, but never got notified, even though I had notifications enabled.

Update: I got the code fixed up and have the device showing up in HA now with all 4 switches. I had some extra code on there from when I had initially flashed the TXU. Once the code was cleaned up, it is now displaying properly. My i2c mics have shown up now, so I will work on that next. Will update when I can.

[DDock]

Just wanted to confirm that I have 3 TXUs set up (probably a 4th by the end of the day) and they all have static on the voice reply. Love the project and have a few more TXUs coming.

[PxPert (Pietro)]

Glad to hear that!
Talking about the static on the voice reply, you need to patch the I2SAudioSpeaker::player_task method in esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp:

 void I2SAudioSpeaker::player_task(void *params) {
@@ -44,7 +44,8 @@
       .sample_rate = 16000,
       .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
       .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
-      .communication_format = I2S_COMM_FORMAT_STAND_I2S,
+//      .communication_format = I2S_COMM_FORMAT_STAND_I2S,
+      .communication_format = I2S_COMM_FORMAT_STAND_MSB,
       .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
       .dma_buf_count = 8,
       .dma_buf_len = 128,

That’s because TX Ultimate codec uses a different communication format. This can be changed in the media_player component but it’s still hardcoded in the I2SAudioSpeaker, so I had to manually patch it.
PS Yes, I’ll need to make it configurable and post a PR to the esphome project, whenever I’ll have some time

[DDock]

Awesome, thanks for that! I’ll give that a try soon. I’m guessing I’ll have to recompile the code and upload once I changed the include? I’m running ESPHome in a docker, so I’ll have to find that file and edit it.

Do you know offhand if I will need to go into the other config files to change the on/off times for the night lights?

— Not sure if you will see this —

Somehow I’ve reached my maximum limit for this thread and it’s not letting me message more since I’m a “new” user. I don’t even see a way to PM you. Not sure if your willing to message me on Discord, but it’s the same username and profile picture.

I’ve noticed the lights turn on after sunset, but they actually turn off at midnight my time for some reason. Any thoughts?

[PxPert (Pietro)]

Oh well,
If you’re using docker you’ll need to manually edit (at least that’s what I’ve done right now) the cpp located here:
/esphome/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp
and then recompile and reupload the updated firmware.
Make sure that the patch is applied by looking at the corrisponding i2s_audio_speaker.cpp created by the build. it should be in
/config/.esphome/build/<PROJECT_NAME>/src/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp

The night light is automatically turned on at sunset and off at sunrise by using the esphome sun component. Just make sure the correct latitude, longitude and timezone values are set in the includes/tx_ultimate_base.yaml file (yup didn’t notice they’re actually hardcoded, sorry for that )

[DDock]

I was able to get the speaker patched. For some reason the files would always get patched went attempting to install the update. I would just have to throw the edited file in just after it attempted to download a new copy.

I should have the time zone fixed on my configs now to be EST not Rome time. I’ll update the next time I’m up past midnight lol.

[TommySharpNZ (Tommy)]

Would you consider doing a bit of a write up somewhere now that you seem to have it working nicely?

[DDock]

Yes, I plan on doing something, both for my records and to help others. I would like to do a video on it, but life has gotten busy and haven’t had many weekends to work on fun projects again.

[PxPert (Pietro)]

Hi,
Just to inform you I’ve created a PR for ESPHome to be able to set the correct communication format for our Sonoff TX Ultimate devices. You can find the PR here: Added i2s_comm_fmt parameter to i2s speaker component by PxPert · Pull Request #7449 · esphome/esphome · GitHub

[DDock]

Oh nice, this is the big thing I’ve been waiting for! Is there any way to help upvote this code edit to try and get it pushed through quicker?

It is quite the pain as each push to the switch from ESP Home wants to download an “updated” version of the edited file and breaks the speaker each time. I’ve found the best way is to intercept during the compile process and put the edited file in each time. Sadly there isn’t a single place to do this and you have to do it within each project folder.

I will be having some free time to get going again, and I just got my new batch of TX Ultimates in the mail last week.

[EdwardTFN (Edward Firmo)]

Ciao!
I couldn’t find your yaml… Do you mind sharing it?
I’m more interested on how you set the lights with esp-idf.
Also, I see your PR for changing i2s_comm_fmt… I will play with that later. I was previously using the media_player component, but again couldn’t find a way to use that with esp-idf, yet.

Have to find time to play deeper with the TX Ultimate…

[PxPert (Pietro)]

Hi @EdwardTFN , I’ve renamed my repo to esphome-config as I used the esphome name to fork (and work on) the esphome code. You can find it here: GitHub - PxPert/esphome-config: esphome configuration. Please note that the media_player component is not compatible with esp-idf framework, but only with arduino. You’ll need to use the speaker component with esp-idf. For the leds, you’ll need to use the esp32_rmt_led_strip platform. You can find references to it here: esphome-config/includes/tx_ultimate_base_leds.yaml at a96509448c5aec69f91359b9b7614b0eff948092 · PxPert/esphome-config · GitHub

@DDock The PR seems to be approved and in “merge after release” state, so it’ll be present in the next esphome release (i hope )
In the meantime you can checkout my i2s_comm_fmt branch in my esphome fork here: GitHub - PxPert/esphome: ESPHome is a system to control your ESP8266/ESP32 by simple yet powerful configuration files and control them remotely through Home Automation systems.
I’ll also push a docker image in dockerhub so if you are using docker to deploy the firmware you’ll be able to use it by just switching the esphome image link.
Please note that that’s a dev branch so Bugs can occour, and there is a change in the yaml configuration parameter of the speaker component:
mode: mono has been renamed to channel: mono