Year of the Voice - Chapter 5

[donburch888 (Don Burch)]

if the new commands aren’t working for you …

… did you remember to Restart Home Assistant after setting up your new satellite device ?

Seems to have made a huge difference for me

[sparkydave (🇦🇺 Dave)]

I realise that the “what’s the temperature” was very vague but it was a repeat of what the previous person had tried so I was curious.

“What is the temperature in the kitchen” I thought might have worked since I have a sensor in that area.

[sparkydave (🇦🇺 Dave)]

Not sure if this was a reply to me, but my screen shots were just from a web browser, not using a satellite

[tetele]

@sparkydave

“What is the temperature in the kitchen” I thought might have worked since I have a sensor in that area.

Unfortunately, that sentence works only with climate entities for now.

[Fraddles (Chris)]

It took me a while to figure that out…
“What is the temperature in the bathroom?” - does not work
“What is the bathroom temperature?” - works as expected

Also took me a while to figure out why I could not open the ‘blind’ in ‘Harry’s Room’… because Assist calls it a ‘shade’

However just tried the new wyoming-satellite and it works very well!

[donburch888 (Don Burch)]

OK, I have today repurposed my test RasPi 3B to a Chapter 5 satellite with wyoming-satellite and wyoming-openwakeword. Very straight-forward using the tutorial.

• it seems to be working well, but of course still lots of testing
• “nevermind” has to be the whole command. “oops … nevermind” will time out with no benefit

[donburch888 (Don Burch)]

@synesthesiam

This is what I did for my Pi Zero 2 W satellite, just LEDs for feedback: https://github.com/rhasspy/wyoming-satellite/blob/95f2e58c3a4d4d91aaebbbd3784ed7c7c6e631c2/docs/tutorial_2mic.md

I was using HermesLedControl for that with Rhasspy. Easy to make fun patterns with all the LEDs on the reSpeaker 4-mic HAT so I’m wondering how easy to change it from MQTT to Wyoming. Didn’t you previously write a document for that ?

[Lakini]

@donburch888

reSpeaker 4-mic HAT

Did you get that running together with your pi3 if i understood that right?

[donburch888 (Don Burch)]

@Lakini

Did you get that running together with your pi3 if i understood that right?

Yes, one of my voice assistant satellites is a RasPi 3A with reSpeaker 4-mic HAT, running Rhasspy and HermesLedControl.

Seeed made at least 3 different 4-mic devices with very similar confusing names

• ReSpeaker 4-Mic Array for Raspberry Pi which is a HAT connecting by GPIO
• ReSpeaker 4-Mic Linear Array Kit
• ReSpeaker USB 4 Mic Array

I was particularly impressed that they made these devices, made a driver and some demo software … and then abandoned them. When I bought mine their support was advising customers to use RasPi OS that was over 2 years old because their firmware wouldn’t run on newer OS versions The hardware has 2 or 4 mics, and the demo programs (no source code available) apparently can use all 2- or 4- mics … but their device driver only uses one mic. They have provided NO support.

[Lakini]

Ah, too sad with the one-mic limitation, and the lack of current support. I do have the HAT version laying around, from my try-outs with Snips a few years ago, on a Pi3b.
Was originally hoping I could potentially revive that combination with the new HA satellite solution, but doesnt sound like it then…

[Lakini]

Oh, someone seems to have taken matters into their own hand recently:

ReSpeaker 4-Mic Array for Raspberry Pi | Seeed Studio Wiki ·...

[ddppddpp]

I have an area called ‘Kitchen’ and a device called TRV Kitchen that exposes a climate entity with the area set accordingly. When I ask ‘What is the temperature in the kitchen’ I get the same error as above - unexpected error during intent recognition. Same for ‘What’s the Kitchen temperature’ and ‘what’s the TRV Kitchen temperature’. How can I troubleshoot this?

[donburch888 (Don Burch)]

@Lakini

hoping I could potentially revive that combination with the new HA satellite solution, but doesnt sound like it then…

Not what I was saying. They do work OK, and I don’t think there will be any problem upgrading my RasPi + ReSpeaker satellite to chapter 5.

I was moaning that the Seeed reSpeaker devices don’t live up to their potential, and so don’t justify their price tag.

@Lakini

someone seems to have taken matters into their own hand recently:

Yes, one of their users, HinTak has been updating the driver to work with newer RasPi OS kernels … but he does not attempt to improve the functionality of the driver beyond this.

[chpego]

@system

Area awareness

Voice satellites can be placed all around the house, and it’s important to keep their area in mind when interpreting commands like “turn on the lights”. This command will now turn on all of the lights in the satellite’s area, and “turn off the lights” will do the opposite. You can still target the lights in a different area, of course, by specifying: “turn on the lights in the bedroom”.

Voice satellites make use of the area they’re in.

This is a small start to satellites being aware of their context, and adjusting behavior accordingly.

How can we do that, with a m5 atom echo ?

[tetele]

Set the device area (device page > pencil top right > Area)

[donburch888 (Don Burch)]

Unfortunately area awareness doesn’t seem to be working for me. One of my LIFX bulbs is in Study, and my Wyoming device (a RasPi 3 running wyoming-satellite and wyoming-openwakeword) is set to in Study.

However the log from the device shows that it couldn’t understand “turn off the light” unless I called it “study light”.

Dec 16 22:42:48 HA-voice-2 run[1273]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 22:42:53 HA-voice-2 run[945]: DEBUG:root:Streaming audio
Dec 16 22:42:53 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 22:42:53 HA-voice-2 run[1275]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 22:42:59 HA-voice-2 run[945]: DEBUG:root:Event(type='transcript', data={'text': ' Turn on study light.'}, payload=None)
Dec 16 22:42:59 HA-voice-2 run[945]: INFO:root:Waiting for wake word
Dec 16 22:42:59 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 22:42:59 HA-voice-2 run[1277]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 22:42:59 HA-voice-2 run[945]: DEBUG:root:Event(type='synthesize', data={'text': 'Turned on light', 'voice': {'name': 'en_GB-alan-low'}}, payload=None)
Dec 16 22:43:00 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 22:43:00 HA-voice-2 run[1280]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 23:18:02 HA-voice-2 run[945]: DEBUG:root:Streaming audio
Dec 16 23:18:02 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 23:18:02 HA-voice-2 run[1312]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 23:18:08 HA-voice-2 run[945]: DEBUG:root:Event(type='transcript', data={'text': ' Turn off the light.'}, payload=None)
Dec 16 23:18:08 HA-voice-2 run[945]: INFO:root:Waiting for wake word
Dec 16 23:18:08 HA-voice-2 run[945]: DEBUG:root:Event(type='synthesize', data={'text': "Sorry, I couldn't understand that", 'voice': {'name': 'en_GB-alan-low'}}, payload=None)
Dec 16 23:18:08 HA-voice-2 run[1314]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 23:18:08 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 23:18:09 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 23:18:09 HA-voice-2 run[1317]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 23:18:14 HA-voice-2 run[945]: DEBUG:root:Streaming audio
Dec 16 23:18:14 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 23:18:14 HA-voice-2 run[1319]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 23:18:19 HA-voice-2 run[945]: DEBUG:root:Event(type='transcript', data={'text': ' Turn off study light.'}, payload=None)
Dec 16 23:18:20 HA-voice-2 run[945]: INFO:root:Waiting for wake word
Dec 16 23:18:20 HA-voice-2 run[945]: DEBUG:root:Connected to snd service
Dec 16 23:18:20 HA-voice-2 run[1321]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Dec 16 23:18:20 HA-voice-2 run[945]: DEBUG:root:Event(type='synthesize', data={'text': 'Turned off light', 'voice': {'name': 'en_GB-alan-low'}}, payload=None)
Dec 16 23:18:21 HA-voice-2 run[945]: DEBUG:root:Connected to snd service

[chpego]

Thanks, that’s what I did, even though there’s no information written down anywhere.

[Hedda]

@donburch888

@Hedda

Hope we will soon see ready-made smart speaker hardware projects

Definitely. Available options (Raspberry Pi, conferencing speakers, Onju-voice and ESP32-S3 BOX) are getting better, but still noticeably lower quality and more expensive than alexa/google devices.
ESP32-S3 BOX seems a bit of an overkill (I would be happy with a couple of LEDs instead of a screen) and I understand some of its desirable features are still locked away.

Would be awesome if both Nabu Casa as well as third-parties could replicate the functionality and relatively high-quality audio of the speaker and microphone hardware in second-generation Google Nest Mini at under $99 or less with a powerful enough ARM-64 SoC to handle custom wake-words onboard.

IMHO, replicating the whole Google Home / Google Nest smart speaker and smart display series with fully Open-Source Hardware (OSH) compliant projects at affordable prices should preferably be the goal!

Great if we could order such products like we can build + make or buy the Home Assistant Yellow today!

GitHub - NabuCasa/yellow: Home Assistant Yellow - hardware for Home Assistant...

Hardware – Home Assistant Yellow

Home Assistant Yellow

[pcwii]

I would be very surprised if this ever happens, not because it would not be great, I think we all would love something like this but that hardware from google, Amazon, or Apple is actually worth far more money than you can buy them for. The companies are prepared to loose money when they sell them because they expect a financial return on the insight you provide by your voice data that they collect. I think this is part of the reason many of the original suppliers are loosing money with their smart speaker decisions and closing or reducing services. This would certainly not be possible with any open source project. This doesn’t mean I wouldn’t love to see it but I won’t hold my breath on this one.

[Lakini]

@donburch888

Available options (Raspberry Pi, conferencing speakers, Onju-voice and ESP32-S3 BOX) are getting better, but still noticeably lower quality and more expensive than alexa/google devices.

Which quality aspect is hurting the most for you? Reliable wake word? Range? Hardware build quality? Reliable command recognition?

Why I’m asking: with year of the voice where it stands, I’m wondering how much of the delta between HA on the one side, and google/alexa on the other side, is due to hardware, code, training data, algorithms, …
Depending on the ratio, the potential to close the gap might be anything between “totally doable” and “sorry, this is the maximum for a non-billionaire-run company”