Here’s an example run through of the task I’m using. It’s snappy, and the 8 seconds before the back button gives enough time for the assistant to finish speaking… most of the time.
I’m using the jarvis file from the apk. Downloading the file sounds like a good idea, I’ll have to try that.
My tasker looks similar but i dont click ‘stop listening’ which sounds like a good idea considering it will jsut keep listening to my kids babbling and yelling all the time. lol.
Yeah, that step really makes it much more usable imo. It works in both portrait and landscape because it’s searching for and clicking the text rather than a fixed coordinate, which is how I originally had it.
What would be great is if we could use tasker to monitor an mqtt property for tts end or something and hide the popup. I am getting weird stuff happening with the toaster popup sometimes not displaying (although i hear the ding from auto input) and i wonder if i have something misconfigured somewhere.
When you’re running this over the companion app, do you get a slight flicker at times where the header/task bar for android will display when tasker triggers things? Do you have anything set for the view to make things look different?
I’ve run into the same issue with the popup not displaying or only displaying partially. I don’t want to derail this thread with conversation about the ThinkSmart, as I believe that’s where the issue lies. Feel free to send me a PM, as I think I’ve found a fix.
I don’t have the android status bar hidden in any way, it’s always displaying, so I am not experiencing the flickering issue. I may have a solution for you, though.
Hotword Plugin, Auto Input, and Tasker all send/update notifications that show in the status bar next to the clock. Hiding these should do the trick.
Go into your notification settings, then silence and minimize the notifications for these apps. That will stop them from displaying in the status bar, which should prevent the status bar from flickering and reappearing.
Any chance you could post your xml for your task? I am not sure what is past your Stop listening click step and my automation is kinda acting weird.
Have a look
<TaskerData sr="" dvi="1" tv="6.2.22">
<Task sr="task6">
<cdate>1709656631301</cdate>
<edate>1711672721906</edate>
<id>6</id>
<nme>HAVA</nme>
<pri>6</pri>
<Action sr="act0" ve="7">
<code>20</code>
<App sr="arg0">
<appClass>io.homeassistant.companion.android.assist.AssistActivity</appClass>
<appPkg>io.homeassistant.companion.android</appPkg>
<label>Home Assistant:assist.AssistActivity</label>
</App>
<Str sr="arg1" ve="3"/>
<Int sr="arg2" val="0"/>
<Int sr="arg3" val="0"/>
</Action>
<Action sr="act1" ve="7">
<code>30</code>
<Int sr="arg0" val="0"/>
<Int sr="arg1" val="4"/>
<Int sr="arg2" val="0"/>
<Int sr="arg3" val="0"/>
<Int sr="arg4" val="0"/>
</Action>
<Action sr="act2" ve="7">
<code>107361459</code>
<Bundle sr="arg0">
<Vals sr="val">
<EnableDisableAccessibilityService><null></EnableDisableAccessibilityService>
<EnableDisableAccessibilityService-type>java.lang.String</EnableDisableAccessibilityService-type>
<Password><null></Password>
<Password-type>java.lang.String</Password-type>
<com.twofortyfouram.locale.intent.extra.BLURB>Actions To Perform: click(text,Stop listening)
Separator: ,
Check Millis: 1000</com.twofortyfouram.locale.intent.extra.BLURB>
<com.twofortyfouram.locale.intent.extra.BLURB-type>java.lang.String</com.twofortyfouram.locale.intent.extra.BLURB-type>
<net.dinglisch.android.tasker.JSON_ENCODED_KEYS>parameters</net.dinglisch.android.tasker.JSON_ENCODED_KEYS>
<net.dinglisch.android.tasker.JSON_ENCODED_KEYS-type>java.lang.String</net.dinglisch.android.tasker.JSON_ENCODED_KEYS-type>
<net.dinglisch.android.tasker.RELEVANT_VARIABLES><StringArray sr=""><_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0>%ailastbounds
Last Bounds
Bounds (left,top,right,bottom) of the item that the action last interacted with</_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES0><_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1>%ailastcoordinates
Last Coordinates
Center coordinates (x,y) of the item that the action last interacted with</_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES1><_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2>%err
Error Code
Only available if you select &lt;b&gt;Continue Task After Error&lt;/b&gt; and the action ends in error</_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES2><_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3>%errmsg
Error Message
Only available if you select &lt;b&gt;Continue Task After Error&lt;/b&gt; and the action ends in error</_array_net.dinglisch.android.tasker.RELEVANT_VARIABLES3></StringArray></net.dinglisch.android.tasker.RELEVANT_VARIABLES>
<net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>[Ljava.lang.String;</net.dinglisch.android.tasker.RELEVANT_VARIABLES-type>
<net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>parameters plugininstanceid plugintypeid </net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS>
<net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>java.lang.String</net.dinglisch.android.tasker.extras.VARIABLE_REPLACE_KEYS-type>
<net.dinglisch.android.tasker.subbundled>true</net.dinglisch.android.tasker.subbundled>
<net.dinglisch.android.tasker.subbundled-type>java.lang.Boolean</net.dinglisch.android.tasker.subbundled-type>
<parameters>{"_action":"click(text,Stop listening)","_additionalOptions":{"checkMs":"1000","separator":",","withCoordinates":false},"_whenToPerformAction":{"notInAutoInput":false,"notInTasker":false},"generatedValues":{}}</parameters>
<parameters-type>java.lang.String</parameters-type>
<plugininstanceid>0b945194-12e0-41c8-b976-1df435404d21</plugininstanceid>
<plugininstanceid-type>java.lang.String</plugininstanceid-type>
<plugintypeid>com.joaomgcd.autoinput.intent.IntentActionv2</plugintypeid>
<plugintypeid-type>java.lang.String</plugintypeid-type>
</Vals>
</Bundle>
<Str sr="arg1" ve="3">com.joaomgcd.autoinput</Str>
<Str sr="arg2" ve="3">com.joaomgcd.autoinput.activity.ActivityConfigActionv2</Str>
<Int sr="arg3" val="60"/>
<Int sr="arg4" val="1"/>
</Action>
<Action sr="act3" ve="7">
<code>30</code>
<Int sr="arg0" val="0"/>
<Int sr="arg1" val="8"/>
<Int sr="arg2" val="0"/>
<Int sr="arg3" val="0"/>
<Int sr="arg4" val="0"/>
</Action>
<Action sr="act4" ve="7">
<code>245</code>
</Action>
</Task>
</TaskerData>
1 Like
Much appreciated. I think what i was seeing was my autoinput action for Stop Listening was holding everything up because it couldnt find the stop listening button due to the fact that i was asking it short questions (it would just jump to Start Listening and tasker would just sit and wait). I added a timeout to the autoinput step and Continue on Error checked so it would jump to the next wait section before successfully hitting the back button.
I like your “Hey Computer” reccomendation. That seems to be working much better. “Hey Google” was also working well.
So, I’m using Automate and trying to figure out how to tell when Assist is done “talking”. I ended up using AutoVoice’s ability to listen for noise levels with some extra delay nodes around it to time out when it should be listening. When Assist stops talking it will fire the “back” interact node clearing the dialog window and allowing me to start over with a new command.
Does anyone else have a better implementation yet? I tried checking if media is playing, but I couldn’t seem to get that to work.
I should note I’m not normally an android user. This is all pretty fresh for me. Hoping someone here that is way smarter than I has figured some of this out.
This integration from AlexxIT is an absolute game-changer for me to use Assist !
Now I can have every camera and every phone/tablet to act as voice assistant.
I already configured it and works great, the only 2 small issues I have are how to increase the sensivity of camera’s michrphones and that if I select player entity my alexa media players, they do not work. If I select my wall tablet, it works.
1 Like
Hi folks, Running A11 on the thinksmart lenovo ex Teams device.
What is the preferred method for local wakeword detection?
Tasker/Automate + Wakeword plugin → ?
Do you need to hand off to StreamAssist or run HA app and hand off to the assistant.
My goal would be to use local wakeword, then capture the audio stream to send to my Home Server (with a GPU) for the Piper/Wyoming and even LocalAI stuff.
Does anyone have a good write up of the steps?
TIA
You can either do local wakeword detection with tasker/automate & hotword plugin to call the HA assist pipeline through the HA app. Or you can stream continuous audio to openwakeword on your HA server through streamassist.
The first method is described in the top post.
The second method can be found in more detail here: Home · dinki/View-Assist Wiki · GitHub
1 Like
Hello guys,
i’m trying to use this method Home · dinki/View-Assist Wiki · GitHub with an android phone as remote mic but it doesn’t work with openwakeword. If i open up the assist interface (only trough remote connection, not local because of https) and click on the microphone i can use the phone correctly. No way to get it work with openwakeword. Any ideas?
You may want to put questions about Dinki’s method in his thread: View Assist - Visual feedback for Assist voice assistant on an Android tablet (Install info provided on Wiki)
Or on his github page: dinki/View-Assist · Discussions · GitHub
But it sounds like you’re trying to mix his method with the HA companion app for android, which isn’t possible. His method bypasses the companion app and streams all audio continuously to the HA server for wakeword detection (using openwakeword) and STT processing.
If you want to use the HA companion app “assist” function, you’ll have to stick to the method described in the top post, which is currently incompatible with openwakeword, and uses snowboy for local wakeword detection instead.
1 Like
As @Endlessvoid rightly answered, you will not be able to use the companion app or Assist via web browser without manual interaction. You should be able to set up the device as a Wyoming satellite device with always listening microphone and wake word detect on the HA server. I have full instructions for that on the wiki as well as Youtube videos for all of those steps.
The View Assist system and controls is currently in beta testing and I hope to have full instructions for that out soon.
Thank you. I’ve just used rtpmic on my android phone, installed and configured openwakeword and streamassist on ha server but checking the services of streamassist, my voice is not recognized. I don’t know if it just doesn’t understand the wakeword or there’s something else not working. The STT sensor is in idle.
I don’t know if it’s relevant, but i’m using azure TTS and STT instead of the local piper.
This may be obvious, but make sure the mic is not muted. I have used whisper and HA Cloud for STT both successfully. I’m not sure about Azure STT. You may want to install whisper and piper and test to see if it is a problem with Azure.
Also try turning up the gain on rtpmic.
1 Like