Mobile app TTS notification queueing

adrien.b · May 6, 2024, 7:25pm

Hello, I’m facing an issue with the TTS notifications in the mobile app (latest version, but it was the case with previous versions too) when sending multiple TTS notifications to the same device either in a single automation or from multiple automations running in parallel.

I would expect HA to queue the notifications sent to the app, so they are all played in the order of appearance in the list, but it doesn’t. It looks like if a notification is sent while there is an ongoing notification text spoken, the new one is ignored.

Example automation:

alias: Test multiple TTS
description: ""
trigger: []
condition: []
action:
  - service: notify.mobile_app_pixel_6_pro
    data:
      message: TTS
      data:
        priority: high
        ttl: 0
        tts_text: 1, 2, 3, 4, 5, 6, 7, 8
    enabled: true
  - service: notify.mobile_app_pixel_6_pro
    data:
      message: TTS
      data:
        priority: high
        ttl: 0
        tts_text: 9, 10
    enabled: true
mode: single

The second ”9, 10" text is ignored. I would expect it to be queued and played after the “8”.

How to ensure no notification is lost ?
Thanks

dshokouhi · May 6, 2024, 7:39pm

we do indeed add to the queue

github.com

home-assistant/android/blob/3ca3ab240bc171c2132d91076c20f9e2bd8497c7/common/src/main/java/io/homeassistant/companion/android/common/util/TextToSpeech.kt#L96


      
                  }

              }

              textToSpeech?.setOnUtteranceProgressListener(listener)

              if (data[NotificationData.MEDIA_STREAM] in NotificationData.ALARM_STREAMS) {

                  val audioAttributes = AudioAttributes.Builder()

                      .setContentType(AudioAttributes.CONTENT_TYPE_SONIFICATION)

                      .setUsage(AudioAttributes.USAGE_ALARM)

                      .build()

                  textToSpeech?.setAudioAttributes(audioAttributes)

              }

              textToSpeech?.speak(tts, TextToSpeech.QUEUE_ADD, null, "")

              Log.d(TAG, "speaking text")

          } else {

              Handler(Looper.getMainLooper()).post {

                  Toast.makeText(

                      context,

                      context.getString(R.string.tts_error, tts),

                      Toast.LENGTH_LONG

                  ).show()

              }

          }

check notification history to see if they were received in a different order?

settings > companion app > notification history

adrien.b · May 6, 2024, 7:58pm

I checked the history, it correctly received both notifications, but only played the first one.

dshokouhi · May 6, 2024, 8:53pm

any errors in the companion app logs when the issue occurs?

adrien.b · May 6, 2024, 9:13pm

It logs the two calls to TTS service, but that’s all.

dshokouhi · May 6, 2024, 9:23pm

hmm so just tested ths out, seems like in some cases the queue may be lost if things are not sent fast enough because we are required to shutdown TTS once done. In some cases I can see an item in queue getting lost, if I wait 1 second its fine, if I send multiple very quickly those also mostly get queued up. Not really sure we can detect if teh TTS queue is full will need to check. Feel free to submit an issue on github in the meantime.

adrien.b · May 6, 2024, 9:43pm

github.com/home-assistant/android

Mobile app multiple TTS notifications queueing issue

opened 09:43PM - 06 May 24 UTC

ScratMan

bug

**Home Assistant Android app version(s):** 2024.4.1-full **Android version(s…):** 14 **Device model(s):** Google Pixel 6 Pro **Home Assistant version:** 2024.5.1 **Last working Home Assistant release (if known):** **Description of problem, include YAML if issue is related to notifications:** Hello, I'm facing an issue with the TTS notifications in the mobile app (latest version, but it was the case with previous versions too) when sending multiple TTS notifications to the same device either in a single automation or from multiple automations running in parallel. I would expect HA to queue the notifications sent to the app, so they are all played in the order of appearance in the list, but it doesn't. It looks like if a notification is sent while there is an ongoing notification text spoken, the new one is ignored. Example automation: ```yaml alias: Test multiple TTS description: "" trigger: [] condition: [] action: - service: notify.mobile_app_pixel_6_pro data: message: TTS data: priority: high ttl: 0 tts_text: 1, 2, 3, 4, 5, 6, 7, 8 enabled: true - service: notify.mobile_app_pixel_6_pro data: message: TTS data: priority: high ttl: 0 tts_text: 9, 10 enabled: true mode: single ``` The second ”9, 10" text is ignored. I would expect it to be queued and played after the "8". How to ensure no notification is lost ? Thanks **Companion App Logs:** ``` --------- beginning of main 05-06 23:35:10.642 24168 24168 D IntegrationRepository: isAppLocked(): false. (LockEnabled: false, appActive: false, expireMillis: 1715031222110, currentMillis: 1715031310642) 05-06 23:35:10.660 24168 24168 D VRI[SettingsActivity]: Start draw after previous draw not visible 05-06 23:35:10.664 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.664 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.664 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.665 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:10.691 24168 24168 D IntegrationRepository: isAppLocked(): false. (LockEnabled: false, appActive: false, expireMillis: 1715031222110, currentMillis: 1715031310691) 05-06 23:35:10.691 24168 24168 D IntegrationRepository: setAppActive(): true 05-06 23:35:12.873 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:12.983 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:13.140 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:13.269 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:13.432 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:13.518 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:14.416 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:14.534 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:14.942 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:15.026 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:15.626 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:15.718 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:16.285 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:16.344 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:16.411 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:16.837 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:16.902 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:17.026 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:17.321 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:17.471 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:17.721 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:17.812 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:17.938 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.037 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.122 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.412 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.453 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.487 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.529 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.587 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:18.648 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.364 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.390 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.414 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.448 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.464 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.480 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.498 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.513 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.530 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.556 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.573 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.613 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.632 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.655 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.673 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.706 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.724 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.755 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.773 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.797 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.823 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.855 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.881 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:20.906 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:21.415 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:21.891 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:22.024 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:22.131 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:22.291 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:22.701 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:23.538 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:23.538 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:23.538 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:23.538 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:26.605 24168 10140 D LogcatReader: Read logcat for pid 24168 05-06 23:35:26.667 24168 10140 D LogcatReader: Done reading logcat for pid 24168 05-06 23:35:29.640 24168 24168 D LocBroadcastReceiver: Received location update. 05-06 23:35:29.646 24168 24168 D ForegrndServiceLauncher: Check if service HighAccuracyLocationService is running. Service running = false 05-06 23:35:29.646 24168 10140 D LocBroadcastReceiver: Last Location: 05-06 23:35:29.646 24168 10140 D LocBroadcastReceiver: Coords:(45.3266959, 5.5503222) 05-06 23:35:29.646 24168 10140 D LocBroadcastReceiver: Accuracy: 100.0 05-06 23:35:29.646 24168 10140 D LocBroadcastReceiver: Bearing: 0.0 05-06 23:35:29.646 24168 10140 D LocBroadcastReceiver: Begin evaluating if location update should be skipped 05-06 23:35:29.647 24168 10140 D LocBroadcastReceiver: Received location that is 39 milliseconds old, 1715031329607 compared to 1715031329646 with source fused 05-06 23:35:29.647 24168 10140 D LocBroadcastReceiver: Duplicate location received, not sending to HA 05-06 23:35:47.816 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:47.816 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:47.816 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:47.816 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:47.817 24168 24168 I ImeTracker: io.homeassistant.companion.android:88ceda40: onRequestHide at ORIGIN_CLIENT_HIDE_SOFT_INPUT reason HIDE_SOFT_INPUT_CLOSE_CURRENT_SESSION 05-06 23:35:54.225 24168 24168 W WindowOnBackDispatcher: sendCancelIfRunning: isInProgress=falsecallback=androidx.activity.OnBackPressedDispatcher$Api34Impl$createOnBackAnimationCallback$1@d7c3539 05-06 23:35:54.250 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.250 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.250 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.250 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.250 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:54.251 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:35:58.821 24168 24168 D IntegrationRepository: setAppActive(): false 05-06 23:35:58.821 24168 24168 D IntegrationRepository: setAppActive(): false 05-06 23:35:59.195 24168 24168 D VRI[SettingsActivity]: visibilityChanged oldVisibility=true newVisibility=false 05-06 23:35:59.214 24168 24168 D VRI[SettingsActivity]: Not drawing due to not visible 05-06 23:36:29.675 24168 24168 D LocBroadcastReceiver: Received location update. 05-06 23:36:29.679 24168 24168 D ForegrndServiceLauncher: Check if service HighAccuracyLocationService is running. Service running = false 05-06 23:36:29.679 24168 10140 D LocBroadcastReceiver: Last Location: 05-06 23:36:29.679 24168 10140 D LocBroadcastReceiver: Coords:(45.3266959, 5.5503222) 05-06 23:36:29.679 24168 10140 D LocBroadcastReceiver: Accuracy: 100.0 05-06 23:36:29.679 24168 10140 D LocBroadcastReceiver: Bearing: 0.0 05-06 23:36:29.680 24168 10140 D LocBroadcastReceiver: Begin evaluating if location update should be skipped 05-06 23:36:29.680 24168 10140 D LocBroadcastReceiver: Received location that is 71 milliseconds old, 1715031389609 compared to 1715031389680 with source fused 05-06 23:36:29.680 24168 10140 D LocBroadcastReceiver: Duplicate location received, not sending to HA 05-06 23:36:43.887 24168 24168 D SensorReceiver: Received intent: android.media.VOLUME_CHANGED_ACTION 05-06 23:36:43.890 24168 24168 D SensorReceiver: Sensor(s) [volume_accessibility, volume_alarm, volume_call, volume_dtmf, volume_notification, volume_music, volume_ring, volume_system] corresponding to received event android.media.VOLUME_CHANGED_ACTION are disabled, skipping sensors update 05-06 23:36:45.960 24168 24168 D SensorReceiver: Received intent: android.media.VOLUME_CHANGED_ACTION 05-06 23:36:45.962 24168 24168 D SensorReceiver: Sensor(s) [volume_accessibility, volume_alarm, volume_call, volume_dtmf, volume_notification, volume_music, volume_ring, volume_system] corresponding to received event android.media.VOLUME_CHANGED_ACTION are disabled, skipping sensors update 05-06 23:36:46.910 24168 24168 D SensorReceiver: Received intent: android.media.VOLUME_CHANGED_ACTION 05-06 23:36:46.912 24168 24168 D SensorReceiver: Sensor(s) [volume_accessibility, volume_alarm, volume_call, volume_dtmf, volume_notification, volume_music, volume_ring, volume_system] corresponding to received event android.media.VOLUME_CHANGED_ACTION are disabled, skipping sensors update 05-06 23:36:47.481 24168 24168 D SensorReceiver: Received intent: android.media.VOLUME_CHANGED_ACTION 05-06 23:36:47.483 24168 24168 D SensorReceiver: Sensor(s) [volume_accessibility, volume_alarm, volume_call, volume_dtmf, volume_notification, volume_music, volume_ring, volume_system] corresponding to received event android.media.VOLUME_CHANGED_ACTION are disabled, skipping sensors update 05-06 23:37:04.700 24168 19607 W FirebaseMessaging: Unable to log event: analytics library is missing 05-06 23:37:04.700 24168 19607 D FCMService: From: 331041709873 05-06 23:37:04.703 24168 24168 D TextToSpeech: Sending text to TTS 05-06 23:37:04.718 24168 24168 I TextToSpeech: Sucessfully bound to com.google.android.tts 05-06 23:37:04.720 24168 31605 I TextToSpeech: Connected to TTS engine 05-06 23:37:04.752 24168 25859 I TextToSpeech: Setting up the connection to TTS engine... 05-06 23:37:04.754 24168 24168 D TextToSpeech: speaking text 05-06 23:37:04.903 24168 19607 W FirebaseMessaging: Unable to log event: analytics library is missing 05-06 23:37:04.903 24168 19607 D FCMService: From: 331041709873 05-06 23:37:04.904 24168 24168 D TextToSpeech: Sending text to TTS 05-06 23:37:04.908 24168 24168 I TextToSpeech: Sucessfully bound to com.google.android.tts 05-06 23:37:04.913 24168 17583 I TextToSpeech: Connected to TTS engine 05-06 23:37:04.915 24168 25859 I TextToSpeech: Setting up the connection to TTS engine... 05-06 23:37:04.917 24168 24168 D TextToSpeech: speaking text 05-06 23:37:08.130 24168 6986 I TextToSpeech: Disconnected from TTS engine 05-06 23:37:12.166 24168 24168 D IntegrationRepository: isAppLocked(): false. (LockEnabled: false, appActive: false, expireMillis: 1715031359821, currentMillis: 1715031432166) 05-06 23:37:12.194 24168 24168 D VRI[SettingsActivity]: Start draw after previous draw not visible 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.205 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:12.233 24168 24168 D IntegrationRepository: isAppLocked(): false. (LockEnabled: false, appActive: false, expireMillis: 1715031359821, currentMillis: 1715031432233) 05-06 23:37:12.233 24168 24168 D IntegrationRepository: setAppActive(): true 05-06 23:37:13.449 24168 24168 W TextView: onProvideContentCaptureStructure(): calling assumeLayout() 05-06 23:37:13.449 24168 24168 W TextView: onProvideContentCaptureStructure(): calling assumeLayout() 05-06 23:37:15.146 24168 24168 W WindowOnBackDispatcher: sendCancelIfRunning: isInProgress=falsecallback=android.app.Dialog$$ExternalSyntheticLambda2@a896525 05-06 23:37:15.199 24168 26284 D HWUI : endAllActiveAnimators on 0xb400007842a5cc10 (RippleDrawable) with handle 0xb4000077528ab2d0 05-06 23:37:15.227 24168 24168 D IntegrationRepository: isAppLocked(): false. (LockEnabled: false, appActive: true, expireMillis: 1715031359821, currentMillis: 1715031435227) 05-06 23:37:15.227 24168 24168 D IntegrationRepository: setAppActive(): true 05-06 23:37:16.378 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:16.379 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:16.379 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:16.379 24168 24168 E mpanion.android: Invalid resource ID 0x00000000. 05-06 23:37:17.676 24168 10140 D LogcatReader: Read logcat for pid 24168 05-06 23:37:17.728 24168 10140 D LogcatReader: Done reading logcat for pid 24168 05-06 23:37:18.654 24168 10140 D LogcatReader: Read logcat for pid 24168 ``` **Screenshot or video of problem:** **Additional information:** Discussed here also: https://community.home-assistant.io/t/mobile-app-tts-notification-queueing/726312/1

Done