Audio stream as notification sound / play on phone while screen is off?

Mobile Apps Home Assistant Companion for Android
1

Been using Home Assistant for a little over a year and have been loving it!

I built a custom Baby Monitor similar to the one in the guide, but have been trying to find a way to pass on the audio from the monitor camera to the phone as the notification sound (or, alternatively played even if the phone screen is off in addition to the notification sound) so that we can instantly hear it without having to open the companion app, go to the correct lovelace page, click on more info for the camera, and press play to get the audio.

Things I’ve done so far:

° Setup camera as RTSP stream into Home Assistant

° Use ffmpeg and automations to send a notification to the companion app when noise is detected

° Set the notification to be priority and alarm stream so it goes to the phone fast and reliability

Things I’ve tried so far:

° Set custom notification sound for that channel on my phone since Android app does not support custom notification sounds on the Home Assistant side (phone allows mp3 or other media file, but I cannot find a way to point to an audio stream and not a singular file)

° Use the notification commands to turn on the screen (although it does not get past the lockscreen even if the phone is unlocked via Smart Lock but needs to be swiped to get to the home screen) and open the app to the lovelace page for the monitor

° Use Browser Mod to expand the More Info card for the camera (which sometimes auto-plays but does not other times, I have been unable to figure out the right conditions to make it auto-play every time and have been unable to find a setting to always auto-play)

° Use Browser Mod to make the companion app a media player (able to play mp3 files either locally or online using the media_player.play_media service, but cannot get it to play an audio stream and it only plays if the phone screen is on and the app is open in the foreground)

° Use Browser Mod media player from the camera:play_stream service (works on other audio-only media players like Google Home Mini but does not work for the Browser Mod media players)

° Use “command broadcast intent” in the notification to get it to play on VLC on the phone (also seems to only work if the screen is on and phone is unlocked and have been unable to figure out the right intents to get it to play a stream)

Any help at all would be greatly appreciated! Thanks!

2

I got it to work! Thanks to /u/shakuyi on Reddit for their help as well as ChrisRosenkreuz23 on the discord!

In the end I used the code below to have it play the RTSP stream on VLC in the background in addition to a loud notification. I also have separate commands to set the media volume and then end playback when noise is no longer detected. NOTE: This plays the audio only in the background even if the phone is locked, just as I was looking for. If you need it to pull up a video on the screen this will only get you part of the way there.

    service: notify.mobile_app_<your phone>
    data:
      message: "command_activity"
      title: "rtsp://<your RTSP stream>"
      data:
        channel: "org.videolan.vlc"
        tag: "android.intent.action.VIEW"

Other helpful commands used in this project:

  1. Set Volume level for media playback (to make sure it is audible):
    service: notify.mobile_app_<your phone>
    data:
      message: "command_volume_level"
      title: 20
      data:
        channel: "music_stream"
  1. Stop playback and end VLC ongoing notification:
    service: notify.mobile_app_<your phone>
    data:
      message: "command_media"
      title: "stop"
      data:
        channel: "org.videolan.vlc"
  1. Turn on the screen so notifications are more likely to wake me up at night:
    service: notify.mobile_app_<your phone>
    data:
      message: "command_screen_on"
  1. General Notification with image from the camera which will take you to correct lovelace page in the app when selected (You can also use the custom channel to have a unique notification sound accompany the stream audio in your phone’s settings)
    service: notify.mobile_app_<your phone>
    data:
      message: Sound was detected in the Baby Room
      title: Baby Crying
      data:
        image: >-
          <your image uri as shown in Dev Tools for the camera under 'entity_picture'>
        ttl: 0
        priority: high
        channel: baby_monitor
        importance: max
        actions:
          - action: URI
            title: Open Monitor
            uri: /lovelace/baby
8 Likes
3

Is there a list of commands that someone can access to know what’s available?

4

Here’s the doc for Notification Commands. I also needed to know the format for the app i was trying to control with the command, in my case VLC, but there are other commands that would not require that.

1 Like
5

HI,

can you please explain me the use of this service call?

6 
service: notify.mobile_app_<your phone>
    data:
      message: "command_media"
      title: "stop"
      data:
        channel: "org.videolan.vlc"

That code is going to send a “Stop Media” (or Stop Playback) command to the VLC app on my phone. VLC has a persistent notification while it is playing something in the background, so once the playback is stopped, the notification goes away.

In my case, my automation waits for the noise binary sensor to go back to “Off” and then sends this command to end playback on my phone. If, however I’ve already stopped the media playback on my phone, I just get a notification that says “stop media”. It’s not the best solution, but it works for now.

1 Like
7

Can you help me set a general notification (if possibile) with lovelace tab and sound in the backgroud? I don’t know what custom channel is…

This is what I made so far:

alias: BabyMonitor Kiosk (Duplica)
description: ''
trigger:
  - entity_id:
      - sensor.foscam_1_motion
      - sensor.foscam_1_sound
    platform: state
    to: Detected
condition:
  - entity_id: input_boolean.operazioneadele
    state: 'on'
    condition: state
    enabled: true
action:
  - service: notify.mobile_app_mi_6
    data:
      message: command_volume_level
      title: 20
      data:
        channel: music_stream
  - service: notify.mobile_app_mi_6
    data:
      message: command_screen_on
  - service: notify.mobile_app_mi_6
    data:
      message: Sound was detected in the Baby Room
      title: Baby Crying
      data:
        image: >-
          /api/camera_proxy/camera.cameretta?token=xxxxxxx
        ttl: 0
        priority: high
        channel: baby_monitor
        importance: max
        actions:
          - action: URI
            title: Apri Baby Monitor
            uri: /lovelace-babymonitor/babymonitor
  - service: notify.mobile_app_mi_6
    data:
      message: command_activity
      title: rtsp://Foscam1:[email protected]:88/videoMain
      data:
        channel: org.videolan.vlc
        tag: android.intent.action.VIEW
8

I can certainly try!

service: notify.mobile_app_<your phone>
    data:
      message: Sound was detected in the Baby Room
      title: Baby Crying
      data:
        image: >-
          <your image uri as shown in Dev Tools for the camera under 'entity_picture'>
        ttl: 0
        priority: high
        channel: baby_monitor
        importance: max
        actions:
          - action: URI
            title: Open Monitor
            uri: /lovelace/baby

This is one part of the notification code I used. I’ll try to break it down so you can edit it as needed.

service: notify.mobile_app_<your phone>
    data:
      message: Sound was detected in the Baby Room
      title: Baby Crying

This part sends a notification to the phone. The notification that shows up has a title of “Baby Crying” which shows up even if the notification is not selected. The message, “Sound was detected in the Baby Room”, appears once you select the notification in the drop down of the phone.

     data:
        image: >-
          <your image uri as shown in Dev Tools for the camera under 'entity_picture'>

This includes a picture from the camera feed in the drop down notification along with the text above.

        ttl: 0
        priority: high
        channel: baby_monitor
        importance: max

These are all ways to get the notification to come to phone as fast as possible, and creates a new “notification channel” on the phone which allows me to set custom notifications sounds for only this type of event. For me, this was better than “alarm stream” since that is tied to the alarm noise on your phone. The notification settings in your phone for the home assistant app will allow you to edit the sounds and vibration for each notification type (channel) once it receives one of the type in question.

        actions:
          - action: URI
            title: Open Monitor
            uri: /lovelace/baby

This adds a button to the bottom of the notification, both dropped down and not, which can perform an action. In my case, it opens the app to the “/baby” lovelace page where my baby monitor info is.

Now, the notification itself does not open the app or take you to the correct page, but the button does when pressed.

I believe that you can send a command in the notification that would open the app to the correct page, but that only works if the phone is already unlocked and the screen is on.

As for playing sound in the background, that’s where the VLC notification comes in. I have an automation to send them back to back.

service: notify.mobile_app_<your phone>
    data:
      message: "command_activity"
      title: "rtsp://<your RTSP stream>"
      data:
        channel: "org.videolan.vlc"
        tag: "android.intent.action.VIEW"

This is the notification code I use to turn on the VLC app on my phone and have it open and play the RTSP stream from my camera. That plays in the background if the phone screen is off, or opens in the foreground in the phone is unlocked and the screen is on.

If you wanted it to play in the background and open the home assistant app to a certain lovelace page, assuming your phone was unlocked and the screen was on, you could use an automation to send the VLC command first, then the lovelace command next. That should open VLC for just a split second before opening the companion app to a certain lovelace page, while continuing to play the VLC audio in the background.

You could use the “command_webview” command to have it open the companion app to a specific lovelace page as outlined in this doc.

1 Like
9

Sorry for the second reply but I missed your edit before I sent the reply.

uri: /lovelace-babymonitor/babymonitor

It seems like this is not formatted correctly. I believe you need something like “/lovelace/babymonitor”, assuming that is the correct path, to get it to work. If it’s working for you, then great!

As for the second part, where it opens the companion app and plays VLC in the background, I think you only need to add the “command_webview” notification after the VLC command to have it work, again as long as the phone is already unlocked (I see that you are using the screen on command as well, but for me it didn’t work unless the phone was also unlocked).

Something like this:

action:
      - service: notify.mobile_app_<your_device_id_here>
        data:
          message: "command_webview"
          title: "/lovelace/babymonitor"
10

The full url is :
http://xxxxxxxxxx:8123/lovelace-babymonitor/babymonitor

and when I use it in the Action URI works:

actions:
  - action: URI
    title: Apri Baby Monitor
    uri: /lovelace-babymonitor/babymonitor

when I use it in the command_webview it doesn’t

service: notify.mobile_app_redmi_note_7
data:
  message: command_webview
  title: /lovelace-babymonitor/babymonitor

then you confirm me that device have to still unlocked in order to work? It is a big issue then…maybe can be bypassed using Fully Kiosk Browser but is a pity…

I checked also with another device and another title but it still does not work

service: notify.mobile_app_mi_6
data:
  message: command_webview
  title: /energy
11

This is an update using fully kiosk browser. Now I can start without having HA running already.

But the part related for VLC does not work. The app is not started.

  - type: turn_on
    device_id: c1ba36acb06195b7e4beb337f12add6f
    entity_id: light.mi_phone_screen
    domain: light
  - device_id: c1ba36acb06195b7e4beb337f12add6f
    domain: button
    entity_id: button.mi_phone_bring_to_foreground
    type: press
  - service: notify.mobile_app_mi_6
    data:
      message: command_activity
      title: rtsp://Foscam1:A%[email protected]:88/videoMain
      data:
        channel: org.videolan.vlc
        tag: android.intent.action.VIEW
  - device_id: c1ba36acb06195b7e4beb337f12add6f
    domain: button
    entity_id: button.mi_phone_load_start_url
    type: press

The automation switch on the screen, put Fully Kiosk Browser app on top, “try” to open VLC and then load HA dashboard.

12

I suspect that this is the problem:

title:  rtsp://Foscam1:A%[email protected]:88/videoMain

Does that path work if you open VLC and put it in as a stream? Mine is formatted quite differently, but it could be a difference in the way our cameras are setup.

Mine is this:

rtsp://192.168.120/live0

Maybe yours would work like this?

rtsp://192.168.2.114:88/videoMain

or this?

rtsp://192.168.2.114/videoMain
1 Like
13

Yes. RTSP stream has authentication. But if I rembember well then it asks me for user and pass once again. I will try to remove it.

EDIT: now it works. The only struggle is that home assistant app should be on…I need to find a way to bypass this. I saw that Fully Kiosk Browser should be able to load app…but I didn’t find how to do it…I already asked for support but no luck so far.

14

Thank you so much for all this information, it’s really been useful. One issue I’m having though, is I can’t get it to play a local file that is already stored on the Android device itself.

I’m pushing it via Node Red, but I believe the fields are interchangeable with Home Assistant automation.

The following (using a random test RTSP stream) works:

{
    "message": "command_activity",
    "title": "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4",
    "data": {
      "channel": "org.videolan.vlc",
      "tag": "android.intent.action.VIEW"
    }
}

But I can’t get it to use the local file.

{
    "message": "command_activity",
    "title": "/storage/emulated/0/Download/EmergencySirenAlarmSeriousHarsh.mp3",
    "data": {
      "channel": "org.videolan.vlc",
      "tag": "android.intent.action.VIEW"
    }
}

I’ve tried it with different formats, such as file:///. If go to VLC >> More >> New stream >> and paste in /storage/.../EmergencySirenAlarmSeriousHarsh.mp3, VLC is able to play the file straight away, but every time the Companion App reports Unable to send activity intent, please check command format.

I’ve been reading around (and saw your Reddit post!). The Android Player Intents page indicates that file:/// should work, but the example also shows the need for setDataAndTypeAndNormalize. I can’t see how to add it.

Any thoughts would be very much appreciated.

1 Like
15

No problem! I’m glad it’s helping someone!

Since you’re playing a single file that doesn’t change, could you just set it as a notification ring tone on the phone and then use the notification channels to call it when needed?

I believe there are apps out there (maybe even vlc) that allow you to set a file as an available ring tone. Then you’d just want to use a specific channel in your notification for this project, and once it’s used once you can set a specific sound for each notification channel in android.

16

Thanks for the thought pattern, very appreciated.

I’ll explain what I’m trying to do…

When I’m On-Call, I want my personal phone (Pixel 6) to alert me somehow, so in the middle of the night it’ll wake me up. In the previous hospital I worked at, I used my personal phone and it always woke me up, but the hospital I work at now have given a cheap Samsung Galaxy A12 which isn’t very loud and although it does wake me up, I don’t want to risk not getting the call if I’m fast asleep.

I’ve got it working (from another thread on here) so it sets a 1 second timer. This works, but for timers, you can’t set a tone for each timer (you can for alarms, but I can’t set the alarm to go off in 1 minute as the call will have ended by then!), it’s the one tone for all timers. And as I use timers regularly, I don’t want to change the tone as it’ll freak me out too often!

So my thought is to play a ringtone through VLC or any other media player.

Working Call Service node data on NodeRed for the Service mobile_app_my_pixel_6

{
    "message": "command_activity",
    "data": {
      "channel": "com.google.android.deskclock",
      "tag": "android.intent.action.SET_TIMER",
      "group": "android.intent.extra.alarm.SKIP_UI:true,android.intent.extra.alarm.MESSAGE:WORK PHONE RINGING!!!,android.intent.extra.alarm.LENGTH:1"
    }
}

I’m not sure I follow when you say set it as a notification ring tone, but it sounds interesting. Do you mean set it as a notification for an unused app, say Live Transcribe or something, then call that notification to fire?

Thanks again :slight_smile:

[edit] I’ve just seen Critical-notifications[/edit]

17

OK I think I understand now.

In your edit, alarm stream is one “channel” that android recognizes and plays your default alarm sound. That means that by changing the default alarm ring tone to be X, then sending a notification with the “channel: alarm-stream” command, your phone will use the same path that it uses for alarms for that notification. It will play the same sound, in this case X.

Alternatively, you could set that “channel” to be anything you’d like. Say “channel: work-call”. Now, once one of those types of notifications has been received by your phone, in the Android notification settings for the companion app, you can change the ring tone for that specific channel (again, work-call) to be your custom ring tone. This means that all other ring tones stay the same, but once the phone gets a notification in the companion app with that channel tag, it will play that sound.

2 Likes
18

Awesome, thank you!! I was over complicating it by trying to play an audio file.

I’ve got it working exactly as you described, and I’ll include it here for anyone else who might need it. It should be self-explanatory how to convert the fields to the Automation within HA, everything in the “data” if there is one, is what goes into the ‘data’ field at the bottom.

Guidance from Notifications Basic.

Set the ring volume to max in case it is low:

{
    "message": "command_volume_level",
    "title": "20",
    "data": {
      "channel": "ring_stream"
    }
}

Turn the screen on:

{
    "message": "command_screen_on"
}

Send the alert:

{
    "message": "Work phone is ringing",
    "title": "URGENT Work phone is ringing",
    "data": {
      "channel": "WorkPhoneRinging",
      "importance": "high",
      "ttl": "0",
      "timeout": "600",
      "visibility": "public"
    }
}

The first time it’s called, the notification will appear, probably without an alarm. Press and hold on the notification, then click the cog. Find and click the channel (it will probably be highlighted), then change the Sound to whatever ringtone is needed and finally enable Vibration and Override Do Not Disturb.

The second time it’s called, it should be loud.

The Node-RED flow is:

image
image893×127 14.2 KB 

[{"id":"892cd52e3adb74df","type":"tab","label":"Phones","disabled":false,"info":"","env":[]},{"id":"69bed461aedee7c0","type":"server-state-changed","z":"892cd52e3adb74df","name":"My Work Phone Ring","server":"a3a41549.e2ab08","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"sensor.my_sm_a127f_phone_state","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"ringing","halt_if_type":"str","halt_if_compare":"is","outputs":2,"output_only_on_state_change":true,"for":"0","forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"","valueType":"triggerId"}],"x":150,"y":120,"wires":[["797afb493254a905"],[]]},{"id":"764b10ec58c4e2ea","type":"inject","z":"892cd52e3adb74df","name":"Inject Ring Test","props":[{"p":"payload"},{"p":"text","v":"Test","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"mobile_app_my_pixel_6","payloadType":"str","x":180,"y":200,"wires":[["797afb493254a905"]]},{"id":"797afb493254a905","type":"api-call-service","z":"892cd52e3adb74df","name":"Volume Up","server":"a3a41549.e2ab08","version":5,"debugenabled":false,"domain":"notify","service":"{{payload}}","areaId":[],"deviceId":[],"entityId":[],"data":"{\t    \"message\": \"command_volume_level\",\t    \"title\": \"20\",\t    \"data\": {\t      \"channel\": \"ring_stream\"\t    }\t}\t","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"payload","valueType":"msg"},{"property":"text","propertyType":"msg","value":"text","valueType":"msg"}],"queue":"none","x":410,"y":160,"wires":[["8d0f1c6887c306d4"]]},{"id":"8d0f1c6887c306d4","type":"api-call-service","z":"892cd52e3adb74df","name":"Screen on","server":"a3a41549.e2ab08","version":5,"debugenabled":false,"domain":"notify","service":"{{payload}}","areaId":[],"deviceId":[],"entityId":[],"data":"{\t    \"message\": \"command_screen_on\"\t}\t","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"payload","valueType":"msg"},{"property":"text","propertyType":"msg","value":"text","valueType":"msg"}],"queue":"none","x":590,"y":140,"wires":[["575d7d9be0fd8795"]]},{"id":"575d7d9be0fd8795","type":"api-call-service","z":"892cd52e3adb74df","name":"Alert","server":"a3a41549.e2ab08","version":5,"debugenabled":false,"domain":"notify","service":"{{payload}}","areaId":[],"deviceId":[],"entityId":[],"data":"{\t    \"message\": \"Work phone is ringing\",\t    \"title\": \"URGENT Work phone is ringing\",\t    \"data\": {\t      \"channel\": \"WorkPhoneRinging\",\t      \"importance\": \"high\",\t      \"timeout\": \"600\",\t      \"visibility\": \"public\"\t    }\t}","dataType":"jsonata","mergeContext":"","mustacheAltTags":false,"outputProperties":[],"queue":"none","x":750,"y":120,"wires":[[]]},{"id":"a3a41549.e2ab08","type":"server","name":"Home Assistant","version":2,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true,"heartbeat":false,"heartbeatInterval":30}]
1 Like
19

I’m glad you got it working! Congrats! Happy to help!

20

hello,

it doesnt work anymore, something as changed, i read the docs and channel is now media_stream but i can find any other option like you give, as importance, ttl, or timeout and visibility…

have you change your code or it still work @Aubs please?

my phone doesnt ring at all…