Audio (microphone) Support for a Video Doorbell Integration

Mobile Apps Home Assistant Companion for Android
1

Hi, i have succesfuly integrated a DIY Doorbell in my Home Assistant like in this thread : “DIY Smart Doorbell with Android notifications - Share your Projects! - Home Assistant Community (home-assistant.io)”.

The Project uses the doorpi-card (same as doordroid-card: DoorDroid installation · rdehuyss/DoorDroid Wiki · GitHub but uses a SIP implementation rather then using WebRTC),

Both Projects works very well for me as far as i use normal Web Browsers to access the Lovelace Interface with these custem doorpi or doordroid cards. Audio and Video works without any Problem. But if i use the Home Assistant Companion App - then i receive the Question to allow the App the use of the microphone, which i confirm. After that i receive the following error Message : “check media device permissions on media and origin”.

Is there any Chance to use the Mikrophone in the Companion App for this ? That would be great.

Cheers Sascha

1 Like
2

Does the companion app have permission to access the microphone on your android device?

Also, are you saying you’ve gotten SIP calling TO a doorbell to work from the web browser?

3

this support was added recently and is found in the beta but not the production version

4

hi, is this already in production? or still beta

5

its in the beta still

6

aha, can you link me the PR? so i can follow up :slight_smile:

7

or can i also usbscribe to beta in android appstore to receive that new functionality

8

yea just sign up for the beta and you will receive the update :slight_smile:

1 Like
9

ok, seems asterisk/sip calling now works… are the any settings in beta for the mic/speaker?
ot is it just enabled without any settings?

10

no settings, the change was just to request proper permissions.

11

thnx m8

appreciated

1 Like
12

one more question, i am testing SIP with Companion app, seems with beta indeed audio/mic now works
Now testing video … seems the companion app allows VP8 video codec…
We are now testing with h264 codec, but seems then the calls fail if we call to the companion app… while if we use a chrome browser it works

my question; are there video codecs inserted in the companion app? like was VP8 added? is it possible to add more?

thnx

13

get the on device logs from App Configuration > Show and Share logs to see what the issue is then file an issue with the traceback.

14

thnx,

here you go, first call is with h264 , indeed in log its unsupported:

12-20 16:18:24.317 31654 31844 W VideoCapabilities: Unsupported mime image/vnd.android.heic
12-20 16:18:24.319 31654 31844 W VideoCapabilities: Unsupported mime video/divx
12-20 16:18:24.319 31654 31844 W VideoCapabilities: Unsupported mime video/divx4
12-20 16:18:24.321 31654 31844 W VideoCapabilities: Unrecognized profile/level 0/3 for video/mpeg2
12-20 16:18:24.321 31654 31844 W VideoCapabilities: Unrecognized profile/level 0/3 for video/mpeg2
12-20 16:18:24.322 31654 31844 W VideoCapabilities: Unsupported mime video/x-ms-wmv
12-20 16:18:24.387 31654 31774 I AudioManager: In isBluetoothScoOn(), calling application: io.homeassistant.companion.android
12-20 16:18:24.387 31654 31774 I AudioManager: In startbluetoothSco(), calling application: io.homeassistant.companion.android
12-20 16:18:24.396 31654 31774 I AudioManager: In isSpeakerphoneOn(), calling application: io.homeassistant.companion.android
12-20 16:18:24.400 31654 31774 I AudioManager: In setMode(), mode: 3, calling application: io.homeassistant.companion.android

second call is when i enable/force vp8 , it succeeds

12-20 16:19:23.298 31654 31774 I AudioManager: In isBluetoothScoOn(), calling application: io.homeassistant.companion.android
12-20 16:19:23.299 31654 31774 I AudioManager: In startbluetoothSco(), calling application: io.homeassistant.companion.android
12-20 16:19:23.312 31654 31774 I AudioManager: In isSpeakerphoneOn(), calling application: io.homeassistant.companion.android
12-20 16:19:23.326 31654 31774 I AudioManager: In setMode(), mode: 3, calling application: io.homeassistant.companion.android
12-20 16:19:23.386 31654 31774 I libOpenSLES: Emulating old channel mask behavior (ignoring positional mask 0x1, using default mask 0x10 based on channel count of 1)
12-20 16:19:23.404 31654 31774 I AudioRecord: createRecord_l(3114): AUDIO_INPUT_FLAG_FAST successful; frameCount 0 -> 4096
12-20 16:19:23.408 31654 31654 E chromium: [ERROR:web_contents_delegate.cc(228)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
12-20 16:19:23.508 31654 31774 I AAudio  : AAudioStreamBuilder_openStream() called ----------------------------------------
12-20 16:19:23.508 31654 31774 I AudioStreamBuilder: rate   =  48000, channels  = 2, format   = 5, sharing = SH, dir = OUTPUT
12-20 16:19:23.508 31654 31774 I AudioStreamBuilder: device =      0, sessionId = -1, perfMode = 12, callback: ON with frames = 192
12-20 16:19:23.508 31654 31774 I AudioStreamBuilder: usage  =      2, contentType = 0, inputPreset = 0, allowedCapturePolicy = 0
12-20 16:19:23.508 31654 31774 D         : PlayerBase::PlayerBase()
12-20 16:19:23.509 31654 31774 D AudioStreamTrack: open(), request notificationFrames = -8, frameCount = 0
12-20 16:19:23.514 31654 31774 I AudioTrack: createTrack_l(0): AUDIO_OUTPUT_FLAG_FAST successful; frameCount 0 -> 1536
12-20 16:19:23.514 31654 31774 D AudioTrack: Uid 10243 AudioTrack::setVolume left 1.000000 right 1.000000
12-20 16:19:23.514 31654 31774 W AudioStreamTrack: open() flags changed from 0x00000104 to 0x00000004
12-20 16:19:23.515 31654 31774 I AAudio  : AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#1 ----------------
12-20 16:19:23.516 31654 31654 E chromium: [ERROR:web_contents_delegate.cc(228)] WebContentsDelegate::CheckMediaAccessPermission: Not supported.
12-20 16:19:23.517 31654 31774 D AAudio  : AAudioStream_requestStart(s#1) called --------------
12-20 16:19:23.517 31654 31774 D AudioTrack: ClientUid 10243 AudioTrack::start 
12-20 16:19:23.525 31654 31774 D         : PlayerBase::start() from IPlayer
12-20 16:19:23.525 31654 31774 D AAudio  : AAudioStream_requestStart(s#1) returned 0 ---------
12-20 16:19:23.525 31654 31671 D AudioStreamLegacy: onAudioDeviceUpdate() devId 2 => 2
12-20 16:19:23.677 31654 31760 I cr_MediaCodecBridge: create MediaCodec video decoder, mime video/x-vnd.on2.vp8
12-20 16:19:23.677 31654 31760 D MediaCodec:  name = video/x-vnd.on2.vp8
12-20 16:19:23.685 31654 32447 I OMXClient: IOmx service obtained
12-20 16:19:23.705 31654 32446 I MediaCodec: MediaCodec will operate in async mode
12-20 16:19:23.707 31654 31760 W MediaCodec: RETYR:0
12-20 16:19:23.707 31654 32446 D SurfaceUtils: connecting to surface 0x7b072c7010, reason connectToSurface
12-20 16:19:23.707 31654 32446 I MediaCodec: [OMX.qcom.video.decoder.vp8] setting surface generation to 32413697
12-20 16:19:23.707 31654 32446 D SurfaceUtils: disconnecting from surface 0x7b072c7010, reason connectToSurface(reconnect)
12-20 16:19:23.708 31654 32446 D SurfaceUtils: connecting to surface 0x7b072c7010, reason connectToSurface(reconnect)
12-20 16:19:23.709 31654 32447 I ExtendedACodec: setupVideoDecoder()
15

what device is this and have you confirmed that android system webview is upto date?

16

i’m testing with a oneplus 6T, running android 10
system webview is up to date

if i test with other SIP clients, then video works with h264

17

those oneplus devices run lineage OS
i have also an samsung tablet, gonna test it, give me a sec

18

you need to use googles version of android system webview and not the one that ships with the custom ROM they are known to be incompatible.

19

hmm, i have “android system webview” installed, running version 96.0.4664.104

20

you need this one https://play.google.com/store/apps/details?id=com.google.android.webview&hl=en_US&gl=US

notice package name is slightly different.