WIP - DoorDroid - Smart Doorbell with Lovelace integration, video and 2-way audio, without cloud

greengolfer · August 29, 2019, 7:41pm

Hello,
I am more or less in @jiiins situation. I have a built-in SIP doorbell (GDS3710 grandstream). I am therefore interested on the client side. Can you detail what you have installed on the tablet to receive the call? I assume you are using what is there https://github.com/rdehuyss/DoorDroid but I don’t understand what to do.
It seems you are using the jssip library. So, in theory, it should be in the browser of the android phone? Am I correct?
Thanks for the hints and sharing your project.
GV

rdehuyss · August 29, 2019, 7:58pm

So, my indoor station (an in-wall Android Tablet) is running standard Lovelace in Chrome (= browser of the Android phone in your words). I created a custom lovelace component which was called by the DoorDroid application (in your case the GDS3710 grandstream). It indeed used jssip and asterisk.

Have you taken a look at the wiki? Everything is described there…

greengolfer · August 29, 2019, 8:07pm

I did!! But " * Download and install the Android app from the releases page of this repository" from the wiki is not clear to me.
What do you need to install on the android phone? I need (somehow?) to build an apk?
In my understanding with the sipjs library everything was in the browser from the client side. So, in theory, nothing to install on the phone/tablet… Or at least that was my understanding. Hence, my confusion
Thanks
GV

nickrout · August 29, 2019, 8:10pm

The apk is on the releases page.

rdehuyss · August 29, 2019, 8:15pm

In your case you don’t need to install an apk since your doorbell already supports SIP.

You need to:

setup asterisk for SIP Call Management (as in WIKI)
configure your GDS3710 grandstream station to call the SIP account you will use at your indoor station
Add the lovelace card to your home assistant
That should be it, I think

greengolfer · August 29, 2019, 8:17pm

Missed that… Sorry! Nevertheless, the question on sipjs remains. In theory, it should be possible to run everything in the browser, and not requiring an apk.
We had an exchange on this page https://community.home-assistant.io/t/sip-client/60964.
GV

greengolfer · August 29, 2019, 8:20pm

Would be great!! I’ll try that. It is going to keep me busy this week-end
In your setup, why do you need the apk? What is the purpose of the app?
GV

rdehuyss · August 29, 2019, 8:22pm

The APK is running on a phone which is mounted at my door, so the equivalent of your GDS3710 grandstream.

greengolfer · August 29, 2019, 8:29pm

OK. Thanks. Will try that and if I have some issues I’ll ask
GV

greengolfer · August 29, 2019, 9:54pm

I don’t know if I should report here or on github…
Line 36 of doorpi-camera-view.js is :

imgEl.src = 'https://192.168.1.66:9090/stream/video.mjpeg';

It should be a reference to the settings in doordroid.yaml. The mjpeg settings:

mjpeg_url: https://192.168.1.66:9090/stream/video.mjpeg

No?
GV

greengolfer · August 31, 2019, 12:29pm

Hello,
Some report on my progress so far. I have installed doordroid and jssip on my HA box. I have created the extension on my asterisk serevr to register by able to register the “card” of doordroid.
Up to that point all good.
When I open lovelace and go to the view with the webrtc phone I can see the phone registering on asterisk.
When I call the webrtc extension, the two additional buttons are displayed. However when I click on “accept call” buttons I have an error message in the developper console. See below.

Loading SIPPhone                                 doordroid-card.js?v=0.0.1:80 
initialized camera view                               doordroid-card.js?v=0.0.1:151: 
SIPPhone registered with SIP Server             doordroid-card.js?v=0.0.1:100: 

jssip-3.3.6.min.js:9 Uncaught INVALID_STATE_ERROR: Invalid status: 5
    at t.value (http://192.168.0.150:8123/local/plugins/jssip-3.3.6.min.js:9:117457)
    at HTMLElement.<anonymous> (http://192.168.0.150:8123/local/plugins/doordroid-card.js?v=0.0.1:131:71)
r @ jssip-3.3.6.min.js:9
t @ jssip-3.3.6.min.js:9
value @ jssip-3.3.6.min.js:9
(anonymous) @ doordroid-card.js?v=0.0.1:131
doordroid-card.js?v=0.0.1:121

And I am stuck!! Any suggestion where to look at and what to try?

Thanks,
GV

rdehuyss · August 31, 2019, 6:12pm

To be honest, I don’t use this setup anymore as mentioned above. So, it may be that the code contains bugs/errors.

I did Google your error and found this: https://github.com/versatica/JsSIP/issues/527

Are you answering the call twice?

greengolfer · September 2, 2019, 7:38am

Hello,
Thanks for your answer.
In fact, my “SSL” setup (for hassio, asterisk,…) was not correct and webrtc seems to be very picky when it comes to connection over unsecured channel. So, once HTTPS, WSS was ok, it when well.
In your doordroid-card.js I have added:

        JsSIP.debug.enable('JsSIP:*');

which helped greatly!

I believe that my configuration is OK (SIP extensions, asterisk,…) the only missing bit is incoming audio from the SIP contact. And I believe, I understand why, even if I can’t see what to do…

In my setup the SIP extensions calls the webrtc endpoint. I can accept the call and it can be seen in the debug console:

JsSIP:RTCSession receiveRequest() +14ms
jssip-3.3.6.min.js:9 JsSIP:RTCSession session confirmed +1ms
jssip-3.3.6.min.js:9 JsSIP:RTCSession emit "confirmed" +0ms
doordroid-card.js?v=0.0.4:124 call confirmed
jssip-3.3.6.min.js:9 JsSIP:WebSocketInterface received WebSocket message +13s

However, looking at the code, the next event should be:

             session.on("addstream", (e) => {

the event addstream AFAIK doesn’t exist any more in the JSSIP 3. It was part of the API in 2.X.

With my (extremely) limited understanding of JS and JSSIP, I can’t figure out what should be the final (?) step to “link” the RTP audio (seen in the wireshark capture) with the audio of my PC.

Any suggestion?

Thanks,

GV

rdehuyss · September 2, 2019, 10:42am

You are correct.

I updated my github with my latest version of the DoorDroid Card.

And indeed, without https, webrtc refuses to work except for localhost.

greengolfer · September 2, 2019, 11:36am

Thanks !!
Will try that when I am back home.
If I can achieve what I want I will “Share my project” with a lot of credit to you
Next (last ?) episode tonight
GV

greengolfer · September 3, 2019, 7:40am

Hourrah !! It works. Thank you for your help.
I have (kind of) took some bits of the new version provided yesterday with the previous one. I don’t need the part with the “everywhere” or not
After this first success, I have continued my investigations. When the call is accepted I would like to turn on an input_boolean in hass to do further action.
I thought it would be straightforward. It is not. At least for me.
Here is what I have done:

                session.on("peerconnection", () => {
                    session.connection.addEventListener("addstream", (e) => {
                        console.log('adding audiostream')
                        // set remote audio stream (to listen to remote audio)
                        // remoteAudio is <audio> element on page
                        const remoteAudio = document.createElement('audio');
                        //const remoteAudio = droidCard.getElementById('audio-player');
                        remoteAudio.srcObject = e.stream;
                        remoteAudio.play();
                        hass.callservice('input_boolean', 'turn_on', { entity_id: 'input_boolean.portail' });
                    })

The callservice at the end of the function…

I can’t see the change in lovelace. The input_boolean.portail stays off
Is there something else to do?

How do I debug this? I don’t even know in the developer console if the callservice is called or not.

GV

rdehuyss · September 3, 2019, 8:28pm

Have you tried debugging it? Just place a breakpoint and see if it is hit. I’m also not sure whether hass is known as you are in a callback.

Debugging it will tell you…

greengolfer · September 5, 2019, 5:57pm

I was able to “fix” my code. In another post https://community.home-assistant.io/t/sip-client/60964 I am giving a status.
Thanks for your help and providing 99% of the work for making this possible
GV

greengolfer · September 24, 2019, 3:52pm

Hello,
I still have some issues with my setup and I was wondering if you could help!
My (tweaked) version of doodroid.js is running in a view that is not always displayed. Sometimes the tablet falls asleep. As a consequence the webrtc connexion is dropped.

However, by looking at the code I can see:

set hass(hass) {
        if(this.notYetInitialized()) {
            this.initJsSIPIfNecessary(hass);
            this.initCameraView(hass);
        } else if(this.cameraEntityHasNewAccessToken(hass)) {
            this.updateCameraView(hass);
        }
        this.initDoorbellRinging(hass)
    }

and below:

    notYetInitialized() {
        return window.JsSIP && !this.sipPhone && this.config;
    }

My understanding of javascript gives me the feeling that when the tablet is back on and the doordroid view is displayed, the webrtc connexion is not restarted as it has been Initialized once.

To be on the safe side, wouldn’t it be better to “initialize” again the webrtc phone when the doordroid view is displayed? What would be the best way to do that?

As explained above, JS and I are not the best friends. So, I may miss something here!!

Thanks.

GV

rdehuyss · September 24, 2019, 6:55pm

My tablet was always on so I did not have the problem.

I checked the docs of JsSip and found this: https://jssip.net/documentation/3.3.x/api/ua/

You could check if this.sipPhone.isConnected(). So, it would be something like:

   notYetInitialized() {
        return window.JsSIP  && this.config && !this.sipPhone && !this.sipPhone.isConnected();
    }

Do note that I did not test this so you will need to investigate a bit (and my head can’t process that much booleans right now) .