How to log notify service calls or view notify service attributes (sms integration)

Configuration
1

Question
Is there a way of viewing a notify services’ attributes in the Template editor, or logging the actual data sent by a notify service call (e.g. attributes, message, etc).

Background
I’m facing a recent peculiar issue with one of my notify services and trying to identify the root cause.
I’m running HA Container v.2021.10.7 in docker on Ubuntu 18.04 and using an unlocked Huawei E220 USB modem to send and receive SMS with the GSM modem integration.

Phone numbers are stored in secrets.yaml

sms_usera: '+44xxxxxxxxxx'
sms_userb: '+44xxxxxxxxxx'
sms_homealarm: '+44xxxxxxxxxx'

and the secrets are used to configure the notify recipients

notify:
   # sms
   - name: sms_usera
     platform: sms
     recipient: !secret sms_usera
   - name: sms_userb
     platform: sms
     recipient: !secret sms_userb
   - name: sms_homealarm
     platform: sms
     recipient: !secret sms_homealarm
   - name: all_sms
     platform: group
     services:
       - service: sms_usera
       - service: sms_userb

and loaded into template sensors for incoming SMS matching

  sensor:
    - name: sms_usera
      unique_id: sms_usera
      state: !secret sms_usera
    - name: sms_userb
      unique_id: sms_userb
      state: !secret sms_userb
    - name: sms_homealarm
      unique_id: sms_homealarm
      state: !secret sms_homealarm

I have various automations that handle:

  • sms notifications (when no internet connection)
  • incoming sms from authorised senders to perform specific actions
  • storing incoming sms from unknown phone numbers
  • periodic automated SMS communications between HA and our house alarm panel’s GSMComm (basically to keep the house alarm’s pay as you go SIM active by occasionally sending and receiving SMS).

Problem
This setup has all worked without issue for some time, however SMS communications with the house alarm panel from HA no longer seem to generate the response.
Note that SMS communications between HA and usera and HA and userb still function correctly, and SMS communications between the house alarm panel and phones functions correctly. This seems to be purely an recent communication issue between HA and the alarm panel only.

Because this automation is basically a closed loop I need to find a way of checking actual message content and recipient data - my suspicion is that HA is not actually sending the required messsage text to the alarm panel or the phone number is not configured correctly when loaded into the notify recipient.

With the logger set to debug for homeassistant.components.sms.gateway some data is logged but nothing regarding actual notifications.

Any help appreciated.

1 Like
2

It is still not clear to me what the problem is. What do you mean by not generating a response?

3

Thanks for responding.

My apologies for not stating the issue clearly enough.

Our house alarm is equipped with a GSM COM module which allows communication and control by SMS.
When queried with usercode status it replies with SMS to the sender with a message containing various status elements; the usercode is a 4 digit number which I have stored in secrets.yaml as is the phone number for notify.sms_homealarm service recipient.

I have an “Comms Check” automation in place to query the status every 60 days - this ensures that the PAYG SIM in the alarm’s COM module remains active. This automation worked prior to 2021.7.x, but there have been numerous changes throughout HomeAssistant since then.

I have established that the following elements work as expected:

  • notify.sms_usera - sending and receiving SMS.
  • notify.sms_userb - sending and receiving SMS.
  • notify.all_sms - sending SMS.
  • GMS COM status request and response from my (usera) phone number.
  • the " status" message is generated correctly by the automation.

Because the “Comms Check” automation is a closed loop and the GSM COM module has no sent / received SMS logging I cannot currently validate:

  1. the number actually held by the recipient attribute of the notify.sms_homealarm service
  2. whether the SMS is actually received by the COM module

NB: I am confident that a return message is not missing as I log any SMS received by HomeAssistant regardless of the sender number.

For the moment I’m trying to establish that there isn’t some issue with the recipient phone number load, but cannot find:

  1. A way of actually viewing the “live” attributes of the service, e.g. within the Developer Tools Template Editor
  2. A way of setting logger to actually log sent message details including recipient number and message.
4

This is the part I simply do not understand, to me this is a sensor with a state equal to a phone number that never changes, what does this sensor do? what am I missing?

5

The sensors are used to match the sender of an incoming SMS, providing some HA actions by SMS.

I load them into template sensors so that I only have to update phone numbers in secrets.yaml, rather than having them hard-coded in multiple files.

6

I could still really use some insights on this.

I’ve confirmed from billing that the SMS is sent from HomeAssistant to the correct number for the alarm panel, however the panel continues to not respond.

This is the only status query from any phone number I use to not respond.
I intend to ask my alarm engineer to check the panel’s communication module on Wednesday during the annual service but would still really like to validate the notification service’s sent message formatting at point of send.

7

After having the alarm engineer check and reboot the com module on the panel, I’ve retested and still have this issue.

If I send the same SMS directly from the docker host command line using gammu it works and the response is received by HomeAssistant successfully.

Sending the SMS from within HA Container in docker obtains no response, although I’m billed for it by my provider.

I can only assume that the SMS is being sent but that it arrives in a format that the alarm panel ignores.

I cannot figure out how to send the SMS from the command line within the docker container and without any way of logging exactly what the notifier properties are I have no way to resolve this.

8

Had a brain wave this morning and retried the command line test using gammu from the docker host, adding the unicode parameter.
No response.

It looks as though the panel does not support receiving SMS in Unicode format, a change introduced in Support Unicode in SMS messages by ocalvo · Pull Request #56468 · home-assistant/core · GitHub.

I will confirm this by manually adjusting the unicode parameter at the weekend. I’ve also reached out to the alarm panel module manufacturers to confirm lack of unicode support even though their installation manual makes no mention of encoding.

If confirmed, would you be open to making unicode a configuration option @Oscar_Calvo ?

9

Confirmed. Reverting this change within the container instantly resolves this issue for me.

10

That shouldn’t make a difference, what’s your notification service look like? Are you using target and your ! secret number?

11 
notify:
   # sms
   - name: sms_homealarm
     platform: sms
     recipient: !secret sms_homealarm

With Unicode enabled the text is definitely sent from HA (they show on my provider account usage) and the expected response is not returned by the alarm panel (or sent based on that account usage details).

I believe the issue is in the receiving Texecom GSM COM Module message parsing.

It’s parsing the received SMS looking for a <usercode> <command> <options> pattern.
My automation sends <usercode> status to generate a response from the panel back to the sender.

With Unicode enabled it is not responding.
With Unicode disabled by manually reverting the change, and using the same automation to send the message the response SMS is generated by the panel and received by HA.

I’ve raised a ticket with Texecom technical support querying their support for Unicode but I’m not holding my breath for a timely response based on previous experience.

12

That’s still not the service I’m talking about.

When you call a service, you use a service call. In your example it would be notify.sms_homealarm. What does that look like when you call it. Thanks.

13

Apologies.

# run part one of home alarm sms comms check
# send status request and set flag to show waiting for response
# time condition is each saturday and more than 55 days 23 hours since last sent
- id: sms_alarm_comms_check
  alias: "SMS Alarm Comms Check"
  mode: single
  trigger:
    platform: time
    at: "10:30"
  condition:
    condition: and
    conditions:
      - condition: state
        entity_id: input_boolean.holiday_mode
        state: "off"
      - condition: time
        weekday:
          - sat
      - condition: template
        value_template: "{{ ((now().timestamp() - state_attr('input_datetime.last_alarm_comms_check', 'timestamp')) | int(0)) >= 4834800 }}"
  action:
    - variables:
        user_code: !secret alarm_usercode
    - service: notify.sms_homealarm
      data:
        message: "{{ user_code }} status"
    - service: logbook.log
      data:
        entity_id: automation.sms_alarm_comms_check
        name: "Requested Status from Home Alarm: "
        message: "successful"
        domain: automation
    - service: input_boolean.turn_on
      data:
        entity_id: input_boolean.alarm_comms_check
    - service: input_datetime.set_datetime
      data:
        entity_id: input_datetime.last_sms_sent
        time: "{{ now().time().strftime('%H:%M:%S') }}"
        date: "{{ now().date().strftime('%Y-%m-%d') }}"
14

Have you tried sending this information without the template through the service tester?

Can you provide an example user_code (not your real user code) using similar characters.

15

Yes.

I’ve been testing extensively since I first became aware this automation stopped receiving a response over a month ago, which was why I originally started this thread. The problem is not with the message being sent but on the receiving (alarm panel) end.

User code is simply a 4 digit PIN, e.g. 1972

16

can you give an example of the sms_recipient. I’m trying to confirm that your secrets aren’t biting you in the ass because unicode can behave differently depending on the characters that you have in your strings.

17

Fictional version (same format):

sms_homealarm: '+447708452742'

The SIM provider account usage shows the number the message was sent to which is correct.
FWIW the other SMS notifiers (to personal mobiles) are also stored in secrets.yaml in the same format and they continue to work without issue.

NB: when testing I also tried localised versions (for UK, replacing +44 with 0)

18

right but some characters in strings can cause issues with unicode. It’s just how things work. I’m trying to verify that it’s not biting you. Other notification platforms may not support unicode so it doesn’t affect that. Either way, that’s not the problem. You’ll have to write up an issue against the integration.

19

I appreciate the assist and the sanity checking.

I was planning to write up the issue tomorrow (I’ve a job interview in a couple of hours that I need to prepare for).
If the integration author is okay with adding a configuration option to enable / disable Unicode (probably defaulting to true) I suspect that would be the best way to resolve. In the meantime I can just continue to modify the code inside the container after any updates.
I might even be able to sort the issue at the Texecom end, although I can’t find any relevant documented configuration options.

For the moment I’m just happy to get the automation working again - it’s been driving me nuts.

20