Help creating a template sensor for IMAP

Jakesa · November 2, 2023, 10:21am

Hi, I have the Yale alarm integration and annoyingly, the alarm triggered entity is not exposed by the API, and I want to create an automation triggered by the alarm. However, I do receive an email each time the alarm is triggered, and I was hoping I could use a template sensor to start my automation.

Unfortunately, I have no experience with templates and every time I attempt to get somewhere with it, I get bamboozled (it doesn’t take much for my brain). Is anyone able to guide me through it, please? I am happy to repay the favour in coffee/beer.

My email:

logo-home-notification
Yale Smart Living Alarm Notification

Dear *Redacted*,

An event has been detected by your alarm:

2023-11-02 04:25:46 Area:1 Device Name:Garage PIR Burglar from Account:7054598

Ideally, I would like to identify which sensor has been triggered so I can set TTS notifications of where I need to run to. Still, if that isn’t possible, a notification the alarm has been triggered to activate lights will suffice.

As you can see from the email timestamp, the alarm was triggered early hours this morning, and I was left running around the house, in the pitch black, in my pants with no idea what sensor had been triggered.

Didgeridrew · November 2, 2023, 1:31pm

template:
  - trigger:
      - platform: event
        event_type: "imap_content"
        event_data:
          sender: [email protected]
    sensor:
      - name: Yale Alarm Event
        state: |
          {{ trigger.event.data['text'] 
          | regex_findall(find='(?<=Device Name:).*?(?=\sfrom Account:)', ignorecase=False)| first }}
        attributes:
          timestamp:  "{{ trigger.event.data['date'] }}"

Jakesa · November 2, 2023, 9:40pm

Thank you. Much appreciated

Jakesa · November 3, 2023, 9:56am

Hi, I made some automations from this, but it is acting a bit strange and I can’t work out why.

So from the original email already provided, the template gives the state Garage PIR Burglar from" However, I tried some different PIR’s and was expecting the state to change to Living Room PIR Burglar from" instead, all I got was "

Here is the notification email for the living room PIR:

logo-home-notification
Yale Smart Living Alarm Notification

Dear **Redacted**,

An event has been detected by your alarm:

2023-11-03 09:28:35 Area:1 Device Name:Living Room PIR Burglar (Interior) from Account:7054598

Didgeridrew · November 3, 2023, 2:03pm

It looks like I inadvertently left a trailing quotation mark on the template, that’s the source of the ". But that doesn’t explain the lack of the device name.

When you say “… from the original email already provided, the template gives the state Garage PIR Burglar…” do you mean the actual sensor produces that output or just the Template editor tool? If it’s not the actual sensor it’s possible that the emails are encoded. For testing you could setup a second sensor to pull the message body as both plaintext and decoded:

template:
  - trigger:
      - platform: event
        event_type: "imap_content"
        event_data:
          sender: [email protected]
    sensor:
      - name: Testing Yale Event
        state: "{{ trigger.event.data['date'] }}"
        attributes:
          plaintext: |
            {{ trigger.event.data['text'] | regex_findall(find='(?<=Device Name:).*?(?=\sfrom Account:)') }}
          base64: |
            {{ trigger.event.data['text'] | base64_decode | regex_findall(find='(?<=Device Name:).*?(?=\sfrom Account:)') }}

Jakesa · November 3, 2023, 2:07pm

Sorry, I struggle to articulate myself sometimes. Yes, the sensor provides that output:

If I mark my original email (Garage PIR) as unread, it changes to:

but all other emails involving the living room PIR result in the "

Didgeridrew · November 3, 2023, 2:09pm

It looks like we both struggle…

I meant these formatted-text boxes:

I need to know if that is how the message text is coming into HA or if you copy/pasted it from elsewhere.

Jakesa · November 3, 2023, 2:18pm

Ahh I am with you now. I think I complicated things here, this is copied and pasted directly from my email (I just added it with the email screenshots for ease of reading).

Didgeridrew · November 3, 2023, 2:24pm

It looks like I inadvertently left a trailing quotation mark on the template, that’s the source of the " (I have corrected it and removed the “from” in my post above). But that doesn’t explain the lack of the device name for the others.

Jakesa · November 3, 2023, 2:25pm

I tried the second sensor but it doesn’t seem to be able to pull the text.

Didgeridrew · November 3, 2023, 2:48pm

While it would be weird for one to be encoded and the other not, just to rule it out… let’s change the testing sensor to make it as simple as possible:

  - trigger:
      - platform: event
        event_type: "imap_content"
        event_data:
          sender: [email protected]
    sensor:
      - name: Testing Yale Event Plaintext
        state: "{{ now() }}"
        attributes:
          plaintext: |
            {{ trigger.event.data['text'] }}
      - name: Testing Yale Event Base64
        state: "{{ now() }}"
        attributes:
          base64: |
            {{ trigger.event.data['text'] | base64_decode }}

EDIT: It’s probably better to split them just in case…

Jakesa · November 3, 2023, 4:06pm

This also displayed as unknown

Didgeridrew · November 3, 2023, 4:09pm

Even after an email is received from the correct sender?

Jakesa · November 3, 2023, 4:26pm

Yes, I am marking the email as unread, so it appears as a new email. I did this when testing the original template, and it worked ok. I also have an IMAP sensor to notify me when I receive emails, and that confirms a new email has been received.

I just restarted my config again to make sure it wasn’t something like that, but it doesn’t seem to have changed anything. Looking at the sensor history, it does look like it changed from unknown to unavailable, but I am not sure if this was caused by restarting HA.

Jakesa · November 7, 2023, 10:15am

This is all working find now, thank you. The quotation mark seemed to have confused it but after changing it there are no issues.

Do you have a coffee/beer link?

Jakesa · December 1, 2023, 1:49pm

So I am not sure what has changed but for some reason this has stopped working. Whenever I receive emails from [email protected] it does not update the state. I have no idea on where to start on troubleshooting this, can anyone point me in the right direction, please?

Template:

template:
  - trigger:
      - platform: event
        event_type: "imap_content"
        event_data:
          sender: [email protected]
    sensor:
      - name: Yale Alarm Event
        state: |
          {{ trigger.event.data['text'] 
          | regex_findall(find='(?<=Device Name:).*?(?=\sfrom Account:)', ignorecase=False)| first }}
        attributes:
          timestamp:  "{{ trigger.event.data['date'] }}"

Didgeridrew · December 1, 2023, 3:23pm

The place to start is the email. Has something changed that would cause the regex portion to fail? This is one of the major drawbacks to using emails from a third party. I have a theory that Marketing and Customer Service departments make pointless changes just so they can tell their bosses that they contributed to the quarterly system update…

The state will only update when it changes, so repeated triggers of the same sensor will have no effect on the sensor state. This is why several of my earlier examples used the “date” value or now() for the state… it helps insure that a series of messages about 1 sensor don’t get lost.

Jakesa · December 2, 2023, 11:11am

Thank you for your reply @Didgeridrew. The emails look exactly the same, I also marked old emails as unread to test and this was also not working. However, I agree with your theory.

Jakesa · December 2, 2023, 11:17am

I tried this too to make it simple but they both show as unknown

Didgeridrew · December 2, 2023, 3:10pm

Is your IMAP sensor still catching the yale emails?

If the emails are being picked up by the IMAP sensor, then set up a basic IMAP content sensor using the existing sender trigger. You can leave “Yale Alarm Event” in place and just add the following below it (make sure to indent everything properly):

      - name: Basic Yale Event
        state: "{{ trigger.event.data['text'] }}"
        attributes:
          timestamp:  "{{ trigger.event.data['date'] }}"

Reload template entities or restart HA then trigger one of the sensors.