Imap integration broken in 0.110.0 and 0.110.1?

Hi,

I use the imap integration to monitor the state of my house alarm.

  - platform: imap_email_content
    name: Verisure E-mail
    server: imap.gmail.com
    port: 993
    username: !secret gmail_username
    password: !secret gmail_app_password
    senders:
      - [email protected]

This has been working without problems since I set it up many months ago (must be more than a year). Now, the sensor reports Unknown. But, wait, not quite. Every time the alarm’s state is changed (from armed to not armed and vice versa), the sensor reports “armed” or “not armed” for 30 seconds (the polling period of the imap integration?). Then it reverts to Unknown.

image

I’ve tried debugging it, but there are no error messages and I’ve checked that the app password is correct.

Has anyone seen this, or does anyone have an idea what I can do to fix it. I’m thinking about raising an issue, but wanted to check here first that I’m not blundering.

1 Like

Yes, seems normal to me according to release notes…
Does it matter for you that’s unknown? Your automation was already fired to trigger your alarm panel to armed/disarmed?

IMAP Email Content - The state of the imap_email_content is replaced when there is no email found. With this change, any other thing like an automation can react to the arrival of an email - (@isk0001y - #35123) (imap_email_content docs

Ah, yes, of course. I did read the release notes, but didn’t quite understand the significance of that statement. :flushed: Thank you for pointing it out to me. :+1:

Thank you for posting this question, I too had trouble finding what was broken. Nothing really.

But what I did find is:
  on boot:
     if no messages in my inbox I get errors in the log,  
     if old messages already in my inbox, I get notified again.
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 279, in async_update_ha_state
    await self.async_device_update()
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 472, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/imap_email_content/sensor.py", line 230, in update
    email_message = self._email_reader.read_next()
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/imap_email_content/sensor.py", line 127, in read_next
    return self._fetch_message(message_uid)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/imap_email_content/sensor.py", line 106, in _fetch_message
    raw_email = message_data[0][1]
TypeError: 'NoneType' object is not subscriptable
2020-05-22 06:02:10 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.alert_control_email_vtext fails
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 279, in async_update_ha_state
    await self.async_device_update()
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 472, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/imap_email_content/sensor.py", line 230, in update
    email_message = self._email_reader.read_next()
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/imap_email_content/sensor.py", line 127, in read_next
    return self._fetch_message(message_uid)
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/imap_email_content/sensor.py", line 106, in _fetch_message
    raw_email = message_data[0][1]
TypeError: 'NoneType' object is not subscriptable

No message in the inbox leading to error message is probably a bug - not handling the exception well.

New notification on reboot (on a message that has been seen earlier) is, I guess, to be expected. If you don’t want to react to it again on reboot, you need to keep track of when you last reacted to a message and compare that time with message’s timestamp.

Funny thing, after a couple reboots the error message has disappeared from the logs.
New notifications on reboot - currently changing my automatons to remember that I already took action.

Great! :+1:

It does seem, after all, that the behaviour I originally asked about was not desired. HA 0.110.3 includes an update to the imap integration so that as long as you don’t delete the latest e-mail message, the imap_email_content continues to contain the latest e-mail message.

1 Like

Sorry but I don’t understand:
Imap_email_content gives me a sensor.XXXXXXXX where in the attribute body I can read my email content. Perfect. But after midnight when I restart home assistant automaticanlly, the sensor become “unknown”. If I login in gmail I can see the email and the status “unread” so, why the component “Imap_email_content” can’t retriver it and it gives me “unknown”?
I read about to create another sensor template that change its state only if the sensor.XXXXX is not “unknown” but I’m not sure that when I restart homeassistant the new sensor can remember its old state.
My home assistant core version is: 0.115.3

I get a monthly email with a report… I need a sensor that continue to take the last report email also if I restart home assistant and automatically update when I recived a new email (which happens once a month).
Thanks for your feedback…

I’m not able to replicate your problem. I have a sensor defined as in the original post. It returns the latest e-mail content even after reboot.

Can I ask if you use Gmail? My problem appears when I restart the home assistant but after 1 day I got the mail. If I restart home assistant the same day I received the email, the sensor updates correctly.

My configuration.yaml

sensor:
  - platform: imap_email_content
    server: imap.gmail.com
    port: 993
    username: !secret user_email
    password: !secret password_email
    senders:
      - [email protected]

If I activeted the “debug” in logger: for the component homeassistant.components.imap_email_content

logger:
  default: error
  logs:
    homeassistant.components.imap_email_content: debug

The home-assistant.log file:

2020-10-05 17:01:11 INFO (SyncWorker_52) [homeassistant.components.imap_email_content.sensor] Connection to imap.gmail.com lost, attempting to reconnect
2020-10-05 17:01:11 INFO (SyncWorker_52) [homeassistant.components.imap_email_content.sensor] Reconnect to imap.gmail.com succeeded, trying last message
2020-10-05 17:01:42 INFO (SyncWorker_47) [homeassistant.components.imap_email_content.sensor] Connection to imap.gmail.com lost, attempting to reconnect
2020-10-05 17:01:42 INFO (SyncWorker_47) [homeassistant.components.imap_email_content.sensor] Reconnect to imap.gmail.com succeeded, trying last message
2020-10-05 17:02:12 INFO (SyncWorker_41) [homeassistant.components.imap_email_content.sensor] Connection to imap.gmail.com lost, attempting to reconnect
2020-10-05 17:02:12 INFO (SyncWorker_41) [homeassistant.components.imap_email_content.sensor] Reconnect to imap.gmail.com succeeded, trying last message
2020-10-05 17:02:42 INFO (SyncWorker_53) [homeassistant.components.imap_email_content.sensor] Connection to imap.gmail.com lost, attempting to reconnect
2020-10-05 17:02:43 INFO (SyncWorker_53) [homeassistant.components.imap_email_content.sensor] Reconnect to imap.gmail.com succeeded, trying last message
2020-10-05 17:03:12 INFO (SyncWorker_24) [homeassistant.components.imap_email_content.sensor] Connection to imap.gmail.com lost, attempting to reconnect
2020-10-05 17:03:12 INFO (SyncWorker_24) [homeassistant.components.imap_email_content.sensor] Reconnect to imap.gmail.com succeeded, trying last message
2020-10-05 17:03:42 INFO (SyncWorker_34) [homeassistant.components.imap_email_content.sensor] Connection to imap.gmail.com lost, attempting to reconnect
2020-10-05 17:03:42 INFO (SyncWorker_34) [homeassistant.components.imap_email_content.sensor] Reconnect to imap.gmail.com succeeded, trying last message

Is it normal this INFO Connection and Reconnect ?

I will try to add name: XXXXX in my configuration.yaml, this is the only difference between my config and the original post.

And I will edit my recorder: to include the sensor.XXXXX maybe it need to take from data history.

If you have any idea or how to debug to find what happened to it I’m here… Thanks

I am not sure what causes your problems, and I’m sorry that you’re having this troubles.

If state is updated sometimes - then at least you have the user name and password (an app password - not your regular gmail password) correct. Note also that the “senders” variable is the address of the e-mail sender for which state is to be updated.

Do you delete the e-mail after reading it (automatically or manually)? I don’t. I can imagine that if the e-mail is not still there, then reading it will fail, but I’m not sure because I haven’t tried.

(an app password - not your regular gmail password)

to be honest, it’s my regular gmail password… however how did you say if it’s work when I received a new email it means that it works. Could you check if you use your regular password or a specific “app password”. I can try too…

Note also that the “senders” variable is the address of the e-mail sender for which state is to be updated.

Yes, I receive emails in another my gmail account and I have an automatic filter forward to my gmail account set in home assistant IMAP Email Content.

Do you delete the e-mail after reading it (automatically or manually)? I don’t.

I don’t delete too. I never open the gmail account to read o delete the mail. (I created a specific gmail account specifically to home assistant =) ).

However thanks for the help =)

I believe that for a gmail account, you MUST use an app password. I use an app password.

You need to log into the gmail account in question and generate an app password. Set your password_email to the app password in the secrets.yaml file.

however how did you say if it’s work when I received a new email it means that it works.

It’s possible that I misunderstood you. I thought you meant that the imap entity state is correctly updated on the same day as you receive the e-mail. If that is not the case, it is probably because you’re not using an app password for login to the gmail account.

Also check that your forwarded mail correctly has the address given in the senders: variable.

Thanks for your help. You understood well. The IMAP entity state is correctly updated on the same day as I receive the e-mail. (So the password it’s ok. So the senders. Also because I checked the configuration in that Gmail account and I can NOT create an app password because I activated insecure app permission (less secure app permission). However, don’t lose your time…I give up. I appreciated your help! Thanks! If I will fix it, I will post the solution here! Thanks

OK. I wish you the best of luck. :smiley:

1 Like