Imap_email_content sensor go through all emails in inbox after restart

I just started playing with the imap_email_content sensor component.

I notice whenever I restarted my HA, the sensor state will change by going through every emails I have in my inbox in 30 seconds interval. That means, if I have 100 emails in my inbox, it will take 50 minutes before it can reach the latest email.

I can manually clear my inbox everyday but I am wondering is there other workaround for this or is it only happen on me?

I have same problem. It would be great if sensor show only unread messages.

I found a workaround by using a python script with shell command platform.

This is the script I used (taken from https://stackoverflow.com/a/8255579)

#!/usr/bin/python3
import imaplib
box = imaplib.IMAP4_SSL('imap.mail.microsoftonline.com', 993)
box.login("[email protected]","paswword")
box.select('Inbox')
typ, data = box.search(None, 'ALL')
for num in data[0].split():
   box.store(num, '+FLAGS', '\\Deleted')
box.expunge()
box.close()
box.logout()

save it as a file, e.g. emptyinbox.py in /home/pi/.homeassistant

Then add this in HA…

shell_command:
  emptyinbox: python ~/emptyinbox.py

Finally create this automation to run it whenever HA start…

- alias: 'Empty Inbox'
  initial_state: 'on'
  trigger:
    platform: homeassistant
    event: start
  action:
    service: shell_command.emptyinbox

Warning: This will delete every messages in inbox and clear the trash folder every time HA restarts.

2 Likes

I found a sollution to inplement this on gmail to (on the same stackoverflow page)

The (3) is suppose to delete mail older then 3 days (was 365 in their example)

But i only get this error in hass.io when i restart my hass.io

2018-04-18 10:33:42 ERROR (MainThread) [homeassistant.components.shell_command] Error running command: `python ~/emptyinbox.py`, return code: 127
NoneType: None

Im not competent enogh to find out whats wrong.
Is there anyone that is using a funktion like this?
(And yes, in my code i have entered my e-mail and password)

#!/bin/python3

import datetime
import imaplib

m = imaplib.IMAP4_SSL("imap.gmail.com")  # server to connect to
print "Connecting to mailbox..."
m.login('gmail@your_gmail.com', 'your_password')

print m.select('[Gmail]/All Mail')  # required to perform search, m.list() for all lables, '[Gmail]/Sent Mail'
before_date = (datetime.date.today() - datetime.timedelta(3)).strftime("%d-%b-%Y")  # date string, 04-Jan-2013
typ, data = m.search(None, '(BEFORE {0})'.format(before_date))  # search pointer for msgs before before_date

if data != ['']:  # if not empty list means messages exist
    no_msgs = data[0].split()[-1]  # last msg id in the list
    print "To be removed:\t", no_msgs, "messages found with date before", before_date
    m.store("1:{0}".format(no_msgs), '+X-GM-LABELS', '\\Trash')  # move to trash
    print "Deleted {0} messages. Closing connection & logging out.".format(no_msgs)
else:
    print "Nothing to remove."

#This block empties trash, remove if you want to keep, Gmail auto purges trash after 30 days.
print("Emptying Trash & Expunge...")
m.select('[Gmail]/Trash')  # select all trash
m.store("1:*", '+FLAGS', '\\Deleted')  #Flag all Trash as Deleted
m.expunge()  # not need if auto-expunge enabled

print("Done. Closing connection & logging out.")
m.close()
m.logout()
print "All Done."

Your post is precious to me, as I am an absolute beginner for Linux and Hassio, so I wish to thank you for sharing your solution.

My extra problem is that my istance of Hassio is running on a dedicated Raspberry PI machine. That means I don’t know how to access the directory /home/pi/.homeassistant. Would you please waste two more minutes and explain how to do the same in my case?

Thank you in advance

scicchi

Since then library seem to be changed a bit so for gmail, I’m using this:

from imapclient import IMAPClient

obj = IMAPClient('imap.gmail.com', ssl=True)
obj.login('"[email protected]"','password')
obj.select_folder('inbox')
msg_ids = obj.search(('FROM', '[email protected]'))
obj.delete_messages(msg_ids)
obj.logout()

Hello, I’m interested too the delete of the e-mails, and I experimented that the code of masterkenobi I happily used doesn’t work since a few months (before was perfect).

Could you please explain me what changes I have to make to solve the issue?

Best regards, Marco

the same code is still working for me.

Code (python3) to delete gmail messages in two different inbox:

#!/bin/python3
#
# program to delete Emails in GMAIL older than 10 days in "Inbox" and "Protection" folders
#
#

import datetime
import imaplib

#user credentials 
my_email= "[email protected]"
app_generated_password = "your_gmail_app-password" 

#initialize IMAP object for Gmail
imap = imaplib.IMAP4_SSL("imap.gmail.com")

print("Connecting to GMAIL...")

#login to gmail with credentials
imap.login(my_email, app_generated_password)

print("GMAIL connected...")

print("Deleting Emails in INBOX mailbox...")

imap.select("Inbox")

before_date = (datetime.date.today() - datetime.timedelta(10)).strftime("%d-%b-%Y")  # date string, 04-Jan-2013
typ, message_id_list = imap.search(None, '(BEFORE {0})'.format(before_date))  # search pointer for msgs before before_date

#convert the string ids to list of email ids
messages = message_id_list[0].split(b' ')

if (len(messages) != 1):
    print("Deleting mails...")
    count = (len(messages))
    while count > 1:
        # mark the mail as deleted
        imap.store(messages[count-1], "+FLAGS", "\\Deleted")

        print(count, "mail(s) remaining to deleted")
        count = count - 1
else:
    print("No mail to delete...")

print("Deleting Emails in PROTECTION mailbox...")

imap.select("Protection")

before_date = (datetime.date.today() - datetime.timedelta(10)).strftime("%d-%b-%Y")  # date string, 04-Jan-2013
typ, message_id_list = imap.search(None, '(BEFORE {0})'.format(before_date))  # search pointer for msgs before before_date

#convert the string ids to list of email ids
messages = message_id_list[0].split(b' ')

if (len(messages) != 1):
    print("Deleting mails...")
    count = (len(messages))
    while count > 1:
        # mark the mail as deleted
        imap.store(messages[count-1], "+FLAGS", "\\Deleted")

        print(count, "mail(s) remaining to deleted")
        count = count - 1
else:
        print("No mail to delete...")

print("All selected mails has been deleted")

# delete all the selected messages 
imap.expunge()
# close the mailbox
imap.close()

print("Closing connection with GMAIL...")

# logout from the server
imap.logout()

print("Connection closed with GMAIL...")


Hi. Looking for a way to delete my emails. Will test your script out. Will this work if I drop it in the python_scripts folder and call it to run with

python_script.emptyinbox.py

Or do I need to create a shell_command ?

Also, any way to hide the email and app pass via a secret file??

You can run it directly from the terminal… or use a shell_command…

Regarding the user/password in clear, that’s something I have to do is to pass parameters to the script from HA or from the terminal…

I will try to do it in the next few days…

I have tried adding it to python_script folder but it throws an error. Seems it cannot import. I used shell_command unstead.

Here is the new python script with parameters and hidden user/password.

First create the following command in secret.yaml file (this is depending where you stored the python script, mine is in “/config”):

#
# Command to clean Emails older than x days
#
cleanup_command: python3 /config/empty_mailbox.py my_email my_password 10 Inbox Protection > /config/empty_mailbox.log

You can omit the “Inbox Protection” (my two folders to cleanup), by default, the script will clean the “Inbox” folder…

than create the shell_command in configuration.yaml:

shell_command:
  empty_mailbox: !secret cleanup_command

Here is the script:

#!/bin/python3
#====================================================================================================================================
#
# program to delete Emails in GMAIL older than x days in specific folders
# Arguments:
#   Arg1 (required): Username to access the mailbox (example: [email protected])
#   Arg2 (required): Password to access the mailbox (example: my_password)
#                    Recommendation: create an application password in GMAIL to avoid some login problems when connecting to the mailbox
#   Arg3 (required): Days to keep (example: 10)
#   Arg4 (optional): List of folder names to clean (if no arguments are provided, the default folder is "Inbox") (example: Inbox Protection Commercial)
# Example of commands (the first one with 3 folders to cleanup, the second one will cleanup "Inbox" only):
#   empty_mailbox.py [email protected] my_password 10 Inbox Protection Commercial
#   empty_mailbox.py [email protected] my_password 10
#
#====================================================================================================================================

import datetime
import imaplib
import sys

#====================================================================================================================================
#
# create function to delete mail in a mailbox older than x days
# arguments: mailbox name, number of days to keep
#
#====================================================================================================================================

def delete_email_in_mailbox(mailbox,days):

    print("Deleting Emails in",mailbox,"older than",days,"days...")
    
    imap.select(mailbox)
    # create the list of mails to delete based on the number of days to keep
    before_date = (datetime.date.today() - datetime.timedelta(days)).strftime("%d-%b-%Y")  # date string, 04-Jan-2013
    typ, message_id_list = imap.search(None, '(BEFORE {0})'.format(before_date))  # search pointer for msgs before before_date
    
    # convert the string ids to list of email ids
    messages = message_id_list[0].split(b' ')
    
    if (len(messages) != 1):
        print("Deleting mails older than",days,"days...")
        # count the number of messages to delete
        count = (len(messages))
        # loop to delete messages
        while count > 1:
            # mark the mail as deleted
            imap.store(messages[count-1], "+FLAGS", "\\Deleted")
            # decrement the counter by one
            count = count - 1
            # print the number of remaining mails to delete
            print(">>>>",((datetime.datetime.today()).strftime("%Y-%b-%d %H:%M:%S")),"   ",count, "mail(s) remaining to delete")
    else:
        print("No mail to delete...")

#====================================================================================================================================
#
# Main Program
#
#====================================================================================================================================

print (">>>>",(datetime.datetime.today()).strftime("%Y-%b-%d %H:%M:%S"))

# read arguments: username, password and number of days have to be provided, folders are optional
arg = sys.argv

# not enough arguments, error message and exit
if (len(arg) <= 3):
    print("Error ! not enough arguments: at least 3 are required.")
    print("        User, password and number of days are mandatory.")
    exit()

# store arguments in variables
my_email = sys.argv[1]
app_generated_password = sys.argv[2]
days = int(sys.argv[3])
folder = sys.argv[4:] 

# if no folder has been provided, "Inbox" is the default
if (len(folder) == 0):
    folder = ["Inbox"]
 
# initialize IMAP object for Gmail
imap = imaplib.IMAP4_SSL("imap.gmail.com")

print("Connecting to GMAIL...")

# login to gmail with credentials
imap.login(my_email, app_generated_password)

print("GMAIL connected...")

count = (len(folder))
while count > 0:
    print(" ")
    print(" ")
    count = count - 1
    # delete Emails in folder older than x days
    delete_email_in_mailbox(folder[count],days)
else:
    print(" ")
    print(" ")
    print("All selected mails have been deleted")

# delete all the selected messages 
imap.expunge()
# close the mailbox
imap.close()

print("Closing connection with GMAIL...")

# logout from the server
imap.logout()

print("Connection closed with GMAIL...")

print (">>>>",(datetime.datetime.today()).strftime("%Y-%b-%d %H:%M:%S"))

1 Like

Ok. Thanks for the rewrites. Will give this a go. If we only provide one folder , the script will ONLY delete what is inside folder and not Inbox?

@duceduc Correct… If you want, you can comment the line 47 of that script (imap.store(messages[count-1], “+FLAGS”, “\Deleted”) marking the message to be deleted as well as line 105 (imap.expunge())… and run it like that to test, no message will be deleted…

ok. I just tested and it seems to be working. Just have to wait until after the 5th day for my setup. I am using yahoo email for this script. Just change this line to the yahoo imap.

imap = imaplib.IMAP4_SSL("imap.mail.yahoo.com")

I ran a test on a rpi.

sudo python3 purge_imap.py [email protected] appPassword 5 External

log:

>>>> 2022-Aug-06 09:21:02
Connecting to YAHOO...
YAHOO connected...

Deleting Emails in External older than 5 days...
No mail to delete...

All selected mails have been deleted
Closing connection with YAHOO...
Connection closed with YAHOO...
>>>> 2022-Aug-06 09:21:06
1 Like

Probably another opportunity to make this procedure more flexible: add Yahoo/Gmail as a parameter to choose to cleanup one or the other… Thank you for the feedback…

@duceduc The last version…

Change log:

  1. list all available folders in the mailbox
  2. check that the folder(s) provided as arguments exist in the mailbox (!! case sensitive !!). Exact match is required
  3. print for each Email to delete:
    • the date the Email has been sent (not received !)
    • the sender Email
    • the subject
  4. fixed some minor bugs…

Program has only been tested on Gmail !..

#!/bin/python3
#====================================================================================================================================
#
# program to delete Emails in GMAIL older than x days in specific folders
# Arguments:
#   Arg1 (required): Username to access the mailbox (example: [email protected])
#   Arg2 (required): Password to access the mailbox (example: my_password)
#                    Recommendation: create an application password in GMAIL to avoid some login problems when connecting to the mailbox
#   Arg3 (required): Days to keep (example: 10)
#   Arg4 (optional): List of folder names to clean (if no arguments are provided, the default folder is "INBOX") (example: INBOX Protection Commercial)
# Example of commands (the first one with 3 folders to cleanup, the second one will cleanup "INBOX" only):
#   empty_mailbox.py [email protected] my_password 10 INBOX Protection Commercial
#   empty_mailbox.py [email protected] my_password 10
#
# IMPORTANT: tested on GMAIL only...  Use it at your own risks
#
#====================================================================================================================================

import datetime
import imaplib, email
from email.header import decode_header
import sys

#====================================================================================================================================
#
# create function to delete mail in a mailbox folder older than x days
# arguments: folder name, number of days to keep
#
#====================================================================================================================================

def delete_email_in_folder(folder,days):

    print("Deleting Emails in",folder,"older than",days,"days...")
    
    imap.select(folder)
    # create the list of mails to delete based on the number of days to keep
    before_date = (datetime.date.today() - datetime.timedelta(days)).strftime("%d-%b-%Y")  # date string, 04-Jan-2013
    typ, message_id_list = imap.search(None, '(BEFORE {0})'.format(before_date))  # search pointer for msgs before before_date
    
    # convert the string ids to list of email ids
    messages = message_id_list[0].split(b' ')
    
    if (len(messages) != 1):
        # count the number of messages to delete
        count = (len(messages))
        # loop to delete messages
        while count > 0:
            # print the number of remaining mails to delete
            print(">>>>",((datetime.datetime.today()).strftime("%Y-%b-%d %H:%M:%S")),"   ",count, "mail(s) remaining to delete")
            # read email message
            res, msg = imap.fetch(messages[count-1], "(RFC822)")
            # extract subject and from email address
            for response in msg:
                if isinstance(response, tuple):
                    msg = email.message_from_bytes(response[1])
                    subject, From, Date = obtain_header(msg)
            # Printing Subject and From information
            print("    Date   :", Date)
            print("    Subject:", subject)
            print("    From   :", From)
            # mark the mail as deleted
            imap.store(messages[count-1], "+FLAGS", "\\Deleted")
            # decrement the counter by one
            count = count - 1

    else:
        print("No mail to delete...")

#====================================================================================================================================
#
# create function to extract subject, "from" email address information and the date of the current Email to delete
# arguments: mail message
# Return: the subject, the From information and the date of the Email (the date the Email was sent... not the date the Email was received !)
#
#====================================================================================================================================

def obtain_header(msg):
    # decode the email subject
    subject, encoding = decode_header(msg["Subject"])[0]
    if isinstance(subject, bytes):
        subject = subject.decode(encoding)
 
    # decode email sender
    From, encoding = decode_header(msg.get("From"))[0]
    if isinstance(From, bytes):
        From = From.decode(encoding)

    # decode email date (sent)
    Date, encoding = decode_header(msg.get("Date"))[0]
    if isinstance(Date, bytes):
        Date = Date.decode(encoding)

    # transform date to local date (your current timezone)
    datetime_object = datetime.datetime.strptime(Date,'%a, %d %b %Y %H:%M:%S %z')
    #Convert it to your local timezone
    d=datetime_object.astimezone()
    Date = d.strftime("%a, %d %b %Y %H:%M:%S %z")
 
    return subject, From, Date

#====================================================================================================================================
#
# Main Program
#
#====================================================================================================================================

print (">>>>",(datetime.datetime.today()).strftime("%Y-%b-%d %H:%M:%S"))

# read arguments: username, password and number of days have to be provided, folders are optional
arg = sys.argv

# not enough arguments, error message and exit
if (len(arg) <= 3):
    print("Error ! not enough arguments: at least 3 are required.")
    print("        User, password and number of days are mandatory.")
    exit()

# store arguments in variables
my_email = sys.argv[1]
app_generated_password = sys.argv[2]
days = int(sys.argv[3])
folder = sys.argv[4:]

# if no folder has been provided, "INBOX" is the default
if (len(folder) == 0):
    folder = ["INBOX"]
 
# initialize IMAP object for Gmail
imap = imaplib.IMAP4_SSL("imap.gmail.com")

print("Connecting to GMAIL...")

# login to gmail with credentials
imap.login(my_email, app_generated_password)

print("GMAIL connected...\n")

# retrieving list of folders
print("List of various folders:")

folder_list = imap.list()

# error: not able to retrieve list of inboxes
if (folder_list[0] != "OK"):
    print("Error ! Not able to retrieve list of Inboxes")
    exit()

folders = folder_list[1]

# extract the list of folder names between quotes
list_of_folders = ""
for name in folders:
    name_str = name.decode('utf-8')
    name_str = name_str.split('/',1)
    name_str = name_str[1]
    print (name_str[3:-1], end='\n')
    list_of_folders = list_of_folders + name_str[2:] + "\n"

# loop to execute the cleanup based on the number of folders provided as parameters
count = (len(folder))
loop = 0
while count > 0:
    print(" ")
    print(" ")
    count = count - 1
    # test if folder is included in the folder list
    test_folder = ("\""+folder[count]+"\"")
    if (test_folder in list_of_folders):
        # delete Emails in folder older than x days
        delete_email_in_folder(folder[count],days)
        loop = 1
    else:
        print("Error ! Unknown folder in the list of folders:",folder[count])
        print("        See above for the list, the folder name must be exactly the same (the folder name validation is case sensitive)")
else:
    print(" ")
    print(" ")

# delete all the selected messages 
if (loop == 1):
    print("All selected mails have been deleted")
    imap.expunge()
    # close the mailbox
    imap.close()

print("Closing connection with GMAIL...")

# logout from the server
imap.logout()

print("Connection closed with GMAIL...")

print (">>>>",(datetime.datetime.today()).strftime("%Y-%b-%d %H:%M:%S"))

Bummer, the script didn’t work for me using yahoo credentials. Log shows that it connected, list all my folders, but says no mail to delete. I didn’t try for a gmail account.

Format for Yahoo mail is maybe different than Gmail… I will try to look at that…