Haveibeenpwned (HTTP Status_code = 429)

Hi All,

Im using haveibeenpwned for 9 email addresses.
But when I restart HA I get those errors:

2025-01-28 17:17:22.457 ERROR (SyncWorker_7) [homeassistant.components.haveibeenpwned.sensor] Failed fetching data for EMAIL ADRES 1 (HTTP Status_code = 429)

2025-01-28 17:17:33.614 ERROR (SyncWorker_5) [homeassistant.components.haveibeenpwned.sensor] Failed fetching data for EMAIL ADRES 2 (HTTP Status_code = 429)

2025-01-28 17:17:44.290 ERROR (SyncWorker_3) [homeassistant.components.haveibeenpwned.sensor] Failed fetching data for EMAIL ADRES 3 (HTTP Status_code = 429)

2025-01-28 17:18:02.095 ERROR (SyncWorker_8) [homeassistant.components.haveibeenpwned.sensor] Failed fetching data for EMAIL ADRES 4 (HTTP Status_code = 429)

2025-01-28 17:18:12.643 ERROR (SyncWorker_8) [homeassistant.components.haveibeenpwned.sensor] Failed fetching data for EMAIL ADRES 5 (HTTP Status_code = 429)
- platform: haveibeenpwned
  email:
    - !secret haveibeenpwned_1
    - !secret haveibeenpwned_2
    - !secret haveibeenpwned_3
    - !secret haveibeenpwned_4
    - !secret haveibeenpwned_5
    - !secret haveibeenpwned_6
    - !secret haveibeenpwned_7
    - !secret haveibeenpwned_8
    - !secret haveibeenpwned_9
  api_key: !secret haveibeenpwned_api

I removed the sensor for 24 hours to see if I add it again those errors will be gone.
But it didn’t.

What can I do with this?

Check out this KB: https://support.haveibeenpwned.com/hc/en-au/articles/5744766972431-Why-do-I-keep-getting-HTTP-429-Too-Many-Requests-when-querying-within-the-rate-limit

429 is returned when you exceed the rate limit for your key which shoildn’t happen from this integration alone. You might be using the same key from a process somewhere else, try either dropping the request rate on that or rotating your key to ensure it’s not used anywhere else.

Did a renew API. Maybe it will be the issue that I restart HA sometimes multiple times in a row. And with 9 email addresses it will exceed the limit ?

Exceeding the rate limit only returns 429 for a very short period afterwards so a reboot shouldn’t have any long lasting effect. HIBP is my service but this isn’t my integration, I wonder how frequently it’s polling the API? It seems like a 6s sleep between requests would make sense (which would keep it within the smallest rate limit), plus on average there’s only a new breach every ~4 days so running the process daily would certainly be sufficient.

the integration of HIBP doesn’t give you a option to change the scan interval.
Maybe a feature request to change the interval period. For every day one time. Because the breach update is ~4 days.

Looks like it checks every 15 min.
https://github.com/home-assistant/core/blob/012f7112d7fa0e673f30eb5f2980c883c66b6be9/homeassistant/components/haveibeenpwned/sensor.py

Unless you have an automation that invokes homeassistant.update_entity against one (or more of the targets)

Note that the integration is without a codeowner.

If my reading of the code is correct, it’s sleeping for 5 secs on failure. At face value, I suspect it’s making too many requests, raising an HTTP 429, sleeping 5 seconds then trying again. That second try will most likely be ok (network latency adding a bit of delay), but if not, it’d sleep another 5 then try again. That’s just my guess, someone probably needs to dig a bit deeper, it’s a pretty simple API though :slightly_smiling_face:

im not a programmer… but is there any way to change the 5 seconds and 15minutes somewhere in the code local. To test if this will make any difference?

I don’t use that integration, but I got curious and did some digging.

If you’re on the Pwned 1 Tier, you are only allowed to make 10 requests every minute.
You have 9 email addresses and the integration is attempting to resend the request every 5s on failure. You will keep getting that 429 because you’re essentially making 18x the maximum allowed number of requests (20 times per minute X 9 addresses = 180 requests per minute).

Hell, even if you had only 1 address being checked, the default 5s retry on failure is twice the allowed rate limit, so definitely needs to be changed from seconds=5 to minutes=1 (or higher). It’s a relatively small fix which can be done in line 30 here.

You have 3 possible options at the moment:

  • Disable the HIBP integration for at least 1 minute. This will clear your rate limited attempts and should get the integration back in a working state when you re-enable it. (Immediate short-term fix which will have to be repeated each time this happens)
  • Wait for someone to apply the fix to MIN_TIME_BETWEEN_FORCED_UPDATES I suggested, or submit the PR yourself. (Will need to go through the code approval/release process, so might take a while)
  • Go to the integration settings and switch off “Enable polling for updates”, then create an automation which calls homeassistant.update_entity for each of your email entities once a day, like @ludeeus suggested. (Immediate long-term fix which will prevent this from happening).

There are several things that should be done to this integration.
It are lagging behind most current standards for integrations.
PRs against it will most likely take longer due to that it does not have a code owner.
Properly testing the changes require also the author to be an active user if it, as it requires a paid API key (as it should be), but it limits the amount if people that can (with good confidence) help out with it.

1 Like

The integration is not controlled inside the integration portal.
It’s a sensor in the configuration. I don’t know if you can turn it off the Enable Polling for updates:

halfway there is a ZIP file. I unpacked this one in custom_components. Changed in the sensor.py the seconds to 60 and minutes to 60

Hoping HA will use this script instead the one in core.

Yep, was just about to suggest running it as a custom component given your lack of options. Good luck!

EDIT:

Halfway this link there is a ZIP use that one to change update interval see below:

END EDIT

I changed some things to use the custom_compontent even if it will overwrite the core
It takes more time to have the entities back again after a HA restart.

After all emails were done and entities went back. no 429 errors in the log

manifest.json

{
  "domain": "haveibeenpwned_custom",
  "name": "HaveIBeenPwned_custom",
  "documentation": "https://www.home-assistant.io/integrations/haveibeenpwned",
  "codeowners": [],
  "version": "0.0.1",
  "iot_class": "cloud_polling"
}

sensor.py


MIN_TIME_BETWEEN_FORCED_UPDATES = timedelta(seconds=60)
MIN_TIME_BETWEEN_UPDATES = timedelta(minutes=60)

integration in config

- platform: haveibeenpwned_custom
  email:
    - !secret haveibeenpwned_1
    - !secret haveibeenpwned_2
    - !secret haveibeenpwned_3
    - !secret haveibeenpwned_4
    - !secret haveibeenpwned_5
    - !secret haveibeenpwned_6
    - !secret haveibeenpwned_7
    - !secret haveibeenpwned_8
    - !secret haveibeenpwned_9
  api_key: !secret haveibeenpwned_api

Every minute a new email was as entity back again.
P.S. the 6 and 22 Breaches are very old addresses haha…

1 Like

Glad you fixed it. Great job!

will do a request to change the seconds/minutes in the code

What will happen if I change the seconds back to 15 seconds?

After restart HA it will update every email on 15 seconds that is 2 1/2 minutes instead of 9.

And leave the updates on 60.

Personally, I’d leave it alone. It’s already been stated that the API is updated around every 4 days.

Having your sensor values fully populated within 10 mins of a restart is extremely low risk and guarantees you will never be able to trigger a 429 error again

1 Like

I take your advice. Since then no more errors :slight_smile:

1 Like