Haveibeenpwned stopped working: failed fetching data (HTTP Status_code = 403)


#1

Hi, I use haveibeenpwned for quite some time, but since a few weeks it doesn’t work any more. The entity in the Lovelace UI returns unknown and I get this error in the HA log:

failed fetching data (HTTP Status_code = 403)

I did turned on debug logging:

# configuration
default: info
logs:
    homeassistant.components.sensor.haveibeenpwned: debug

And I see this in my HA log:

jan 18 17:31:18 INFO (MainThread) [homeassistant.loader] Loaded sensor.haveibeenpwned from homeassistant.components.sensor.haveibeenpwned
jan 18 17:31:18 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.haveibeenpwned
jan 18 17:31:19 DEBUG (SyncWorker_7) [homeassistant.components.sensor.haveibeenpwned] Checking for breaches for email: [email protected]
jan 18 17:31:19 ERROR (SyncWorker_7) [homeassistant.components.sensor.haveibeenpwned] Failed fetching data for [email protected](HTTP Status_code = 403)
jan 18 17:31:27 DEBUG (SyncWorker_5) [homeassistant.components.sensor.haveibeenpwned] Checking for breaches for email: [email protected]
jan 18 17:31:27 ERROR (SyncWorker_5) [homeassistant.components.sensor.haveibeenpwned] Failed fetching data for [email protected](HTTP Status_code = 403)
jan 18 17:31:33 DEBUG (SyncWorker_7) [homeassistant.components.sensor.haveibeenpwned] Checking for breaches for email: [email protected]

I have checked the haveibeenpwned API documentation and I did found this:

403	Forbidden — no user agent has been specified in the request

And this:

Specifying the user agent
Each request to the API must be accompanied by a user agent request header. 
Typically this should be the name of the app consuming the service, for example "Pwnage-Checker-For-iOS".  
A missing user agent will result in an HTTP 403 response. A valid request would look like:

GET https://haveibeenpwned.com/api/{service}/{parameter}
User-Agent: Pwnage-Checker-For-iOS

The user agent should accurately describe the nature of the API consumer
such that it can be clearly identified in the request. 
Not doing so may result in the request being blocked.

So I checked the HA code https://github.com/home-assistant/home-assistant/blob/master/homeassistant/components/sensor/haveibeenpwned.py to see whether it specifies a User_Agent. And it does specify a User_Agent, so a 403 error shouldn’t be the case…

So now I wonder if there are others that also experience this issue? How can I solve this?


#2

I had this earlier today, but it seems to be working again now. I’m guessing there was a problem server end.


#3

Very recently there’s been another massive data breach discovered so I’m sure HaveIBeenPwnd is getting hammered lately with people checking if they’ve been compromised.


#4

I also facing the same issue. Getting 403 response code.


#5

I was checking a total of 5 e-mailaddresses and aparently that has gotten me blocked :frowning:


Request Blocked

You have been blocked from accessing this resource on Have I Been Pwned.

This may be due to violating one or more of the acceptable use terms of the API.

It may also be due to your traffic patterns being similar to other users who may have violated the acceptable use terms.

Tips to avoid requests being blocked include:

  1. Stick well within the published rate limit
  2. Don't distribute requests over multiple IP addresses in an attempt to circumvent the rate limit
  3. Only query the email addresses of people who have a reasonable expectation that you should do so
  4. Avoid prolonged querying of the API over an extended period of time
--------

I found out by using curl to contact the api:
curl https://haveibeenpwned.com/api/v2/breachedaccount/[email protected]

Anyone else having this? And most importantly how to get rid of it?
I disabled the haveibeenpwned component hoping that I would be able to query their service again in a couple days, but after 3 days i;m still blocked.


#6

Interesting.

I get the same, but haven’t had a 403 for ages. Anyone know how we can get unblocked?


#7

Well this sucks, I set up the component to check my wifes and my email adresses and notify me when something happens. Now i happen to check my hassio logs and find all those errors. So I go to check all adresses manually on their site and turns out I’m in this newest breach. Sucks to find out later than everybody else just because i relied on my raspi. I mean this is exactly the reason why I set it up in the first place.
Has anyone found a way to get this component working again? rebooting, disabling the component for a while all did nothing for me.


#8

same for me. I have it disabled for over a week now, and still not unblocked.
The error message i get when using curl contains this line:
“If you believe your request meets these requirements and was still blocked, please send this entire response body along with any communication you send regarding the error.”
I haven’t found any contact information to address this issue to though. I read haveibeenpwned used a CloudFlare service to block ipaddresses (part of the error message shows "class=“cferror_details”), so maybe I should contact CloudFlare.
I’ve no idea lol