your IP ####### is being flagged as a WebScraper* in our reputation database, due to continuous web scraping activity over an extended timeframe (weeks). - the activity is detected every other day, hence the reputation score never decays ( when score decayed, you would no longer get denied).
It appears you are performing some kind of scraping activity from this IP towards one or many websites/APIs that are accelerated & protected by Akamai (e.g. multiple repeated USPS tracking queries, price list graps etc).
*[Web scrapers are one type of bot that collects web content for a specific purpose. A scraper makes requests to web pages and stores the data it collected. This data could be hotel or airline prices, store locations, current sales or any other valuable data. This stored information is analyzed and used for competitive intelligence, or sold to other parties]
As I unerstand Home Assistant USPS componet is one type of web scraper. Now I am blocked and they cant do anything about it .
I emailed them to [email protected], told them about my problem and after a few emails back and forward they unblocked me, but told me that it could happen again. I am not using usps component for now.
If you email them, you have to send them the picture of this problem, your address, phone number, and your public IP address. if you do not know it, just google “what is my ip” and you will see it.
In your email tells them to forward this information to the networking team.
I had the same issue too. I had been using the USPS component for some time now and recently when the component broke…it kept resulting in session expired, cannot log in. I was not blocked from usps.com, however I was getting “Access Denied” error from homedepot.com, aa.com, tsa.gov, babycenter.com, buybuybaby.com, costco.com, and the list kept growing.
I finally emailed akamai to their support email contact – they responded within a couple of hours to confirm that my IP had in fact been flagged as a Web Scraper and therefore their clients (websites) could begin to block us. Moreover, the negative reputation may spread to other sites you’ve never accessed before.
Kill this component and wait a couple of days for your reputation to go back up.
Hmm, this is indeed the issue with web scraping for any service and it really is a shame that we’re not allowed to have access to the public APIs so that real integrations can be made.
mayker
(maykar (pronounced "maker" with a southern accent))
6
I use this method instead. Uses the daily email sent from USPS. No scraping of USPS necessary.
At the end of 2018 I used this code to pull together support for all of the major US shippers into one.
After trial and error I have finally gotten into a custom component. It works but all configuration needs to be made in the components python code. Looking for help to take it to the next level with config file configuration.
I would assume most addons use an API as they often have a setting to set how often the query is run, for example the DarkSKy sensor.
Is there a way to tell if a HA component is getting it’s info from an API or using Webscraping instead? I’m suprised to see an in built component do this.