Github rate limit error HACS

Is anyone else getting rate limit errors? I thought that was the point of using an API tken.
Saw it last night and then again this morning.
I have now set up a new github account just for HACS api… It caused a few rest sensors to stop working as well. Never had issues before this…

1 Like

You haven’t included logs so this is just a guess.

No, the point of an API token is for a software or service to act on your behalf without you explicitly logging in with your username/password/MFA


API providers impose their own rate limiting (if they have any sense).

If I found the right documentation for GitHub it says:

User-to-server requests are rate limited at 5,000 requests per hour and per authenticated user

So if you are approaching 5K requests in one hour I think there may be an issue.

The log just says API rate limited

Yes I saw that as well… no way I’m doing 5000/hr.
I actually was suspecting it was 60/hr if unauthenticated.
I suppose if it happens again I could turn on debug logging for HACS.
I have been using HACS since it was released and it’s only in last few days I have seen this happen. I was wondering if it was possible it was the latest HACS update causing it.

Really just curious if anyone else has seen this error before I investigate further.

Found this for you which prints out how many requests you have made for unauthenticated requests on your user.

curl -i https://api.github.com/users/YOUR_GITHUB_USER_HERE

You will receive these lines what inform you of remaining calls and total calls:

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 56

I am not certain if HACS uses the User API or one of the other GitHub API, and obviously this doesn’t include requests made with the personal token.

If you have a copy of your token somewhere you should be able to make this request authenticated and find out if you have indeed made 5K requests within an hour.

1 Like

Yeah I have used that as well. It seems to always say less than 60…

I do have a copy but can’t find or work out the command to use it for this.

I thought HACS uses the personal token… that is the rationale for requiring the token given in the HACS repo.

It does (or it is meant to). Regardless, there is a rate limit on requests!

Yes which if it’s using an authenticated request which the API is (I thought) then it should be 5000 hence my questions.

Same command but authenticated

curl -i -u YOUR_GITHUB_USER:YOUR_GITHUB_PERSONAL_TOKEN  'https://api.github.com/users/YOUR_GITHUB_USER'

I would rather check with facts and figures than assume. There can be many points of breaking between making a call and receiving a reply; we don’t know till we analyse your specific situation and why you’re getting that message.

2 Likes

X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4741
X-RateLimit-Reset: 1601003186
Resets in 15 minutes…

and the new ID I set up
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4005
X-RateLimit-Reset: 1601002651
X-RateLimit-Used: 995
Resets in 2 minutes

No idea where the original one is being used…
The bottom one above is the one I configured for HACS this morning.

1000 in an hour is a bunch of requests eh?

That’s 999 more than I expected in an hour :stuck_out_tongue:

Glad you haven’t exhausted the 5000, but it doesn’t rule out that at the time of that message your 5000 allocation had run out.

If you wanted to track this, you could plug that into HA as a REST sensor to count how many requests you’re making haha

1 Like

actually the first one has a second API token and it probably counts against the same account…

Well from the errors I saw, it looks like every single HACS theme, card, integration generates an API call… I don’t know how many calls HACS makes per hour.
I also have a couple of rest sensors that use the API (like getting the version of docker-compose etc). Not sure how frequently rest sensors update…
I think at least having HACS on it’s own API/UserID now will make sure even if HACS falls over that all my other sensors will continue to work properly (which was my main annoyance)

I might actually do that!

1 Like

OK so I did that.
Not sure how to do this with a command line sensor in HA so I just wrote a bash script and cronjob in bash to push the usage to my MQTT Broker:

#!/bin/bash

# Only edit the following setup lines with correct login information

# Mosquitto Broker Details

mqttuser=
mqttpassword=
mqttbroker=
mqttport=
token0=
token1=

# DO NOT ALTER ANYTHING BELOW THIS LINE

githubusage0=$(curl -H "Authorization: token $token0" https://api.github.com/rate_limit)
githubusage1=$(curl -H "Authorization: token $token1" https://api.github.com/rate_limit)

# Get Variables from Strings
usedcalls0=$(echo "$githubusage0" | jq '.resources.core.used')
usedcalls1=$(echo "$githubusage1" | jq '.resources.core.used')
echo $usedcalls0
echo $usedcalls1

usedcalls0='{"state":"'"$usedcalls0"'"}'
usedcalls1='{"state":"'"$usedcalls1"'"}'

# Publish to MQTT Broker

mosquitto_pub -h $mqttbroker -p $mqttport -u "$mqttuser" -P "$mqttpassword" -t github-api-rest -m "$usedcalls0" -r
mosquitto_pub -h $mqttbroker -p $mqttport -u "$mqttuser" -P "$mqttpassword" -t github-api-hacs -m "$usedcalls1" -r

then rest sensors:

# Github Rate Limit
  - platform: mqtt
    name: "Github Main"
    state_topic: "github-api-rest"
    icon: mdi:github
    value_template: "{{ value_json.state }}"
  - platform: mqtt
    name: "Github HACS"
    state_topic: "github-api-hacs"
    icon: mdi:github
    value_template: "{{ value_json.state }}"

image

Still doesn’t explain my original question of why I was rate limited though

Interesting. The account used for this is ONLY used by HACS.

Are those calls remaining?

It would be going up if it was calls made.

It is calls used

Seems to be doing it once a day…
image

Very strange that it goes from 1000 calls up to 3500 (and above).

Perhaps this thread should be raised with the HACS maintainer?
The only other choice beyond that is to log all outbound requests to GitHub to work out what is being requested.

Yeah. I asked Ludeeus on discord and I guess he will respond later today.
I might have to enable debug logging in HACS. Anyway, at least I know for sure HACS is doing it and explains why everything went to shit when it exceeded 5000 calls - which hasn’t’t happened since I split off my other rest sensors that talk with github.

1 Like

Seems HACS typically uses 3500+ calls per hour according to Ludeeus. This number grows as more repos are added to HACS. HACS checks available calls before hitting the API and currently stops when you hit 100 remaining calls. As I was using the same API key for rest sensors that is why it shat itself and why I haven’t hit the limit since I set up a seperate account.
I suspect this is going to be a problem to more people going forward and at least now Ludeeus is aware of this. He said he might set the limit to 1000 instead of 100 and see if that causes any issues.
Anyway, I’m not going crazy.

1 Like