Problem accessing Github servers from Home Assistant

Ok, here are some more updates:

The problem does not happen only from Home Assistant, it also happens from Balena Etcher from a laptop connected to the same network while I try to install HA-OS image on a SD Card or USB-SSD drive. Balena Etcher cannot connect to github servers and retrieve the OS Image file.

We further tested by disconnecting the Netgear Nighthawk R7200 router, and connecting the laptop directly from the modem. The result was the same, it could not connect to github servers.

The modem is fine, it was tested from another location and there was no issue connecting to github servers from Balena Etcher or Home Assistant.

My next idea is that the coaxial infrastructure is defective, or that the Optical Network Unit from the ISP is blocking the communication in some way. I really don’t know what the problem is but it sure looks very tricky. I wish some people could have insight into what is going on here.

Thanks for your reply Vincent.

Is it specifically ghcr.io that you can’t reach, or something else? Can you not access it at all, even a ping?

It sure seems my HA install can at least get to it:

[core-ssh ~]$ curl https://ghcr.io/v2/
{“errors”:[{“code”:“UNAUTHORIZED”,“message”:“authentication required”}]}

I wish I had greater insight into what specifically Home Assistant is attempting. I can’t think of anything on my end that changed in January – after my last successful update – that could cause these symptoms.

Thanks for replying @aming-miyembro,

My HA-OS cannot update its core nor supervisor, and cannot install add-ons… But from a laptop connected on the same network, I can access directly to ghcr.io and any subdomain… It’s very confusing!

It definitely seems like an infrastructure issue… I can’t explain why but, within the same building, I have another contract with the same ISP, and if I transfer my Raspberry-Pi to that other location, HA-OS will be able to run normally and access ghcr.io servers and install any add-on and update its core (tested with 2023.3.3)

Interestingly I’ve just started to encounter a similar error too. For me it’s when trying to update a custom addon (hosted on Github). This process was all working just fine up until yesterday, though now it doesn’t/ Any attempt to update the addon (since the version number has increased in the repo, and an update has been identified) results in:

“Head “https://ghcr.io/v2/home-assistant/aarch64-base/manifests/latest”: received unexpected HTTP status: 503 Service Unavailable”

Ordinarily I wouldn’t post about this, though since this thread is brand spanking new I thought it might help to identify a larger issue perhaps?

Exactly the same issue here:
Cannot install Addons from the community repo, from the official repo, installs work… imo an issue with the repo

Any info might help in identifying a new larger issue, thanks @markhinch
What troubles me is that if I change location and hook my RPi4 to another internet network, it will work.
If you can test this, that might be interesting.

I unfortunately won’t be able to do this (at least without significant chaos being unleashed), since it’s my home’s primary control RPI.

I will, however, give my whole system (including router etc) a full-on reboot and will update you if that somehow fixes things :crossed_fingers:

I’m almost sorry to say, but this fixed it for me. Annoyingly I didn’t make it a phased approach, so I can’t tell you if it’s because my RPI with Home Assistant was restarted or if it was because of some funny games on my router, firewall or DNS etc…

So if you’re able to, maybe give your whole network, modem, router etc a restart and see if it works for you too?

Maybe that’s the larger issue we’re talking about:
Capture d’écran, le 2023-03-15 à 11.30.30

@donparlor, @markhinch, @webberle

Well, there was one change in January. Small really, hardly worth mentioning. Merely that our internet provider did a complete upgrade to the gateway technology we use!!! It changed our gateway addressing, which includes the DNS address.

And I forget to update the static DNS address assigned within Home Assistant. DOH!
With that correct, I now have Home Assistant fully up to date.

I’ve no idea why ghcr.io (or anything really) could resolve in the Home Assistant command line prior to the correction, it must have been a false positive.

Short story: if your system’s network interface is set to static values, you might review them to see if you have the same issue.

I had the same error on a RPI running home assistant OS. After a bit of hunting, I noted that the curl error was due to an updated curl package and that the OS may need to be updated. I then noticed that I had inadvertently skipped the 9.5 home assistant os update. Went into settings → system → updates and the three dots on the top right and selected “show skipped updates” and the 9.5 os update appeared. Installed that and the core update then worked (along with the NodeRed update that was also failing). If you are using some other linux distro, you may just need to ssh into the device and manually update the os, eg debian:
sudo apt-get update
sudo apt-get upgrade

Personally, I already tried the DNS addresses hypothesis, and it’s not the root cause. (1.1.1.1 and 8.8.8.8)

I already confirmed it was not the router by removing it from the equation and connecting a laptop directly in the modem.

I already confirmed it was not the modem because I tested the same modem on a different part of the local neighborhood infrastructure and it was working.

My guess is that maybe there is an optical node upstream that is causing the issue. Otherwise I really don’t now what to blame. This is very confusing and unexplainable… Sometimes when the problem is material, these are the hardest problems to pinpoint.

Replying to myself…
Issue seems to be resolved
I was able to add community add-ons again an hour ago

maybe just a hiccup in their system

We also managed to restore connection to Github servers from our network.
So far I am not 100% sure what was the cause, but what we did was accessing the modem UI through Internet Explorer and deactivate the firewall there. This was the only way to access our Technicolor TC4400 admin dashboard… We also set DNS addresses inside Home Assistant but I don’t think this helped in solving the issue since our router already had proper DNS settings (tested with 1.1.1.1 and 8.8.8.8).

I’ve also seen some other users claim Githubs server issues have been solved yesterday, so this adds to the overall confusing situation…

Same problem from China since January. Cant update to the latest. At the beginning, thought it was caused by GFW. But when I tried to get access to http://ghcr.io/ and https://github.com/home-assistant/supervisor/pkgs/container/amd64-hassio-supervisor on my laptop, it’s 100% accessible.

No luck in changing dns to 1.1.1.1.

Folks, I have been seeing this problem for awhile when I attempt to manually trigger an update.
I have tried the DNS update mentioned above which resulted in no change. My understanding is that ha supervisor will automatically update which seems to be happening since I am at 2023.09.2 but when I was attempting to trigger update to 2023.10.0 (version: Supervisor Version 2023.09.2, Ha OS 2023.10.3, HA Core 2023.1.4) it times out.

From the Home Assistance:Terminal I attempt the following:
image
image
This doesn’t look like a DNS issue. Any other thoughts on this topic?

Seems an Internet related latency issue. Try another day to see if it goes away. Your location to the closest GitHub CDN has an issue. Randomly try after a few hours/days.

I’ve been getting this message for days now when trying to update. I’m also getting critical error.

image

I’ve tried the option on the HA page the error directs me to (Supervisor was not able to update - Home Assistant) including changing my DNS this does not help. Like others there seems to be a problem connecting to github??

Any assistance is appreciated.

Here is an excerpt from my log:

Logger: homeassistant.components.websocket_api.http.connection
Source: components/hassio/update.py:260
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 8:57:10 PM (4 occurrences)
Last logged: 9:08:17 PM

[140579869227216] Error updating Home Assistant Supervisor: Update of Supervisor failed: Can't install ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0: 404 Client Error for http+docker://localhost/v1.42/images/ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0/json: Not Found ("No such image: ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0")
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/hassio/update.py", line 258, in async_install
    await async_update_supervisor(self.hass)
  File "/usr/src/homeassistant/homeassistant/components/hassio/handler.py", line 52, in _wrapper
    raise HassioAPIError(data["message"])
homeassistant.components.hassio.handler.HassioAPIError: Update of Supervisor failed: Can't install ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0: 404 Client Error for http+docker://localhost/v1.42/images/ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0/json: Not Found ("No such image: ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 205, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1965, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2005, in _execute_service
    return await cast(
           ^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 235, in handle_service
    return await service.entity_service_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 848, in entity_service_call
    response_data = task.result()  # pop exception if have
                    ^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1192, in async_request_call
    return await coro
           ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 892, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/update/__init__.py", line 156, in async_install
    await entity.async_install_with_progress(version, backup)
  File "/usr/src/homeassistant/homeassistant/components/update/__init__.py", line 414, in async_install_with_progress
    await self.async_install(version, backup)
  File "/usr/src/homeassistant/homeassistant/components/hassio/update.py", line 260, in async_install
    raise HomeAssistantError(
homeassistant.exceptions.HomeAssistantError: Error updating Home Assistant Supervisor: Update of Supervisor failed: Can't install ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0: 404 Client Error for http+docker://localhost/v1.42/images/ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0/json: Not Found ("No such image: ghcr.io/home-assistant/amd64-hassio-supervisor:2023.10.0")

type or paste code here

Turns out it was a storage issue. Ran out of space.

Hello, did you solve the problem? I have the same problem now
Thanks LG