Change HAOS docker MTU (1500 > 1450)

Installation Home Assistant OS
1

Hi everyone,

I hope, this is the right category for my question. I freshly installed HAOS on a cloud server (please no discussion about if this makes sense or not ^^) and now get an update notification from HAOS 10.2 to 10.3. However, if I cluck on “install” it works for a while, but nothing happens.

When I try “ha os update” on the CLI, I get “processing” for a while and then the following error:

Error: Can't fetch OTA update from https://github.com/home-assistant/operating-system/releases/download/10.3/haos_generic-x86-64-10.3.raucb: Connection timeout to host https://objects.githubusercontent.com/github-production-release-asset-2e65be/115992009/4156f348-1794-46ca-af57-66ccf04071e1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20230621/us-east-1/s3/aws4_request&X-Amz-Date=20230621T065716Z&X-Amz-Expires=300&X-Amz-Signature=94ce94ae42dcd75dc659bb3493e41ffb6c66c9a1a5fe96fc2060be7eee9500f1&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=115992009&response-content-disposition=attachment%3B%20filename%3Dhaos_generic-x86-64-10.3.raucb&response-content-type=application/octet-stream

My network connection looks ok-ish, ping and nslookup work like a charm.
WGET for this file on github works, too.

But “ha network info” shows this:

docker:
  address: 172.30.32.0/23
  dns: 172.30.32.3
  gateway: 172.30.32.1
  interface: hassio
host_internet: true
interfaces:
- connected: true
  enabled: true
  interface: enp0s10
  ipv4:
    address:
    - 10.106.0.6/32
    gateway: 10.106.0.1
    method: static
    nameservers:
    - 8.8.8.8
    - 1.1.1.1
    - 8.8.4.4
    ready: true
[...]
supervisor_internet: false

The problem seems to be that docker is using an MTU of 1500 per default, whileas my cloud server requires 1450 or less.

2: enp0s10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel state UP qlen 1000
    link/ether 86:00:00:4c:9b:a7 brd ff:ff:ff:ff:ff:ff
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:07:a1:62:70 brd ff:ff:ff:ff:ff:ff

Therefore it is possible to ping the update address from within the supervisor container, while it is not possible to wget the update file:

bash-5.1# ping objects.githubusercontent.com
PING objects.githubusercontent.com (185.199.108.133): 56 data bytes
64 bytes from 185.199.108.133: seq=0 ttl=54 time=7.974 ms
64 bytes from 185.199.108.133: seq=1 ttl=54 time=4.865 ms
64 bytes from 185.199.108.133: seq=2 ttl=54 time=4.629 ms
64 bytes from 185.199.108.133: seq=3 ttl=54 time=4.735 ms
64 bytes from 185.199.108.133: seq=4 ttl=54 time=4.662 ms
^C
--- objects.githubusercontent.com ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 4.629/5.373/7.974 ms

bash-5.1# wget https://github.com/home-assistant/operating-system/releases/download/10.3/haos_generic-x86-64-10.3.raucb
Connecting to github.com (140.82.121.4:443)
Connecting to objects.githubusercontent.com (185.199.108.133:443)
<HANG>

pfSense Packet Capture:
17:41:16.825692 IP 10.106.0.6.59488 > 185.199.108.133.443: tcp 0
17:41:16.829946 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 0
17:41:16.831193 IP 10.106.0.6.59488 > 185.199.108.133.443: tcp 0
17:41:16.859241 IP 10.106.0.6.59488 > 185.199.108.133.443: tcp 331
17:41:16.863313 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 0
17:41:16.865745 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420
17:41:16.865768 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420
17:41:16.865774 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 972
17:41:16.866412 IP 10.106.0.6 > 185.199.108.133: ICMP 10.106.0.6 unreachable - need to frag (mtu 1450), length 556
17:41:16.866440 IP 10.106.0.6 > 185.199.108.133: ICMP 10.106.0.6 unreachable - need to frag (mtu 1450), length 556
17:41:16.866658 IP 10.106.0.6.59488 > 185.199.108.133.443: tcp 0
17:41:16.885585 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420
17:41:16.885809 IP 10.106.0.6 > 185.199.108.133: ICMP 10.106.0.6 unreachable - need to frag (mtu 1450), length 556
17:41:17.125617 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420
17:41:17.125911 IP 10.106.0.6 > 185.199.108.133: ICMP 10.106.0.6 unreachable - need to frag (mtu 1450), length 556
17:41:17.597619 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420
17:41:17.597954 IP 10.106.0.6 > 185.199.108.133: ICMP 10.106.0.6 unreachable - need to frag (mtu 1450), length 556
17:41:18.557753 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420
17:41:18.558029 IP 10.106.0.6 > 185.199.108.133: ICMP 10.106.0.6 unreachable - need to frag (mtu 1450), length 556
17:41:20.445729 IP 185.199.108.133.443 > 10.106.0.6.59488: tcp 1420

From within the homeassistant container (running on the host network), both ping and wget are successful.

Any idea, how I can change the HAOS docker´s MTU from 1500 to 1450 and persist that setting? I tried and search pattern on Google I could think of, but wasn´t successful. Creating /etc/docker/daemon.json wasn´t successful, as the file doesn´t persist after a restart.

Thank you in advance,
Joachim

1 Like
2

Ummm… if you can get access to your HAOS shell (virtual console or ssh root@ADDRESS -p22222), then maybe try
ip link set mtu 1450 docker0
just to see if that (temporarily) solves the problem.
I’m also not sure, but you may have to bounce the link to get the new MTU setting to take

ip link set down docker0
ip link set up docker0

[EDIT] I probably need to do more research on linux kernel bridge (which is what docker0 is) as it has the various docker containers attaching to it, and changing the MTU of docker0 may affect the containers too;

1 Like
3

Thx a lot. That wasn´t the solution (because it doesn´t affect the MTU setting within the supervisor container), but it pushed me in the right direction. :grinning:

docker exec -it hassio_supervisor bash
ip link set mtu 1400 eth0

That does the trick. Afterwards wget and “ha os update” work like a charm.
Now I only need a possibility to persist that setting. Maybe I need to add an issue to HAOS github?

4

I’m thinking that anything (HA-Core, AddOns, Supervisor) that wants to send out on enp0s10 is going to have this issue, so if you do post something, whether its a problem or Feature Request, you may want to the MTU size configurable across all interfaces.

1 Like