ESPHome OTA Updates Over Tailscale VPN — Solved with `tailscale serve`

ESPHome OTA Updates Over Tailscale VPN — Solved with tailscale serve

TL;DR: If your remote ESPHome device is behind a Tailscale VPN and OTA updates
show “Update queued. It will install when the device comes back online” forever,
run this on the Tailscale node closest to the ESP32:

sudo tailscale serve --bg --tcp 3232 tcp://<device-local-ip>:3232

The Setup

I have an ESP32 running ESPHome deployed at a remote location — a gated community
entrance ~500m from my house — scanning for BLE beacons to detect when family cars
approach.

The architecture:

  • ESP32 runs ESPHome with MQTT (no native API). Static IP 10.42.0.100 on the
    Pi hotspot subnet.
  • Raspberry Pi 5 creates a WiFi hotspot for the ESP32, connects to the internet
    via a 4G router, and runs Tailscale (Tailscale IP: 100.a.b.c).
  • Home Assistant (HAOS on Proxmox) runs ESPHome and Mosquitto.
    Tailscale IP: 100.x.y.z.
  • Tailscale DERP relay (São Paulo) bridges the connection — the 4G router uses
    symmetric NAT which blocks direct peer-to-peer Tailscale connections.

Everything worked perfectly: MQTT sensor data flowed from the ESP32 to HA, automations
fired, BLE beacons were detected. Except one thing: OTA firmware updates were
impossible.


Why OTA Failed

ESPHome’s OTA update process works like this:

  1. ESPHome compiles the new firmware
  2. It publishes a discovery request to esphome/discover/{device_name} via MQTT
  3. The ESP32 responds with its local IP address10.42.0.100 in my case
  4. ESPHome tries to connect directly to that IP on port 3232 to push the firmware

The ESP32’s IP is on the Pi’s private hotspot subnet — completely unreachable from
Home Assistant across the VPN. The result:

“Update queued. It will install when the device comes back online.”

It never will. The device’s IP is behind a NAT that HA cannot cross.


What I Tried That Didn’t Work

:cross_mark: iptables DNAT

sudo iptables -t nat -A PREROUTING -i tailscale0 -p tcp --dport 3232 \
  -j DNAT --to-destination 10.42.0.100:3232

Why it fails: Traffic arriving on a Tailscale IP (100.a.b.c) is processed by
Tailscale’s userspace networking stack before it ever reaches the Linux kernel
network layer. iptables never sees it. The DNAT rule is in the right place for
normal traffic — but completely invisible to Tailscale-destined packets.

Confirmed with tcpdump -i tailscale0 port 3232: the SYN packet arrives on the
Tailscale interface, but socat and iptables never receive it.


:cross_mark: socat relay

socat TCP-LISTEN:3232,fork,reuseaddr,bind=0.0.0.0 TCP:10.42.0.100:3232

Why it fails: Same fundamental problem. socat listens on the Linux network
stack. Tailscale consumes the packets before they get there.


:cross_mark: ESPHome use_address override

wifi:
  use_address: 100.a.b.c  # Pi's Tailscale IP

Why it fails: use_address works for native API connections, but for MQTT-based
devices ESPHome uses the IP from the MQTT discovery response — which is the ESP32’s
actual hotspot IP (10.42.0.100), not the Pi’s Tailscale IP.


:cross_mark: Tailscale subnet router

sudo tailscale set --advertise-routes=10.42.0.0/24

Approved in the Tailscale admin console. The route appeared correctly on the Pi.

Why it fails: HA’s Tailscale add-on runs in userspace networking mode inside
a Docker container on HAOS. In this mode, no tailscale0 interface is created on
the host. Even with accept_routes: true in the add-on configuration,
ip route show table 52 shows nothing. The subnet route is advertised but HA
cannot use it to reach 10.42.0.x.


The Root Cause

All four approaches fail for the same reason:

When a packet is destined for a Tailscale IP address (100.x.x.x), Tailscale’s
userspace stack intercepts and processes it before the Linux kernel network stack
sees it. Any tool that operates at kernel level — iptables, socat, routing tables —
is completely blind to this traffic.


The Solution: tailscale serve

tailscale serve is Tailscale’s native port forwarding feature. Because it operates
within Tailscale’s own userspace stack — not at the kernel level — it actually
intercepts the traffic and can forward it.

On the Raspberry Pi, run:

sudo tailscale serve --bg --tcp 3232 tcp://10.42.0.100:3232
  • --bg runs it as a background service, persistent across reboots
  • --tcp 3232 listens on the Pi’s Tailscale IP, port 3232
  • tcp://10.42.0.100:3232 forwards to the ESP32’s static IP on the hotspot

Verify:

sudo tailscale serve status

Expected output:

|-- tcp://your-pi-hostname.tail-xxxxx.ts.net:3232 (TLS over TCP, tailnet only)
|-- tcp://100.a.b.c:3232
|–> tcp://10.42.0.100:3232

After this, ESPHome OTA works with a single click from the dashboard, from
anywhere in the world. No USB cable. No physical access required.


Complete ESPHome Configuration

The ESP32 uses MQTT instead of the native API — there is no direct connection
from HA to the ESP32’s IP, so the native API cannot be used:

esphome:
  name: remote-ble-scanner
  friendly_name: Remote BLE Scanner

esp32:
  board: esp32dev
  framework:
    type: esp-idf

# No 'api:' section — MQTT replaces it
# OTA still works via the standard ESPHome platform
ota:
  - platform: esphome
    password: !secret ota_password

mqtt:
  broker: 100.x.y.z          # HA's Tailscale IP
  port: 1883
  username: esp_remote_device
  password: !secret mqtt_password
  birth_message:
    topic: remote_device/status
    payload: online
  will_message:
    topic: remote_device/status
    payload: offline

wifi:
  ssid: "PiHotspot"
  password: !secret hotspot_password
  # Static IP — required for stable port forwarding
  manual_ip:
    static_ip: 10.42.0.100
    gateway: 10.42.0.1
    subnet: 255.255.255.0
  # Tell ESPHome to reach this device via the Pi's Tailscale IP
  # The Pi forwards port 3232 to us via 'tailscale serve'
  use_address: 100.a.b.c
  ap:
    ssid: "Fallback Hotspot"
    password: !secret fallback_password

Note: use_address tells ESPHome where to connect when pushing OTA or
streaming logs. Combined with tailscale serve forwarding port 3232 to the
ESP32, this creates the complete OTA path.


Complete Raspberry Pi Setup

# 1. Install Tailscale
curl -fsSL https://tailscale.com/install.sh | sh

# 2. Connect to your Tailscale account
sudo tailscale up

# 3. Create WiFi hotspot for the ESP32
sudo nmcli con add type wifi ifname wlan0 con-name hotspot \
  autoconnect yes ssid "PiHotspot"
sudo nmcli con modify hotspot 802-11-wireless.mode ap \
  802-11-wireless.band bg ipv4.method shared
sudo nmcli con modify hotspot wifi-sec.key-mgmt wpa-psk \
  wifi-sec.psk "your-hotspot-password"
sudo nmcli con up hotspot

# 4. THE KEY STEP — forward OTA port inside Tailscale's stack
sudo tailscale serve --bg --tcp 3232 tcp://10.42.0.100:3232

# Verify
sudo tailscale serve status

Important: Dedicate wlan0 entirely to the hotspot. If the Pi also connects
to WiFi as a client on the same interface, the radio will conflict and both
connections become unstable. Use Ethernet for the Pi’s own internet access
and reserve wlan0 for the ESP32 hotspot.


Mosquitto MQTT Configuration

The ESP32 authenticates with Mosquitto using a dedicated Home Assistant user account.
On HAOS with the Mosquitto add-on, create a user at Settings → People → Users
with “Can login” unchecked. No Mosquitto configuration changes are needed — the
add-on automatically accepts HA user credentials.

Test connectivity from the Pi before deploying:

mosquitto_pub -h 100.x.y.z -p 1883 \
  -u "esp_remote_device" -P "your-password" \
  -t "test/ping" -m "hello"

Cellular NAT Behavior

This setup was tested with a 4G MiFi using symmetric NAT — one of the strictest
NAT types. Tailscale falls back to DERP relay servers automatically:

pong from homeassistant (100.x.y.z) via DERP(sao) in 156ms

DERP relay adds ~150–200ms latency (São Paulo for Chile). This is acceptable for
both MQTT sensor data and OTA firmware uploads.


Why This Isn’t Documented Anywhere

Most ESPHome remote OTA guides assume either:

  • The device is on the same LAN as HA (native API, no problem)
  • The device has a public IP (port forward on the router)

The combination of cellular NAT + Tailscale VPN + Pi hotspot creates a situation
where all standard Linux networking tools fail for a non-obvious reason: Tailscale’s
userspace packet processing intercepts traffic before the kernel network stack.
tailscale serve is the only tool operating at the right layer to solve it.


Troubleshooting

OTA still shows “Update queued”:

  • Verify tailscale serve status shows the forwarding rule on the Pi
  • Confirm use_address in the ESPHome config matches the Pi’s Tailscale IP exactly
  • Confirm the ESP32 has a static IP matching the forwarding target (10.42.0.100)
  • Check ESPHome logs: it should show Connecting to 100.a.b.c port 3232...

MQTT not connecting:

  • Test from the Pi: mosquitto_pub -h 100.x.y.z -p 1883 -u user -P pass -t test -m hello
  • Check the HA Mosquitto add-on logs for authentication errors

Tailscale serve not persisting after reboot:

  • Use the --bg flag — this is what makes it persistent
  • Verify after reboot with sudo tailscale serve status

Hotspot dropping when Pi connects to WiFi:

  • The Pi’s single WiFi radio cannot simultaneously act as client and hotspot reliably
  • Connect the Pi to the internet via Ethernet and reserve wlan0 for the hotspot only

Hardware Used

Component Model Purpose
SBC Raspberry Pi 5 2GB VPN gateway + WiFi hotspot
Microcontroller ESP32 DevKit ESPHome BLE scanner
Router 4G MiFi (any) Cellular internet
Switch Ubiquiti Flex Mini LAN expansion (optional)

TL;DR — The One Command

sudo tailscale serve --bg --tcp 3232 tcp://<esp32-local-ip>:3232

Run this on the Tailscale node that shares a local network with the ESP32.
Done.


Developed through trial and error. The tailscale serve solution is not
documented for this specific use case in either the ESPHome or Tailscale
official docs. Hopefully this saves someone the debugging time.

1 Like