I’ve been working on numerous disjointed projects lately and found a way to bring them together by making an ESPHome ESP32 device work with a browser and touch! Since it uses my CastKit software that wires into Home Assistant with MQTT, Home Assistant can control what it displays as well!
The display I had lying around
It doesn’t matter which device or display you use, but this one is a 480×320 ESP32-S3 device with capacitive touch. The display quality is awful, but it gets the job done. I configured it to work with my Rip Deck software that lets me monitor and control ripping a bunch of discs at once:
What it looks like up-close
Fake data showing all states on the device itself
The same web view taken from a browser
This is showing the kiosk mode I added to it last night that also works with CastKit. This is what I mean about bringing two unrelated projects together.
I originally picked up this “WT32-SC01 Plus” (2 MB PSRAM) for a 3D filament weigh and scan project, but it ended up not working out for me, so I had it just lying around. Last night, I decided to use it for my Optical Ripper Tower since it sounded like a much better idea than the tiny LED strips I was working on.
It also took only 2 nights to get it working including test-printing the case with whatever random colors I had on hand (the AI agent chose them).
How it works
Chrome isn’t running on the ESP32, it’s running on my NAS in a Docker container, and it’s streaming images from Chrome to the ESP32 (with touch compatibility), effectively making it look and feel like a real browser. Way better than coding the UI in C from scratch! I’m a web engineer by trade, so this is exactly the way I wanted it built.
Technical Details
The way this works is CastKit runs a persistent Chromium session on another device (my NAS) in Docker at the display’s exact viewport size:
1.Converts each browser frame to RGB565.
- Compresses it with
zlib. - Sends it to the device through the encrypted ESPHome native API.
- ESPHome decompresses the frame and draws it on the screen.
- Touch coordinates travel back to CastKit, which “clicks” them in the browser.
Touching is laggy, so I also found a workaround using optimistic updates to make touch respond instantly! More on that below.
From the user’s side, it behaves like a browser running on the display:
web application
↓
CastKit worker → Chromium at 480×320 → RGB565 + zlib
↓ ↑
encrypted ESPHome API touch coordinates
↓ ↑
WT32-SC01 Plus display ──────────────────┘
Dealing with touch lag
The main problem was perceived latency. A touch could reach Chromium quickly, but the user still had to wait for the next rendered frame to cross the network and draw.
I handled that with an optimistic frame cache.
Each application can provide a loading or skeleton page in a small JSON manifest. CastKit renders that page once at startup and preloads the bitmap into the ESP32’s PSRAM.
When the user taps something, the firmware can draw the cached skeleton immediately while Chromium loads the real destination. Now, it feels instantaneous. You don’t even notice the loading screen. It literally takes less than half a second, but it felt super laggy without this feature.
CastKit also binds each touch target to the browser frame that ESPHome actually displayed. If the frame becomes stale, the device disables touch instead of sending a command against a newer, unseen page.
Rip Deck supplies its own kiosk view that can scale to any size as well as a manifest. CastKit is just a proxy loading a URL and some metdata. Nothing special except its ability to serve pages as images.
Home Assistant via MQTT
Because we’re using CastKit, it supports Home Assistant via MQTT just like any other regular CastKit display.
I set this up to only show the ripping stuff when the tower is on. If it’s off, it prioritizes showing a custom media player view that I have hooked up with Music Assistant and the falls back on the time/date/weather w/ an added agenda (calendar events list) if stuff is coming up that day.
My ePaper and other small media screens already do this, but all of those use Raspberry Pi Zeros. This whole concept of sending images was to make it easy to program a Pi Zero (which absolutely sucks to use because it’s slooooow). Now, that same code powers an ESP32 device in the same exact way!
Because it’s using CastKit, I can reprogram the panel on-the-fly without having to flash any firmware!
I haven’t tried doing any animations/video work yet, but that’s something I’ll be looking into at some point. It was streaming images at 60fps, but sending those to the panel at that speed is a different issue entirely.
Sources
The CastKit remote-display setup and protocol are documented here:
I’m curious if anyone is either using CastKit or has a similar setup.