I am running Home Assistant on a Raspberry Pi 4.
It looks like you were right! I loaded up HassOS on a virtual machine, installed the ESPHome add on, created and uploaded the firmware with your config and it seems to be working now.
Hopefully an update will come out that will allow this to work on arm processors.
Thanks for your help.