Check this out, only devices with DIY mode allow direct OTA updates. The devices you mentioned aren’t on the [short] list (Basic R3, RFR3 and Mini).
Other links you might want to check out:
https://esphome.io/devices/sonoff.html
https://devices.esphome.io/devices/Sonoff-TH10
https://devices.esphome.io/devices/Sonoff-THR316
If you use Home Assistant, and a device is ESPHome-compatible, it’d say the migration is a no-brainer; removing third-parties is my favorite pastime. Plus what indeeed said…