Framework on ESP32 boards; arduino or esp-idf?

I have multiple ESP32’s running in ESPHome with the framework; arduino. I just started using esp-idf as framework on a script for a Bluetooth Proxy. Would it be better/necessary to change all my other boards to esp-idf?

I don’t see any reason to change well working setups.
Also, not all components/options work on esp-idf while others only work there.