The Bluetooth range of a Raspi 4 is pretty poor, especially if it is mounted in an aluminum case (less than one meter for me).
So for the initial pairing bring your device as close as possible to the Raspi.
Then use a Bluetooth Proxy (using ESP32 or Shelly Plus devices) for the operation.