Huge delay when triggering switchbot integration via alexa

So, I have a switchbot on my intercom to let people in. Triggering via the ap is instant. Triggering via HA is maybe 1-2 seconds, not bad at all.

But when I ask alexa to run a script that toggles the “switch” or a scene to manually on then off it, it’s maybe a 10 second delay, which is usually enough for the doorbell to time out. (and feels like an eternity when the doorbell is ringing)

Is there a way I can reduce this latency somehow? Alternitively, would there be a way to detect the sound/video somehow to trigger an automation?