Google Assistant SDK being slow to respond to commands

Hi,

I am currently using Google Assistant SDK to turn on and off my lights as their own API is not very good and is rate limited,

Not too long ago it started slowing down alot and now it takes over 1:20 to send a instruction,

Is anyone else getting this and is there a way i can fix it?

Thanks, Harry