HA sidebar, reverse proxy and Authelia - How to keep services secure and still be able to access them from HA?

I have a problem that I don’t see how to solve.
I have HA as a container, and some additional services linked to HA (Music assistant, Rhasspy, Zigbee2MQTT). Those are separate containers. I want to have them in the sidebar in HA. This is done.
Problem : when HA is accessed through https (I am using Caddy as my reverse proxy), all iframes need to be https too. Alright, I can do that and reverse proxy those services too. But I can’t have Rhasspy, Zigbee2MQTT… accessible without any password. So I’ve added Authelia, with forward auth, to secure all this.
Problem : Now I need to enter Authelia login when accessing the tools from the sidebar. Which is anoying, as I’m already logged in HA. I can’t ignore Authelia as it would make my services insecure.

What would be my way out of this ? How to expose those services, keep them secure, while still being able to access them without password from HA ?

I don’t see how to set this up.
Any answer, any idea is welcome. Have a great day.

Would you have an idea on how to solve this issue ?