Are they loaded on startup, or when they’re needed?
and if they are loaded at startup, can I reload them with a Quick Reload, or do I need to restart HA?
Yes at start up when the integration loads.
If the integration config changed as well - yes definitely. If only the secret value changed (and not the secret name) - then maybe (I’m not sure for all integrations).
e.g. When you reload automations only the automations with changed config are reloaded. So if the secret name did not change or any other config in the automation, and only the value pointed to in the secret then no this would not reload.
Same goes for the template integration I think.