I think I solved it myself.
For one I deleted the entry in config/.storage/core.device_registry. There still was a MariaDB entry in there (also an entry for the uninstalled phpMyAdmin.
Not sure if this is crucial, but the error in the log seems to have disappeared.
More importantly:
The move to UI seems to be incomplete for MariaDB integration.
you have to select “Edit YAML” if you want to use secrets. The exact same code works if you copy it into the edit in yaml mode.