Trying to remove Sonoff stick after migration to SLZB-06 crashes HA

Configuration Zigbee
1

I moved my Zigbee devices from ZHA on a Sonoff USB-stick to Z2M on a SLZB-06.
I reconnected all devices to Z2M and everything is working well now in Z2M.

Next I like to remove the USB-stick and stop ZHA.
I stopped ZHA from adding new devices. The only device on ZHA is the Sonoff stick itself.
However, when I try to remove the Sonoff, or even disable it in ZHA, HA starts crashing. The only solution then is to reconnect the Sonoff stick and restore HA.

How do I get rid of the Sonoff stick?

P.S. I’m running HA on Proxmox on a NUC

2

Did you remove the USB pass through from Proxmox ?

3

Uhm, I think so. What is the correct sequence of actions?

I started with the deletion of the usb device from ZHA, but then my system started rebooting and crashing.