A) There is no built-in backup tool in HA, correct?
B) Can I move the stick around and use it in a Windows box while doing the backing and put back to the RPi that runs HA without breaking anything?
The stick holds your zwave config. Network layout, nodes, their neighbors, encryption key etc etc. Without a backup you would be running around having to include every device once again.
Would be great if it was possible from HA itself, but I guess that is rather and edge case for someone to develop as the Windows tool would have to be re-engineered.