Make ZHA forget a device

Yes.
Got to Developer Tools → Services:

Enter the IEEE address of the node an hit ‘CALL SERVICE’

The IEEE address can be seen in the ‘Visualization’ Screen (ZHA configuration dialog). Just enter it without the ‘:’.