Delete uptime_robot binary sensors when configuration changes

I have been the uptime_robot component for a while. Today, I needed to delete a couple of monitors (using the uptime_robot website), and now HA is filling the log with errors because it is still asking the API about those monitors.

How do I get HA to forget about the deleted monitors?

Gareth