Like the tree falling in the forest, is the cheater whose cheating is thwarted still a cheater?
I would take it down. The would-be cheaters will still be “punished” insofar as they’ll get a bad grade (presuming they were counting on accessing the website).