I pull it out, there is no reason to leave it in to possibly spook fish. We are required by law to clearly mark an abandoned hole here in MN. I shove the old blocks back in to the hole and shove a small tree branch or cat tail in the hole to mark it. Snowmobilers don't take kindly to blocks left on the surface for them to hit.