I have a 1955 F250 that I installed a 1976 460, over time the contacts on the solenoid got pitted, occasionally while trying to start it would hit a bad spot and just click until it found a good spot and start. I pulled the starter and cleaned up the contacts with sandpaper and put back together. Everything was fine until today when I went to start it, the starter hung up and continued to run, I had to disconnect the battery in order to stop it. Could I have done something to the contacts when I cleaned/sanded the contacts?

 

I'd get a new one.

 

That happens occasionally when the battery goes low (the starter relay will stick)
Some melt and weld themselves together
In that case the starter relay needs to be replaced
Good luck, have fun