Two choices - Sand or file the pin smooth again or take a needle and make a hole in the cloth before putting the kilt pin through.

More reason to do what Hamish says and not change the pin.