I use elastic bands on the back of my kilt pins, it works, and does not damage the kilt.
Here is a picture of what I mean: