The Wiki entry probably says as much as is known. They definitely go back to Victorian times.

https://en.wikipedia.org/wiki/Kilt_pin