The one time I wore a kilt (borrowed, unfortunately not my own, or it'd have been more than once), I went about my day as normal. Most comments made about the kilt were highly complimentary ones, and only a couple were negative ones regarding the "skirt" I was wearing, which I quickly and effective disarmed with an explanation of why the kilt is more masculine than a pair of pants.

I never felt threatened, afraid, etc. while experiencing the comfort and freedom of the kilt, but people generally don't mess with me anyway. I've been told that I can look scary. People can't always tell when I'm smiling, so imagine what a straight face looks like on me.