I don't think you can change that thinking. I have seen lots of article about confronting outright objections to kilt wearing, but not much about the "normal" aspect.

About all you can do is just keep wearing the kilt, and hope that familiarity will eventually set it.