I am not really offended by the terminology so much as the tone. I have been approached by people who call it a "skirt" and, technically, they are right. I may or may not inform them that it is specifically called a "kilt" but I always thank them for the complement.

But when the comment is delivered with a sneer or other obvious sign of contempt, I respond in kind.