I think you handled the situation well. A few years ago I had a confrontation with a Middle Eastern fellow who, while passing me on my way to my car asked, "What the H*** is THAT?" pointing to my black workman's Utilikilt. I got out a card and handed it to him saying something like, "Here's the card of the company that makes them, you can read all about it on their website." He slapped the card out of my hand and threatened violence against me if I tried something like that again. I just shrugged and walked away.

Roll forward two weeks later. I run into the same guy at the Student Union. He's all apologetic and we get into a discussion about traditional male garments from his culture and others. He said that after he got home the night of his tirade against me he realized that his own traditional clothing might be viewed as women's clothing by someone who didn't know better.