I am not sure if the phrase "conservative" carries the same connotation this side of the Atlantic or not.

If you mean "suit and tie" etc that would be a form of dress in business hours or for semi-formal occasions and they come off at home in favour of shirt or trousers or t shirt and jeans.

If you mean "conservative" as an expectation of bifurbication for men that's a different kettle of fish and is more to do with either ignorance of the possibilities of male clothing or, whilst being aware that they exist, not being used to seeing them.

I think the "boy bits and how one dresses" argument is a bit of a red herring given that only really tight clothing might carry a danger of providing information wanted or unwanted - I see similar arguments regarding the need for a sporran or not.

It's more a question of what people are used to seeing and for many a kilt is a new concept to get used to but my experience is that they do and that compliments are far more likely than negative comment and interested intelligent questions will be asked.