I know the garment doesn't care, but I would feel weird. It's the same reason I don't wear a kimono, non quai thao, ushanka, or other ethnic clothing that I have absolutely no ties to. I would feel like an impostor. Not saying I take issue with anyone else doing it, but it would just make me personally feel like I'm trying to be something I'm not. Especially without a "when in Rome" excuse.