It's definately not an issue. There are plenty of people who wear solid colour kilts at games. I haven't seen many solid colour traditional kilts but the modern ones are quite common. If you are a member of a clan, they may tease you about not wearing the clan tartan but that's about it.