As an US Army Officer, my opinion is that as long as you wear it with respect I see no problem with wearing it. Most people won't know the significance of the tartan or the military insignias and the ones that do will most likely be impressed or honored. While not necessary, you should include your army cadet insignia or something from your grandfather in your outfit, along with the other insignias, to make it personal.