Just to offer some ideas for the fun of it, and of course depending on the level of formality...

I think an outfit something like the ones below would look nice and could allow for a baldric to be incorporated without the styles clashing. Granted a couple of these lads are wearing doublets but a couple are wearing tweed jackets. I personally like the pipers plaid look worn loosely across the chest for a "dressy-casual" look. I think it's a nice variation on the plaid being worn in the typical fashion.