I agree with Matt. Visually I think the main thing that takes the belt to a formal belt is the large shiney waist plate usually found on kilt belts. If you put a less ornate buckle on the same belt it looks so much less formal. That's where the other buckles I'm carrying come in.

The decision maker for me is how often will I wear a formal rig. More times than not, if I am dressing up I will be wearing a waist coat so I won't wear a belt anyway.