The Prince Charlie is almost always worn with a 3 button waistcoat, which should not be worn with a belt, for the reasons Jamie enumerates above.

I've also seen Prince Charlies worn with a cummerbund, which also looks nice, and which also would prohibit the wearing of a belt, for the same reasons.

Basically, if you are wearing anything that's going to cover your waist (top of the kilt), like a vest or a cummerbund, leave the belt at home.

That being said, I see no reason why you couldn't wear the Prince Charlie jacket with a belt, sans waistcoat/cummerbund. It's just not normally worn that way.