I have long worn tweed jackets (with trousers/jeans - I don't yet have a tweed kilt jacket) for 'everyday casual' wear) and no one seems to consider me remotely overdressed. I wear a sweater with my kilt(s) in cooler weather, and I have a lined leather bomber jacket for when it gets cold - that and a sweater have worked for me, kilted, at -20+C. But a good tweed jacket would be wonderful for simply being out and about in a kilt, as far as I'm concerned.