The grey tweed is better, but to really work with the brown you should get a lovat green or lovat blue jacket (either will work with that tartan as well though green would probably be better for the saffron.)

Adam