I would definitely say do not roll down kilt hose. Here in Japan, it used to be the fashion (and there are still some stragglers) for high school girls to wear "loose socks" which if extended would go well above the knee, but they bunched them up much lower. Add to that fact that a lot of the high school girls uniforms are plaid pleated skirts, and if you wore your kilthose rolled down you would probably look like a Japanese high school girl.

As someone mentioned high boots with normal socks look good (see Samuel L. Jackson in The 51st State aka Formula 51 aka Chemical 51). However, I would just call that casual... not "casual smart."

I think that the best idea is, as some have mentioned finding calf high socks such as soccer or rugby socks... though it may be hard to find any without stripes or corporate logos on them.