I have a sportkilt that I use for exactly the same purpose--the kilt equivalent of sweatpants. It stays inside mostly, and I do just like you do with the elastic. My wife thinks it's hideous, but it does make her appreciate the aesthetic of regular kilts more.