Possibly the pleats which are drifting open need to be slightly lifted at the waist to pull them shut.

I think it depends on just where the last pleat falls on the curve of the body which determines if it can lie flat or not without a bit of tweaking of the grain line.

The easiest cure is to sew down the free edge of the pleats slightly above the rest, so a triangle of double fabric then needs to be smoothed down over the raw edge of the waistline and is covered over by the waist band.

This will pull the pleat forward, so you might simply transfer the problem to the next pleat back. The smallest lift which will do the job is advisable, otherwise you need slightly more fabric in the kilt and a lot of gathering and pressing the waist in to narrow the kilt as much as possible above the hips.