I see what you mean about the drawing. Try thinking about it this way. Take a regular sheet of paper, and turn up a fold about an inch at the bottom. Turn the paper so that the folded edge is at the bottom and the flap is on the side you can see. The line in the drawing is the edge of the paper (not the fold) in the flap you folded up.

So, in kilt terms, you turn up a flap toward the inside of the kilt and baste it so that it stays put. Then, you hold the kilt so that you're looking at it from the inside, with the folded edge toward you and the top of the kilt away from you. The herringbone stitch is done catching first a little piece of the folded part and then a little piece of the kilt, then a little piece of the folded part, and then a little piece of the kilt. So, you're making an "X" across the edge that's folded up.

The advantage of using a herringbone stitch as opposed to a blind stitch or a hem stitch is that you aren't stitching tightly at the folded-up edge and you don't wind up with a little ridge that shows from the front. It's also a bit flexible.