I had the same issue due to a rather noticeable swelling in my abdomen

I solved it by buying the sporran hanger that go on your belt. This allows the sporran to swing off the belt and not disturb the apron.