How is the back of your sporran strap finished?

The first one I ever made I left 'raw' smooth veg-tan. I noticed that after a few days of wearing it, my kilt was starting to show a little 'raised fibre'. Ever since then I have always burnished the backs with beeswax for a very smooth, polished finish. That was the end of the problem. I think raw leather is too 'grippy' for constant wear against wool or pv.

The other issue may be the edge of your strap, if not burnished and rounded, the square-cut edge may be dragging across your fabric too much and causing the issue.

The chair... cannot help you there.