The weather might have something to do with it.
Many people fail to realise just how far it is travelling from the south coast of England to the north of Scotland, and the difference in the weather and climate before you even get to Hadrian's wall.

Down here on the coast where I now live there are grapes growing outside, we have outdoor seating in many places, with a roof over as it does rain a bit, but with a few modifications it is possible to live a very Mediterranean lifestyle.

In the Midlands - well - Roman soldiers would probably want to add leggings to their uniforms, and write home asking their mum to send them some vests and thick socks.

Further north still, but still in England, the whole legion would probably be complaining about their feet, the cold and the rain and the wind, and how everything goes rusty, mouldy or just falls apart. They'd probably be volunteering for anything which would give them a reason not to venture further north, such as building a wall.

Another factor could be how slowly trees grow in Scotland, and once cleared woodland doesn't regenerate on its own. That could have made a significant difference to the effort required to construct just about anything.

Anne the Pleater