From a Scottish perspective, many servants historically took the surname of the laird whom they served, so I would see no objection to anyone wearing the tartan of the family who adopted them.