I always wear my Sgian Dubh to the Kirk'n services I attend, Everyone else in highland dress wears them. I am a Presbyterian too, everyone seems to understand that it's part of highland dress.

I suppose it is no different than carrying a pocket knife in your pocket ar a small knife on a keychain into church with you.