Let me ask you this question. Would you wear a tailcoat or a mess dress jacket in a non formal setting? I would not. These are formal jackets meant to be worn with black or white tie. But can you wear it in a non-formal setting? Sure, no one is going to stop you. But those "in the know" will just be polite and keep their comments to themselves.

This is why a black Argyll or similar cut of jacket is the best option if you are only going to have one jacket.