Sometimes you have to fight back - maybe "fight" is not the correct word - stand up for yourself is better. If you don't, you give the other person power over your life, which gives them control. Sounds to me like you responded with the right amount of force. Well done.