Something you can do is cut off the pom and use velcro to reattach it. This also allows you to get other poms from the hobby store and attach ones that match your outfit. (I have both red and black poms for mine, and may add a green one.)

A good wool balmoral does help keep your head warm. I wore mine all last winter and enjoyed it a lot, although it doesn't do a lot for your ears.