NakedSecurity has a nice article on the current results of the CanSecWest PWN2OWN contest where attackers target popular web browsers and companion products for some public shaming. Between PWN2OWN and PWN4FUN, all 4 major web browsers (IE, Firefox, Chrome, Safari) exhibited security holes, with Safari even giving up privilege escalation into root.
Running IE is still a riskier position than running another browser (tempting attack surface, integration into OS, difficulty implementing user-gated authorization of scripts). But the takeaway from events like PWN2OWN is every browser has issues. Users still need to browse the web with care and turn off globally allowing scripting and other packages, no matter which particular web browser they use.
I always get crap for how web pages look in my browser as I disable so many things that sites want to load, but at least I have a little bit more assurance in the added security of my web browsing.