A Google Webmaster Help thread has one webmaster upset about someone trying to link spam their site in order to downgrade their rankings.
We ran a poll and most SEOs believe competitors can hurt you with links but still, many do not believe it. And that is the overall opinion in the thread. Most are saying, the links won't hurt, the links will just be devalued by Google.
What do you think? Not do you think they can hurt but rather, do you worry about it?
In fact, I had a case where Square Peg Web Design was so upset with me that they tried to do all sorts of things to hurt my company, including building nasty links to my site. I wasn't too worried about it, although it encouraged me to keep my post up about what they did to my company and clients.
Forum discussion at Google Webmaster Help.