Google's John Mueller said in a Google Webmaster Help thread said that if you have bad or spammy links pointing to 404 pages, it should have no negative impact on your overall site's rankings.
John said not only should it not happen, he has never seen it happen. He said:
To be clear, I have never seen a case where bad links pointing to URLs that return 404 have ever caused a website any noticeable problem in web-search. 404s are a part of the internet, they're expected to be seen when a non-existent URL is crawled, there's no reason that I can think of where it would make sense to count 404s against a site.
What if you do find a case? Then John says, let him know.
That said, if you ever feel that you've run across a case where it is really causing a problem, I'd love to take a look at the details. Feel free to send me a note directly on my Google+ profile ( http://johnmu.com/+ ) with the details that you've found. In this particular case (assuming you mean the site mentioned in the original post), those 404s are really not having any negative effect, and any changes in ranking that you're seeing definitely have other causes.
I don't get why a link to a dead URL would have any positive or negative impact anyway, but I guess anything is possible.
Forum discussion at Google Webmaster Help.
Image credit to ShutterStock.

Comments:
Jason Damas
01/31/2012 03:58 pm
I always thought that broken links were considered shorthand for poor site maintenance, and could cause reduced rankings like any other technical issue would? I've been telling clients this for like, seven years now, and any time we've cleaned up broken links we've seen a rankings boost. I thought this was one of the oldest tricks in the book!
Michael Martinez
01/31/2012 04:58 pm
In a followup comment John cautioned people not to generalize from that discussion (either way). My only concern about large numbers of 404 errors (that Google reports) would be that the crawler is hitting the server with a lot of dead end requests. That's not a good thing in itself.
Colin
02/01/2012 12:56 am
A popular site will often be linked to with 10,000s of different malformed URLs. It would be an insane - and as JohnMu says essentially pointless to try and fix them all. For example: http://www.seroundtable.com/google-bad... http://www.seroundtable.com/google-bad- instead of http://www.seroundtable.com/google-bad-links-14660.html
Craig Broadbent
02/01/2012 09:57 am
in theory, does this mean if you inherit a penalised site caused by spam links pointing to internal pages, you can let those URLs 404 and subsequently remove the penalty?
Litmus
02/01/2012 12:58 pm
Theoretically, Google can find links anywhere - on other sites linking to you too. If someone misspelled an address, and for older sites with affiliate links it's a very probable situation, than you think it's OK for Google to extend a punishing hand ???
prabhjot singh
02/01/2012 01:41 pm
Soryy but i am not agree with you
Jeff Weber
02/06/2012 02:58 pm
I've always been concerned about these links as webmaster tools reports them as broken and the number of spammy 404's I see regularly is a pretty exhaustive list from a wide range of garbage spam sites. Am glad to hear I was correct in not wasting time trying to remedy these.
Eric Scism
03/02/2012 07:08 pm
That's good to know. Even though I try not to have any 404's they are are bound to happen. So I'm glad you won't get penalized for them.
Mij
03/15/2012 10:13 pm
why would anyone build links to a 404 page anyway? if you have a 404 or broken page just redirect it to your homepage.
Jacob King
08/09/2012 04:08 pm
Hrmm....nice conclusion here. Not!