Back in the old days, referrer spam was a way to get quick rankings by spamming other site's referrer logs. Wikipedia explains it pretty well:
Referrer spam is a kind of spamdexing (spamming aimed at search engines). The technique involves making repeated web site requests using a fake referrer url that points to the site the spammer wishes to advertise. Sites that publicize their access logs, including referrer statistics, will then end up linking to the spammer's site, which will in turn be indexed by the search engines as they crawl the access logs.
This benefits the spammer because of the free link, and also gives the spammer's site improved search engine placement due to link-counting algorithms that search engines use.
This was an issue ages ago, in Internet years at least. These days, you rarely hear about them and for good reason - it doesn't work like it use to.
JohnMu, a Google representative, said in a Google Webmaster Help thread that it generally does not hurt your rankings. He said that after someone complained their Google rankings dropped due to someone referrer spamming him. John said:
I can pretty much assure you that those referrers are not negatively affecting your collection of sites. If you are seeing fluctuations, it's almost certainly due to other issues.
John said repeatedly in this thread that this type of spam 'generally' cannot hurt a web site from ranking well in Google.
Forum discussion at Google Webmaster Help.