(8)
Update: You can block your pages from SEOMoz. SEOMoz documented the procedure, but it is as simple as a meta tag: <META NAME="SEOMOZ" CONTENT="NOINDEX" />
At SMX East, SEOmoz launched Linkscape, a powerful tool that will give you competitive analysis of linkage data. The tool has been lauded as extremely powerful though there have been some criticisms. In the Smackdown blog, Michael VanDemar attacks the credibility of the tool and claims that there is no way to block the user agent, at least not now. Therefore, we're still waiting to hear from SEOmoz on how to block the Linkscape tool from crawling their pages.
In Barry's coverage of the launch of the tool, he noted speaking with Rand about this issue. Rand told Barry that they will announce a single useragent that people can use to block the SEOmoz bots from crawling their site The question is, when will this capability come?
There's a lengthy discussion on Sphinn about the issue.
Comments:
randfish
10/20/2008 04:07 pm
Hey Tamar, I just left this comment over on Michael's blog, but figured it might be relevant here as well: We didn’t release a single UA, but instead a list. In that list might be sources we built and designed to crawl the web and others we use to bolster our data if/when those sources can’t reach data that we know we need to build a comprehensive index. It was not a lie to say that we built a spidering process or an algorithm to determine crawl rate or designed the crawl itself and the de-duping parameters, the way link juice flows, or any of the other claims. None of them were untrue. The problem I see here is that we’re being intentionally deceptive in order to protect the quality of the product and discourage competitors from being able to copy what we do. However, I stand by the fact that a lie of intentional omission is not the same as a direct untruth. Your accusations are lies in that they are not true. My claims are true, but don’t reveal everything. On the last point regarding Barry’s post on SERoundtable - I think that was a miscommunication. I said that we’d be revealing ways to block us and ways to avoid inclusion in our results, but Barry took that to mean a single UA. I do apologize for that - we chatted only briefly in person and at that time, I hadn’t even met with the team to figure out a protocol, and knew only that we were going to be providing a way to be excluded from the public index.
randfish
10/20/2008 04:10 pm
BTW - Tamar - you might want to link to this page - http://www.seomoz.org/linkscape/help/sources - which provides a list of sources (and potential sources) for Linkscape's index, as well as the meta tag information for keeping an individual page out of the results we serve.
Barry Schwartz
10/20/2008 04:12 pm
Rand, We know your intentions are solid and we feel that you have done a great service to the community. Please do not take our post as negative towards SEOmoz. In fact, we feel bad about all of this, but we felt the discussion around the topic warranted a post from us. In any event, thanks for the clarification!
randfish
10/20/2008 04:16 pm
No worries, Barry! I think it's fine to have the discussion and am happy to get the word out about what we're doing and how, as well as all the information we're releasing at this time. Do note that the comment above should be in quotes, as it was responding to criticisms not in Tamar's post but in Michael's. All the best!
Michael Martinez
10/20/2008 11:13 pm
The meta tag solution is not acceptable, since not everyone uses a content management system. SEOmoz should be supporting a robots.txt-level user-agent.
Doug Heil
10/21/2008 08:26 pm
Why did this thread come to a halt all the sudden? I find it pathetic the way this SEO firm is conducting it's business. Here is this Rand's latest statement: "“I do recognize that we are more of this roguish, agressive sort, and I think it’s one of the reasons there’s a lot of hostility. As I said, we’re not changing direction on that now, but we’ll definitely think long and hard about it. Thanks!” That's pathetic. This bot/firm is no better than all other spammers/scrapers we all deal with on servers. Using a meta tag to block linkage data is not possible. After all; isn't this tool all about links? Yep. So what makes anyone think advertising for SEOMOZ in your head tag is going to block out your link data from this tool? It's not. They already have your pages in the tool. They already have your link data in the tool. It was too late to do anything about it a long time ago as your sites were all scraped and stuck into this tool before you knew what happened.
Max Darby
10/22/2008 11:44 pm
I can understand why the community is upset around all of the "negative" things surrounding Linkscape and SEOMOZ. I do not agree with the meta tag opt-out method, but I can understand the sentiment behind what Rand has been saying. Since the sources list has been posted, couldn't you simply block them from crawling your website? Another huge fiasco is if SEOMOZ controls a bot (EG: people crying DotBot), but from what I've read, it was admitted that they have crawlers, why is dotbot so evil? it's just some indie indexing bot project by some guys. I agree with Rand also on the fact that you can't give out every project detail, that would give way to proprietary methods, which means those "evil black hatters" we all despise could just copy this. This would in turn appear that He's just covering up some evil doing, but I just don't see it like that. Maybe I'm missing a piece to the puzzle... I will say this again, I do not agree with that initial opting out method that used an SEOMOZ metatag, but I think Rand has been cooperative through this whole thing, despite harsh criticism. I also think his statement that "He'll just get the data elsewhere" was misinterpreted to that of forcefully snatching your content else ways, when I merely saw it as "We'll find a way to patch up the missing data as reasonably as possible". I'm mixed on this situation, but it hasn't damaged my view of SEOMOZ or Linkscape. I do agree that it should adhere to robots.txt rules, however. Not everyone has enough time to change their meta tags, nor do they want to clutter up their pages with SEOMOZ tags.
Halfdeck
10/27/2008 03:37 am
META Tag solution isn't a solution since your competitors will still have full access to your backlinks. I understand why robots.txt - crawled in a standard way - won't work (since bots SEOMoz doesn't control may spider sites) but robots.txt can be interpreted by linkscape to mean do not display any links pointing to this domain.