Yahoo! Search Engine Archives

It has been a very hard road for Yahoo, virtually since they began running down the path of building their own search engine and Panama (their new search marketing platform). I honestly miss the old Yahoo where we had Overture and a more personal side to them. It seems to me that Yahoo may not be able to turn things around. It seems to me that they are run down and exhausted. But I really hope Yahoo can make some type of come back.

There are stories that Yahoo is doing a major brand overhaul and refocusing. Is it too little, too late? A WebmasterWorld thread has conversation around that possibility. Will it make a difference? Is it simply too late? Take our poll, do you think Yahoo will turn things around?

Here is a flash from the past, the Yahoo "hair" commercial:

The good old days!

Forum discussion at WebmasterWorld.

A few weeks ago, we asked our readers if they like the new Yahoo home page? With 127 responses in, I wanted to share the results.

Most of our readers do like the new home page. 73% said they like the new home page, while 25% said they do not like it. That is a pretty good distribution. Here is the pie chart:

The "other" responses were pretty much in the nature of "I don't care, either way."

Are you surprised by these results?

Forum discussion continued at WebmasterWorld.

Yahoo is frequently testing new home pages. The most recent test was announced a few days ago. The new look looks like this:

Why do I bring it up now? Well, it appears some users are finally noticing the new home page in their daily browsing activities.

A WebmasterWorld thread has several people who said they now see the new home page. Personally, I don't see it. I am not sure if I like it over the current one. What about you?

Take our poll:

Forum discussion at WebmasterWorld.

Yahoo announced their first quarter earnings for 2009 and it was pretty bad. Here are the highlights:

Yahoo! Inc. today reported revenues of $1,580 million for the quarter ended March 31, 2009, a decrease of 13 percent from the first quarter of 2008. Excluding the impact of currency rate fluctuations, revenues for the first quarter of 2009 would have declined 8 percent from the first quarter of 2008. The Company’s non-GAAP operating cash flow for the first quarter of 2009 of $409 million exceeded the midpoint of the outlook range provided by the Company last quarter.

Plus, Yahoo will be cutting 5% of the staff, that is an additional 600-700 employees from the layoffs they had earlier this year and last year. Sounds like Yahoo is bleeding to me.

But the thing is, Yahoo beat Wall Street Estimates and the stock is up a bit in pre-market conditions.

In any event, you can read more about this at Search Engine Land or on Techmeme.

Forum discussion at WebmasterWorld.

Yahoo's new CEO, Bartz, is rumored to be pushing out a major reorganization for the troubled search engine in the upcoming weeks. Kara Swisher at AllThingsD, I believe, broke the news about this coming down. You can see some additional coverage on the news at Techmeme.

Greg Sterling summarizes this new management style as "top down" approach, something Yahoo might not be use to. But Yahoo needs change and maybe this will be it.

Supposedly, Bartz is very tight lipped about who she tells her plans. So I suspect both executives and the normal workers at Yahoo are both a bit antsy on what will take place this week.

Forum discussion at WebmasterWorld.

Yesterday, Yahoo released a new tool to help you take notes while you conduct research with Yahoo Search. The tool is named Yahoo Search Pad and you can learn exactly how it works with this quick video:

There is discussion around this tool at WebmasterWorld. The discussion is not all that positive. Most consider this tool to be an elegant scraping tool, stealing webmaster content, with ease. Some want to know how to block it from working.

Awful news for webmasters who create original content.

Any body found how to specifically block "Search Pad" yet.

Let's see if this tool sticks or if Yahoo ditches it, like they have done to many of their products.

Forum discussion at WebmasterWorld.

Textex at WebmasterWorld is reporting that he is seeing both an Ask.com update and a Yahoo Search update.

He first noticed Yahoo Search changes yesterday afternoon, saying "Seeing movement." He was then backed up by full member, Vimes, who said, "I'm seeing something not sure if I'd call it an update just yet, the sectors i look at there is a shuffle." So this may be the beginning of a Yahoo update or it might be some sector tweaks. We are due a Yahoo update, the last one we noticed was back in November 2008, since then, there have been no confirmed Yahoo Search updates. So having an update now, would not surprise me.

On the Ask.com front, there is a bit more discussion going on, being that the update was reported at WebmasterWorld a bit earlier. Textex called this update "a complete overhaul." Full member, robzilla, confirmed but cautioned that this update doesn't seem to be "an improvement" to their index. Soon later, they both noticed that clearing their cookies reset the "results reverted back" to their previous state. This implies that the results might be a test on some users. I did some of my own testing and the results do seem a bit better. Still not what I consider "fresh" results, but a bit better on the few dozen results I check to see quality. This cannot have anything to do with Ask.com's recent announcement on NASCAR, so I wonder what exactly is going on here?

Forum discussion at WebmasterWorld on both Ask.com update and Yahoo Search update.

I am sure by now, most of you heard the news that is buzzing all over the Internet. Carol Bartz Joins Yahoo as Chief Executive Officer is the headline of the Yahoo press release. Here is her impressive bio:

Carol Bartz, a veteran technology executive who was most recently Executive Chairman of Autodesk (NASDAQ: ADSK), has been named Chief Executive Officer and a member of the Board of Directors, effective immediately.

Prior to becoming Executive Chairman of Autodesk in 2006, Bartz, 60, led Autodesk as CEO for 14 years, transforming the company into a leader in computer-aided design software. During her tenure as CEO, revenues increased from less than $300 million to more than $1.5 billion, and the company's share price increased nearly ten-fold.

In addition to turning around Autodesk, Bartz's extensive executive experience includes hands-on responsibility for leading global operations, engineering, sales and marketing organizations for large technology and engineering companies including Sun Microsystems, Digital Equipment.

Susan Decker is out after a transitionary period. Greg Sterling has more on the news at Search Engine Land.

Yahoo's stock is up 1.74% in pre-market, but I think people are not sure what to do with the news. I think most like the decision, but they think it will be almost impossible to pull Yahoo out of this mess. Many are still hoping for a Microsoft deal, which was shot down time and time again by Jerry Yang. Jerry Yang will step down from his role as the CEO, which he announced a while back. Yang took the role of CEO in June 2007, and I think it is clear that he didn't do all that well.

Good luck to Bartz and Yahoo, I would hate to see such a nice company fail.

Forum discussion at DigitalPoint Forums and WebmasterWorld.

A WebmasterWorld thread asks why does the site command in Google not match up in the number of "indexed" URLs reported in Google Webmaster Tools. A very valid question, let me show you.

A simple site command in Google for site:www.seroundtable.com returns 17,500 results. So that means, Google has indexed approximately 17,500 pages from the www of this domain.

Now, if I login and check my Sitemap data for this site (yea, I finally created a Sitemap file), it shows about half of the indexed URLs. It says Google has indexed 8,813 URLs of the 9,086 I submitted.

For me, the answer is simple. I seem to only sending URLs of the individual blog posts here. So although I have about 9,000+ blog posts at this domain, I still have about twice as many pages on this site, due to the categories, date archives, tag landing pages and so on. Those pages are not included in my Sitemap file. So Google seems to only showing the indexed URLs of what I submitted. Of course, it is hard for me to validate that by just looking at the numbers.

What I found interesting is when I went to Yahoo's Site Explorer, Yahoo told me they h have indexed 16,498 of my pages, but crawled only 15,022 pages and thus know about 16,498 of my pages. I guess via linkage data, they can index more of my pages then they actually crawl?

In fact, Yahoo's numbers for a inurl:seroundtable.com command is almost on target to the numbers they report in Site Explorer, which is nice.

In regards to what is going on with Google... I am not sure if the results are accurate or not. Tedster at WebmasterWorld said:

I'm never surpised when Webmaster Tools information seems peculiar in some way - it happens a lot. Also note that site:example.com results are getting weirder and weirder, often omitting urls that definitely are in the index - sometimes with a simple site:example.com/directory/ query.

Forum discussion at WebmasterWorld.

John Honeck asked in a Google Webmaster Help thread how much more valuable is a number one ranking in Google, when compared to the likes of Yahoo.

He asked it in the form of, "Is being #1 in yahoo as good as being #50 in Google? #100, or?" But let me place a poll below and ask you it in a different format.

Forum discussion at Google Webmaster Help.

Yesterday, the blogsophere was abuzz with news that Yahoo will be retaining data for 90 days. At Search Engine Land, Barry explains that "[t]he data policy is not just inclusive of their search data but also their page views, page clicks, ad views and ad clicks."

The industry has been pretty quiet about anonymizing data since June 2007. But this new announcement, which shortens the length of data retention by more than a year compared to other engines, is a bit eye-opening. It's also perceived as a great move on behalf of Yahoo and one that will put pressure on Google to do the same thing.

Well, it's a great move on behalf of Yahoo if they actually remove all query information after 3 months (and not just IP information). After all, I can't help but think about Thelma Arnold who was identified by her search behavior and not her actual IP location.

What, then, is Yahoo retaining after 90 days and what is Yahoo scrapping? We don't know. Do you?

Forum discussion continues at WebmasterWorld. More blog discussion is on Techmeme.

Want to know the hottest search terms across the various search engines? Yahoo, Ask, and Google [Product Search] have listed the most popular items. Here's what Yahoo has as its top 10 searches:

1. Britney Spears
2. WWE
3. Barack Obama
4. Miley Cyrus
5. RuneScape
6. Jessica Alba
7. Naruto
8. Lindsay Lohan
9. Angelina Jolie
10. American Idol

And Ask.com's most popular list, on the contrary, doesn't seem that interesting (as some say):

1. Dictionary
2. MySpace
3. Google
4. YouTube
5. Facebook
6. Coupons
7. Cars
8. Craigslist
9. Online degrees
10. Credit score

Google's most popular searches are not available (yet?), but Google's most popular product searches are public:

1. nintendo wii
2. wii fit
3. ipod touch
4. xbox 360
5. nintendo ds
6. ipod nano
7. uggs
8. nikon d90
9. zune
10. digital picture frame

Surprised much? Some are.

Forum discussion continues at WebmasterWorld.

Argentina Forcing Google & Yahoo To Censor Search Results from Search Engine Land covers this very well. In short, Argentina is forcing, by court order, Yahoo and Google to censor their search results. The results being censored are for select public officials, models, actors, and sports stars.

For example, a search on popular soccer player, Diego Maradona returns no results at all and a message that means:

On the occasion of a court order sought by private parties, we have been forced to temporarily remove some or all of the search results relating to it.

Here is a screen capture:

Why do results come up at Google? Well, Google is censoring the results, but not completely removing the results like Yahoo.

We have discussion on this topic at Sphinn where one member seems to really know a thing or two on the topic. He said:

This is an incredible story - congrats to OpenNet Initiative for reporting this (and SEL + Ramkarthik for spreading the word :)

My first thought was that the Argentinean government had these results removed - it´s no secret that the Los Kirchner censore with a ´soft hand´ (critical journalists are not invited to press conferences, economical figures are´adjusted´ etc.). I was quite surprised to read that it´s actually private parties who have their online presence removed. I´m no expert in law, but find it hard to understand how an Argentinean private party can oblige international SE´s to have their ´names´removed from the SERP´s (and as a result other people with the same name loose their online presence as well).

Under which law do these SE´s operate? Any ideas?

Under the laws of which they run in that specific country. That is how it works.

Forum discussion at Sphinn.

A bunch of SEOs at DigitalPoint Forums are wondering why certain URLs listed in the Yahoo Search results are redirected through SEO company landing pages.

Some SEOs feel it is a weird for of paid ads within the free Yahoo search results. While others might feel it is a weird form of cloaking and redirection, and that the SEO company tracking these URLs are doing something wrong. This confusion is far from new. Let me show you what they see:

Notice the URL listed in the search results is for ServiceMagic.com, while the URL it is actually being linked to, is at FeedPoint.net.

Yea, that looks weird. But all it really is, is Yahoo's Paid Inclusion program at work. It is called Search Submit and it allows webmaster to submit URLs to Yahoo, for inclusion in the normal search results. It does not make one result rank higher or lower, it just helps ensure the page is indexed. Yahoo charges for this on the pay per click model.

So, don't be confused. This is "legit," but don't get me wrong, it is a hotly debated topic for the past five or more years.

Forum discussion at DigitalPoint Forums.

A WebmasterWorld thread asks how much traffic Yahoo sends to your site. So I thought it would be fun to poll our audience. I assume many of you use Google Analytics, so login to https://www.google.com/analytics/ and click on "Traffic Sources," and then "Search Engines." Then below the chart, but above the graph, it says, "Views", click on the circle or pie chat image. Then it will show you, by search engine, how much traffic each engine sent your way for the past month.

This site received 5.81% of our search traffic from Yahoo. Google sent a whopping 87.19% to us. How about you? Here is our search traffic from Google Analytics:

Please share your stats at this poll and/or do a blog post sharing the chart above:

Forum discussion at WebmasterWorld.

This post was prewritten and scheduled for delivery today.

I reported this news yesterday at Search Engine Land, in short, Yahoo is expected to layoff thousands of employees and cut costs across the board.

Last time Yahoo laid off employees, we were sad - cause we knew several of those hit with the layoffs. I suspect we will know many of those hit by today's announcement.

It is sad to see this happening, and it is not just Yahoo. Many companies are reducing their work force and dropping cause, in preparation for the recessionary times. Yahoo would have likely done this anyway but now they need to do this possibly at a whole new level.

We have discussion from SEOs and SEMs at WebmasterWorld and DigitalPoint Forums. Most are concerned over the competitive landscape of the search space.

Forum discussion at WebmasterWorld and DigitalPoint Forums.

This post was written on Monday, October 20th and scheduled for Tuesday, October 21st.

About a week ago we reported about the news that Yahoo is testing a new home page design. Since then, we have really not heard any chatter about people actually seeing the design in place. I personally see the old design and that is expected because Yahoo is testing out the new design on a test group.

Last night, a DigitalPoint Forums thread started, telling members that he saw the new design. This user was using Internet Explorer and was automatically redirected from www.yahoo.com to m.www.yahoo.com. Clicking on m.www.yahoo.com will not load the new design for everyone, because most users who tried it, including myself, do not see the new design.

Aurora Brown, the individual who saw this new design, posted a screen capture at AuthorityDomains.com:

It is a simpler design, but will Yahoo users like it?

Forum discussion at DigitalPoint Forums.

Reuters reports that Yahoo is doing an overhaul of its homepage, but that not all users can see it yet (like me. I never get the good stuff early). According to the Reuters article, this change is intended to "give users a personalized view of the wider Web."

Here's their screenshot:

I'm not entirely sure who is seeing the new redesign, as Yahoo says less than 15% of users have been exposed to it. Thus far, there's been no forum discussion on the impact of the redesign.

Forum discussion continues at WebmasterWorld.

Go to Yahoo Search and use their search suggestions. All I want you to do is enter in "G" into the search box and obtain the search suggestions.

Yea, Google comes up as a suggestion. This is really a common thing. People go to search engines to find other search engines. In fact, I have seen several clients go to Google.com and type in "google" to get back to Google.com. I am not joking.

The search suggestions typically try to show you the most common searches for your partial search phrase. It is not surprising to me that Google is a popular search at Yahoo. In fact, try typing in "Y" into Google, you will see Google puts YouTube first, but in fact, Yahoo is search more often, according to the numbers:

Again, not surprising.

Forum discussion at DigitalPoint Forums.

I am 99.999% sure this is a new user interface for Yahoo Search. In the past, Yahoo never ever indented search results. In fact, in the past I thought they did do indenting and then stopped, but Yahoo told me that they have "never grouped results."

Many search results, including a search for search engine roundtable return grouped results now. Here is a picture:

This is 99.999% new behavior from Yahoo Search.

Forum discussion at WebmasterWorld.

Yahoo conducted an update to their search engine over the weekend. We reported on the early discussion around the update and it then became official when the Yahoo Search Blog confirmed the update later in the day.

But since then, I have been hearing and reading discussion that the new Yahoo Search results are more like Google. And since Google tends to be the barometer of what is "relevant" and what is not, maybe Yahoo's search results are becoming more "relevant."

Yahoo Search moderator, travelin cat, at WebmasterWorld said:

We have never ranked in the top 100 results in Yahoo since their split with Google. Starting a couple of days ago, our site started showing up in Yahoo serps. Most of our kw phrases are now on either the first or second page.

Plus, we have a whole new DigitalPoint Forums thread on the topic of the Yahoo results being incredibly similar to Google results. Here is a select comment from that thread:

Did anyone noticed changes in Yahoo!' SERP today. Yahoo is following the same way as Google. Frequent change in Google SERP is kind of common now but it's new for Yahoo!. Yahoo! is changing search results very frequently now.

Not only are the results looking a bit similar, in some cases, the frequency of updates seems to be getting similar and the discussion has grown a bit around this update.

Forum discussion at WebmasterWorld and DigitalPoint Forums.

It seems like the day has come, the day that Yahoo has killed the Overture Keyword Suggestion Tool. Now, if you visit inventory.overture.com, you are redirected to http://sem.smallbusiness.yahoo.com/searchenginemarketing/. In fact, it is a permanent redirect, not just a temporary redirect, as you can see by using the URI Valet tool.

We knew this was coming, we have story after story reporting outages and downtime for this tool. Here is a run down of some of the stories we wrote covering the tool:

• Overture Keyword Suggestion Tool Offline? on January 29, 2007
• Overture Keyword Selection Tool Back Online on February 2, 2007
• New Free Yahoo! Keyword Tool Coming; Overture Keyword Tool Suffering; & Other Tools on January 31, 2007
• Yahoo! Removes UK Overture Keyword Suggestion Tool on May 2, 2007
• Yahoo To Stop Updating Overture Keyword Suggestion Tool on May 8, 2007
• Is Yahoo Working on New Keyword Tool with Search Volume Data? on May 10, 2007
• Overture Keyword Suggestion Tool Seems Dead on September 17, 2007
• Overture Keyword Suggestion Tool has Returned on September 24, 2007
• More Server Problems With Overture's Keyword Tool on December 6, 2007

So does this end the saga with the Overture Keyword Suggestion tool or as it moved elsewhere?

Forum discussion at WebmasterWorld.

Last night I was looking up a phone number for my wife and I noticed that Yahoo People Search has been totally redesigned and repurposed. The new design is extremely clean, compared to the old interface. In addition, when you conduct the search, it takes you directly into the Yahoo Search interface, as opposed to just showing you phone numbers.

Here is the before screen shot that I took back in May 2004:

Here is the new home page for Yahoo People Search:

Here is the search results page for a search conducted in Yahoo People Search:

By the way, Yahoo was able to find the phone number I was looking for, while Google failed in that search.

I really like the new interface and manner in which Yahoo People Search functions. So good job on this Yahoo. I have no idea when this was launched, it may have been months ago, but I have not read or seen any reports on this new launch.

Forum discussion at Search Engine Roundtable Forums.

David Eaves reports that UK based users who go to yahoo.com are now being redirected from the .com version to the Yahoo UK domain.

David said this is brand new behavior from Yahoo. 24 hours earlier, if a UK internet user went to Yahoo.com, it would keep them on Yahoo.com. Now, Yahoo is sending UK users off to Yahoo UK.

I honestly have a feeling this has to do with Yahoo Now Showing Geo-Location Under Search Ads from two days ago. Yahoo told me that they rolled out that feature last "week with the roll out of a new “geo labels” feature." I would not be surprised if the same geo-technology was applied to this.

I am going to post screen captures of this at Search Engine Land, with the behavior of how Yahoo, Google and Live handle UK users.

Forum discussion at DigitalPoint Forums.

Barry at Search Engine Land points out that Yahoo India has glue pages, which some people are comparing to Universal Search, because of classic search results on the left hand column, visual information in the center column and bottom right hand corner (in addition to articles), and sponsored results.

Here's a screenshot for a search for trees. Click to enlarge.

The idea is pretty cool and sticky (pun intended) and the results page is definitely more aesthetically pleasing than the standard 10 blue links. It kind of reminds me of the current Ask.com format:

Forum discussion continues at Sphinn.

Over the weekend, Microsoft finally backed off Yahoo and has decided to pull their offer to buy Yahoo and also decided they would not pursue a hostile takeover of the company. That sums it up basically. Now, if you want to read more, let me send you to Search Engine Land where we covered the news extensively.

• Microsoft Yanks Its Offer For Yahoo
• Leaving Las Yahoo: Microsoft's $5 Billion Mistake?
• Yahoo Next Steps: Google Deal, AOL Purchase & Pleasing Investors

Yes, this weekend's news is still dominating Techmeme's front page. So again, if you want to read Microsoft's letter to Yahoo or Yahoo's response or future thoughts, check out those links.

Many now believe Yahoo will begin outsourcing their search ads to Google. The advertisers want it based on the past test, it seems like Yahoo and Google both enjoyed it. So we should hear some news on that this week.

Forum discussion at WebmasterWorld, DigitalPoint Forums, and Search Engine Watch Forums.

Last week Yahoo added a nifty feature to Yahoo Local Search. For example, let's say you are looking for the nearest ATM machine. You can type in ATM, and then your zip code. But you won't be shown the nearest ATM within an X mile radius always.

Yahoo let's you plot the diameter of the circle and drop the center of that circle anywhere you want. Yahoo will then refine the local results based on the radius you selected and the center of the circle. Here is a quick video demo:

It doesn't seem like this tool is without bugs. A WebmasterWorld thread reports a blackhole issue.

I went looking for pizza places in Manhattan and noticed certain places disappeared even if you move the circle a hundred yards or so. You'd suddenly find black holes in the middle of Manhattan where there were, according to yahoo, no pizza places. Budge the circle over an eighth of an inch and the same pizza-free streets would suddenly fill with pizza joints, so there's still some work for them to do.

In any event, a unique and interactive solution to searching locally.

Forum discussion at WebmasterWorld.

Search Engine Watch Forums moderator, abbottsys, posted detailed feedback on Google Ads Being on Yahoo Search. The Search Engine Watch Forums thread has only one post but deserves a lot more participation, in my opinion.

Normally, I would summarize, but I am not feeling so well and abbottsys does a great job, so let me quote him:

1) My ad was subscribed to Google Search and the Google Search Partner Network, but it was not subscribed to the Content Network

2) I bid high so I could be sure of seeing the ad. I also shut down all my native Yahoo campaigns. As soon as I shut them down my adwords ad took over on Yahoo.

3) My ad appeared at the top of Google and Yahoo. My ad ranking on Yahoo was slightly better than on Google.

4) Changes to the ad were reflected with equal speed on both Google and Yahoo.

5) I found it *very* convenient that I could manage my Yahoo campaigns from AdWords. I've always been a big fan of the AdWords management interface, which I feel is *far* superior to Panama, AdCenter, or any others out there. In fact, I was surprised how refeshing it was not having to mess with multiple ad management platforms.

6) As regards ad performance (CTR, conversions) I was very happy. Of course, since this was a Yahoo test I have no idea if these clicks were charged or even reported in my AdWords account. But the ad did perform well.

7) I've been doing this PPC stuff since 1998. My overall comment on this particular experience is that it was great. I really liked it from all aspects.

8) If this test leads to a full ad distribution agreement between Yahoo and Google my only additional request would be that my AdWords ads show up on the Yahoo Content Network. Currently ad distribution on this network is a mess, and needs to be upgraded. Yahoo clearly has infrastructure problems with ad delivery on their Content Network. Letting AdWords handle this would be a much needed upgrade!!!!

9) Bottom line. As an advertiser I simply loved this! Two thumbs up!!!!!!!!

In the meantime, the DOJ is investigating the Google/Yahoo ad test. So even though some advertisers might love to see Google provide the search ads for Yahoo, the DOJ might prevent that from happening.

Forum discussion at Search Engine Watch Forums.

WebmasterWorld members report that Yahoo has announced its new crawler, Slurp 3.0. Now, when you analyze your weblogs, you'll be seeing Slurp 3.0, though it recognizes the same user-agent and robots.txt as before.

More importantly, they are invalidating all other Yahoo Slurp IP addresses and using brand new ones for this update. IP-based rules will need to be changed.

Search Engine Land provides more information, where Barry writes that the new IP addresses will still reverse DNS lookup to crawl.yahoo.net.

On WebmasterWorld, your work is getting more complicated, according to Wilderness. There are still 11 different spiders and 5 user agents to keep track of. But Barry explains this clearly: "If you set directives for "Slurp" or "Yahoo! Slurp," those will remain working, but if you specified "Slurp/2.0," then you may have an issue."

It is kind of funny, in that Yahoo's spiders are having the same old problems that Yahoo's business unit is having. Remember the Peanut Butter Memo, where Yahoo talks about their lack of clarity between all their business units and properties. It's all too familiar once again.

Forum discussion continues at WebmasterWorld and Sphinn.

Note: "Me" in this case refers to Barry. I'm only here to report about Slurp 3.0. ;)

A HighRankings Forum thread asks when did Yahoo stop grouping search results from the same domain together? We reported on an issue with Google not grouping certain results a few weeks ago, but it is rare to see Google not group results in many cases. But Yahoo doesn't seem to be grouping any results from the same domain together, at least not today.

For example, a search on george bush at Yahoo shows three results from the White House web site, and two right next to each other, but none of them are grouped together:

A search at Google on george bush groups a few of the results together, when the domain is the same, including the White House results:

So where did the indenting go Yahoo Search?

The thread also talks about missing how Yahoo used to number the results, but that is an older topic.

Forum discussion at HighRankings Forum.

Update: Yahoo has actually never grouped results. I have official confirmation from Yahoo on that.

Yesterday I reported that one of the many Yahoo announcements were that Yahoo was going to test running Google ads on Yahoo Search USA.

According to a Search Engine Watch Forums thread, the Google ads are now live on Yahoo. Note, if you don't want your Google ads to show on Yahoo, you need to opt your ads out on showing on third party search providers. In any event, some advertisers are noticing that their Google ads are showing up in Yahoo sponsored results.

As expected, the ads don't appear as AdSense ads - they simply look like Yahoo ads. SEW moderator explains, "Yahoo seems to be configured as a Google Search Partner, not as an AdSense publisher." In addition, he adds, "Initially (i.e. earlier today) I was only seeing my ad a small percent of the time (i.e. Yahoo was only including it on a fraction of the SRPs for a given search), but now I'm seeing my ad 100% of the time." But the ad is only showing when he stopped showing his native Yahoo ad, i.e. the Google ad won't show for him until he tells Yahoo to stop showing his Yahoo ad.

His conclusion so far?

The net effect is like a pooled ad resource, where Google and Yahoo have pooled ads to run on Yahoo (and maybe soon on both engines?)

Note, this is supposedly only suppose to impact 3% of Yahoo's search ads in the US. It is also suppose to run only for about two weeks.

Forum discussion at Search Engine Watch Forums.

The saga continues with a very unusual night for Yahoo yesterday. Let me catch everyone up with what happened yesterday. Follow the timeline, all reports are linked via Search Engine Land (times are estimates and EST):

• 15:00 (EST) - Confirmed: Yahoo To Test Google AdSense For Search Ads, has confirmed reports from Yahoo that they will be testing Google's ad on Yahoo.com search results in the US for a 2 week period.
• 16:00 (EST) - Microsoft: Forget It On Paid Search Partnership, Google & Yahoo, has a response from Microsoft that they will take them to court if they made an ad deal with Yahoo. Basically, it would give Google a 90% or more share of the search ad space.
• 21:30 (EST) - Report: Yahoo & AOL Approaching A Deal, has a second report from the WSJ that now even thought they are testing things with Google and Microsoft wants them, they may make a deal with AOL to bring in the AOL division in exchange for a cash investment.
• 22:00 (EST) - News Corp, Google, AOL, "MicroHoo" The Movie Enters A Surreal, Climactic Sequence, now we have reports from the NY Times that News Corp is interested in joining forces with Microsoft to make a bid for Yahoo.

Got all that, now that all happened in a 12 hour period yesterday. It was like, a wow, after a wow, after a wow and then another wow!

Michael Arrington thinks Yahoo is making a huge mistake:

It’s time to end this thing before Yahoo ends itself. I don’t care if they throw AOL, MySpace, and half the rest of the Internet into the deal along with Yahoo. But the health of the Internet demands a counter balance to Google.

He now says, "Yahoo-Microsoft, given the current state of things, is the only reasonable outcome."

I wonder what the major shareholders are thinking right now, especially after Yahoo's second largest investor, Legg Mason backed Yahoo against Microsoft, only now to have to deal with all this crazy news.

How will this impact SEMs? It is unclear right now. But if Google does power Yahoo's search ads, then it will make Google even stronger and that quality score and the Google slap can impact more SEMs then ever. Of course, it will give SEMs once less search ad company to master.

Forum discussion at these following threads:

• Confirmed: Yahoo To Test Google AdSense For Search Ads, Sphinn
• Breaking: Yahoo, AOL Close In on Tie-Up Deal, DigitalPoint Forums
• Microsoft Said to Be Talking With News Corporation About Joint Yahoo Bid, WebmasterWorld
• Microsoft: Forget It On Paid Search Partnership, Google , Sphinn
• Report: Yahoo , Sphinn
• Yahoo , Cre8asite Forums
• Yahoo search result show up Adsense?, Search Engine Roundtable Forums
• Yahoo Search to run Google ads, DigitalPoint Forums
• Yahoo to test search ad outsourcing with Google, WebmasterWorld
• Yahoo! Teams up with Google to Offer AdSense Ads in Yahoo! Search Results, DigitalPoint Forums
• Yahoo, Time Warner closing in on AOL deal: source, WebmasterWorld

Last week, I received a preview of some of the radio commercials Yahoo just started running. The commercials are aimed at trying to build more awareness of Yahoo Search. Some of the commercials target Google.

Elinor Mills has some quotes from some of the ads.

Search engines like Google get you lost in all the links, but not Yahoo search.

You won't find that on your Google page!

I am going to see if I can get some clips of these ads to add to this post later.

What do SEOs and Webmasters think? From a WebmasterWorld thread:

On a serious note, these ads have proven to be extremely dangerous as I almost drove off the road in disbelief when I first heard it.

Dead medium for a dead search engine. Pretty appropriate.

As long as they're polluting the results with paid inclusion, it doesn't matter how many radio spots they buy. People will still prefer *any* other engine which doesn't sell out the results.

Anyway, time will tell if it works. They are currently not doing any TV ads, like some other search company tried to do.

Forum discussion at WebmasterWorld.

Yahoo To Announce "Search Monkey" Enhanced, Annotated Results At SMX West from Search Engine Land explains it better than I can, so read it there. The Yahoo blog calls it the open approach to search.

What is it? Basically, it gives publishers a way to tell Yahoo how to enhance their search results for free in Yahoo. It is basically works on the user end like the Google Coop Subscribed links, where a user needs to opt in to see the enhanced results. But Yahoo told me, they differ in that the publisher doesn't give Yahoo a list of queries on what triggers the results, but instead, Yahoo will convert the results to an enhanced listing when it feels the user will benefit from it, as opposed to when the publisher feels the user would benefit from it. I believe that is the main distinction here and not necessarily what Danny said in his postscript (I can be wrong of course, but I think I am not - reminder to head over to SEL and postscript). :-)

What does it look like?

Before:

After:

How do I implement this for my site? You can't. At least not just yet. This is just an announcement that Yahoo will enable this feature in the future.

Okay, so when Yahoo does launch it, how do I implement it? I am told there is a simple to use feed of some sort. Honestly, they would not give me many details on it at this point. I have a feeling this part is currently being worked on and they can go several routes on the final implementation. But I hear there will be multiple methods to implement this.

How do users get this? They need to opt in via a "Add to Yahoo Search" button.

Forum discussion at WebmasterWorld where Brett Tabke says this is an "awesome development." I am not so sure.

TechCrunch got a tip that Yahoo Search is testing integration Del.icio.us results in the search results. In my opinion, bad idea Yahoo.

Michael Arrington says it "Delicious search is one of the best ways of searching for things when a standard search doesn’t pull up what you are looking for." True, but Yahoo search users won't understand it. They simply won't get it.

We have discussion on this topic at Sphinn where SEOs and SEMs weigh in. One person said this will lead to more Delicious spam, "so begins the mass creation of Delicious accounts meant to spam." Of course, for bloggers and delicious users, we are happy, as one person said, "What a great step that would be, given word of mouth is a bigger influencer of our purchase behaviour than is any other variable." But as Michael said, "excellent idea, horrible integration, stumbleupon nailed it on the head, the delicious integration is is inferior hopefully they will bring it up."

Still, I don't think Yahoo searchers would get it. I don't even think most Google searchers would get it.

Yahoo just sent me a screen capture:

It should work for you on a search for JAVA, but it doesn't work for me in Safari.

Forum discussion at Sphinn.

Bill Slawski recently wrote an interesting post entitled Yahoo Replaces PageRank Assumptions with User Data. In the article, he discusses a "User Sensitive PageRank" patent application filed by Yahoo that addresses the flaws with PageRank. These flaws include the assumption that all links are created equal, bored surfers don't go to random pages, bored surfers don't always go to only trusted pages, pages change value at different rates, and sometimes PageRank calculations cheat.

The Yahoo patent will replace some of these assumptions with user data about how they surf the web.

The title of this post was inspired by the Sphinn discussion that ensued where Matt McGee suggested a new name. Bill further makes the point that PageRank needs to be adjusted:

I think that this points out the value of making a strong site that influences people to stay and look around, focusing upon other things like usability, too.

However, many people believe that Google is doing the same thing. After all, most folks want the PageRank tool to go away, but it's probably just a matter of replacing it with something better.

Forum discussion continues at Sphinn.

In the old days, if you conducted a search at Yahoo Search, Yahoo would automatically number the results for you. So of the 10 listing, Yahoo would display the rank number on the left of the result.

Here is an example of that in action for a search on google at Yahoo:

Now, the same search, does not display the numbers:

Forum discussion at DigitalPoint Forums.

Yahoo has announced they have revamped their search interface with a from "to do" to "done" approach. Basically, they have launched Search Assist, something they have been demoing for a while and revamped how multi-media is represented in the search results.

What is search assist, well the link above explains it, but in short, it is like Ask.com's refine search features but a bit more AJAX'ish. If you do a search on anything and click on the little arrow under the search box, it expands to show you those refinements:

Plus now if you search for multi-media related searches, you may get video results that can be played within the search results, like you can with Google results:

My only complaint is, where is star wars kid video? If I am searching for the star wars kid, I suspect to get a video, Google gives it to me. But typically, if you end your search in video, such as Michael Jordan Video, you will get a video result. Interesting how Yahoo is able to embed Meta Cafe's video into their own site, didn't Google have copyright issues with that?

Here is the detailed release as a PDF file [1.1MB], plus there is a ton of coverage at Techmeme, including Greg's coverage and Loren's coverage.

So now the top four search engines have somewhat blended or made their search results more "universal" within the year. A transformation as to what they have been doing in the past, is now the standard (kind of not at Ask.com but they get it done) at the four major search engines. For past coverage of each engine read:

• Google Revamps Search Results Landscape with Universal Search
• Ask.com Relaunches in 3D with Ask X Interface
• Microsoft Launches New Search Index & Algorithms

Forum discussion at DigitalPoint Forums and WebmasterWorld.

Users from the United Kingdom on the Search Engine Roundtable Forums report that they are now seeing the Yahoo Search Suggest feature.

Yahoo is also rolling out Yahoo Search Assist, which is still in limited beta, and this is a screencast of those new features as taken by Barry yesterday:

I still don't see either of these new features since it's being slowly rolled out to those of us in the US, but Search Engine Land confirms that the Search Suggest feature has been rolled out to all UK users.

Forum discussion continues at Search Engine Roundtable Forums.

Last week we reported on a Yahoo update and a new method of crawling. The new crawl behavior is supposed to help the Yahoo bot, Slurp, be more efficient on your site.

It seems that many SEOs and Webmasters are not happy with this change.

A WebmasterWorld thread has several negative comments:

OK, this is completely bogus and helps nobody. The number of IPs that Slurp uses? WHO CARES...

The fact that Yahoo has multiple crawlers for every division that crawl independently and don't share the common cache, now THAT's a problem that needs to be fixed.

We get 50% of our pages crawled every day.

Well Yahoo is a waste of bandwidth on one of my sites (a large directory). It hammers it almost every day and has only sent about 975 referrers this month. That is so low that I wonder if people even use Yahoo in Ireland for anything other than e-mail. I am strongly considering blocking it.

Forum discussion at WebmasterWorld.

Danny over at Search Engine Land breaks some news about the Yahoo! Search Engine. Yahoo has come out with Search Assist, which is a smart and selective tool to suggest particular search terms and appears when you need it (like when you've stopped typing). It also shows related topics.

Danny's screencast shows what it's all about.

Forum discussion at DigitalPoint Forums.

It seems to be the consensus on a DigitalPoint Forums thread that if you post your URL to Yahoo! Answers, Yahoo will index your site a lot quicker than you'd normally expect.

A member asks how long it takes for sites to start being indexed in Yahoo. Apparently, not long, if you utilize this practice. That's not to say that you should spam your site URL throughout Yahoo! Answers, but if your website is related to a question being asked and can help someone find the answers, then it may be useful.

A much older WebmasterWorld thread says that without this practice of linking in Yahoo! Answers, you can simply wait the standard time of 6 months to one year to be indexed.

Forum discussion continues at DigitalPoint Forums and WebmasterWorld.

Last Thursday, July 19th, the Yahoo! Search Blog announced that they have started rolling out an update.

We've been rolling out some changes to our fresh web data and crawling, indexing and ranking algorithms over the last few days. We expect the update will be completed by the weekend. So, as you know, throughout this process you may see some changes in ranking as well as some shuffling of the pages in the index.

Since then, I have been waiting and seeking out threads discussing changes people have seen to the rankings in Yahoo. But I have yet to see any thread discussing any shifts and changes in the Yahoo Search results.

Typically, the forums spot these updates before they are announced. In this case, the announcement was made and still, there is fairly no discussion about changes in the search results.

There is a WebmasterWorld thread discussing the announcement of the update. There is one person saying he saw an increase in traffic from Yahoo, "One of my site was getting 1 or 2 daily from yahoo, suddenly it got 10 so far now." But outside of that, the only discussion in the forums I track are from the Y!Search Blog announcement.

Forum discussion at WebmasterWorld

Update: People are now discussing minor changes in ranking.

An increased amount of people are noticing that Yahoo has added search suggestions after users type more than two letters in the search field on Yahoo.com. This was first reported by Search Engine Land but was discovered recently by DigitalPoint Forums members.

(Ironic, isn't it?)

The feature, according to many, is called "Search Suggestions." In fact, according to Barry's report on Search Engine Land, you can disable it if you don't like it, which some DigitalPoint Forums members have done.

Forum discussion continues at DigitalPoint Forums.

A Search Engine Watch Forums thread has a member stating that although his site was banned from Yahoo! Search and was denied reinclusion into Yahoo's index, he was still able to guarantee inclusion through Yahoo's paid inclusion program.

Yahoo's paid inclusion is now named Search Submit and it is a program that allows you to send Yahoo your pages, and if approved, Yahoo will guarantee to crawl those pages often, plus give you the ability to send them more meta data.

The major issue with being accepted into paid inclusion but being denied to the normal Yahoo crawl is that they both should follow the same quality content guidelines. Yes, the paid inclusion program has a set of content guidelines. But whatever is displayed within the search results have to meet Yahoo's overall quality guidelines.

The member explained the process:

1. Banned by Yahoo!
2. Made some changes
3. Given the opportunity to participate in Paid Inclusion, which you have to be good enough for regular inclusion to participate in.
4. Attempted regular inclusion...we were told NO.
5. Looking at Paid Inclusion again because we can advertise in the organic rankings with this product.

He said, as soon as they pay Yahoo, they will be included in the search submit program.

About a few weeks later, Yahoo actually included them back into the search index for free. The member said:

We have been miraculously reincluded and we are performing very well in the organic listings without paid inclusion!

As many people know, there is a gray line as to what quality truly is. Even within organizations, one person at Yahoo can review a site and consider it "good enough" to be included, whereas someone else can say it "just doesn't meet the requirements." Is this a case of that?

Forum discussion at Search Engine Watch Forums.

A High Rankings Forum member has run into a very strange issue on Yahoo. She has recently optimized her page and sees that it is updated in Yahoo, but not all elements seem to have been updated.

I just completed optimizing a site and was surprised to see it already cached on Yahoo (not google or MSN). When I click on "cached," I see the brand new content and complete optimized page. The title and description on the SERP, however, remains the same. Why would it have crawled and not change this pertinent information?

There are a few things that could have caused this problem:

The page could be pulling the title and description from the Yahoo directory which can be addressed by adding a NOYDIR directive.

Another possibility, though rare, is that Yahoo! could be pulling the DMOZ title and description.

And finally, it is possible that the data centers still have to synchronize:

It wouldn't surprise me at all of the caching is handled by a completely different set of servers from the datacenters that handle search and the display of title/snippet. If this is the case it should eventually sort itself out as all of their server farms catch up with your current offering.

Forum discussion continues at High Rankings Forums.

WebmasterWorld and DigitalPoint Forums thread report a spike in activity with Yahoo Search's web crawler, Slurp.

People have noticed a large increase in page hits and bandwidth usage, caused by Yahoo! Slurp recently.

I run several sites and I have the same problem. In fact, if Y! were to send me 1 visitor for every 10 bot visits I would need a dedicated server to handle the traffic :o)

i used to get like 4-5 at a time, but in the last few days it jumped like 8-10 and today it was an all time high of about 18 yahoo bots at a time. i havent checked the total of the bots/day but its surely going to be a larger number today

There are enough posts for me to report this, but not enough to say this is happening to a large percentage of site owners.

Forum discussion at WebmasterWorld and DigitalPoint Forums.

Last night I frantically kept on updating my post named Yahoo's CEO, Terry Semel, To Be Replaced By Jerry Yang on Search Engine Land. The news came out in dribs and drabs from several different sources. Soon, it became clear, how the new Yahoo will be organized.

Terry Semel, Yahoo's CEO, has stepped aside to give stockholders and Yahoo users want they want - a new CEO. Yahoo's co-founder, Jerry Yang is taking over the CEO position, with Susan Decker is promoted to President from executive vice president. The statements are now all out and the Techmeme coverage is overwhelming.

Here is a roundup of the larger reports:
- Yahoo! Co-Founder Jerry Yang Named Chief Executive Officer is the press release
- My new job is Jerry Yang's blog post at the Yodel Anecdotal blog.
- Semel Steps Aside as Yahoo CEO from the Wall Street Journal is one of the original places to break the story.

Again, you can see all the coverage at Techmeme.

Danny clarified that Yahoo has no plans to bring in a new CEO. They believe Yang is the man for the job, since he understands Yahoo better than anyone else. I am for that.

My big question is how will this impact you guys, the SEOs and SEMs, in the short and long term. We have Panama, will Yahoo release more features for it soon? We have Yahoo! Search which has not really seen much excitement recently. Will Yahoo now begin to make things happen with both? Don't get me wrong, Panama is a huge undertaking and investment - and they have done a good job with that. But the Yahoo! Search side seems to be lagging behind.

I suspect Yahoo to quickly come out with some technology announcements to back the managerial changes that just happened. Or at least, I would hope Yahoo would back the management changes with technology changes. The question is, in which parts of the company will we see it?

There are many wishes from the SEM community in the forums. Most focused around the YPN product.

Forum discussion at WebmasterWorld, Cre8asite Forums, DigitalPoint Forums and Search Engine Watch Forums.

Yesterday was Father's Day and some of the search engines sported new logos for the special day.

Google had:

Dogpile had:

Yahoo had flash:

And here at the Search Engine Roundtable:

Ask.com did not have a logo for the day, which is not like them, but maybe it has to do with their new interface?

Forum discussion at Search Engine Roundtable Forums.

Earlier this week, we reported that Yahoo was experimenting with a blue header for their search interface. I just did a search at Yahoo and I see a purple header being tested.

Here is a screen capture:

Clicking on the "options" button on the right, opens a menu with "advanced search," "preferences," "advertising program," and "about this page." Those links also appear on the normal Yahoo! Search interface.

The Yahoo! Search home page remains to be the same white background and same user interface for me.

Forum discussion at Search Engine Roundtable Forums.

Has anyone seen Yahoo's new look? Jim Boykin, Karl Ribas, and folks on the DigitalPoint forums did. I can't reproduce it at the moment, so I'm assuming that they're rolling it out slowly, but I like it. The shiny blue bar is actually pretty cool.

So what do forum members think? "It looks nice!" says one. Jim's blog commenters also agree: "I think a change would be good," says another.

Only one gripe has been mentioned so far:

Although I think it looks nicer, it doesn’t display the number of results found in a search. That number is a very important number so I hope they’re working on including it.

Yeah, I think that needs to be there too. Otherwise, it's pretty and would be a welcome change.

Forum discussion continues at DigitalPoint Forums.

The Yahoo! Search Blog has announced that webmasters will now see Yahoo's spider, Yahoo! Slurp, returning a new domain name in your logs. The same IP addresses now render to the domain name crawl.yahoo.net and no longer return the domain name inktomisearch.com.

As of today, the transition is complete and all machines crawling as Slurp are now in crawl.yahoo.net. You can see this change in your web server logs, where the page accesses from inktomisearch.com are being fully replaced by crawl.yahoo.net contacts. Note that this does not cover other Yahoo! crawlers, such Yahoo! China, and other verticals, like Yahoo! Shopping, Yahoo! Travel, etc., which have their own user-agent.

The good news: you don't have to change your robots.txt file over to prevent Yahoo! from crawling your site (if that's what you're aiming to do) because Yahoo says that the crawler user-agent is still Yahoo! Slurp and the IP addresses have not changed.

Discussion continues at WebmasterWorld.

Last night the Y! Search blog announced a weather report indicating that the Yahoo index was updating overnight. Tim Mayer who usually lets us know about these updates told us to expect some changes most likely in response to spam reports and feedback from webmasters. The Y! Search Blog says:

We just rolled out a new search index last night. So, as usual, you may see some changes in ranking as well as some shuffling of the pages that are included in the index throughout this process. This update should be complete very soon.

I have spent the morning tracking down some forum threads on the recent update. While it may still be too soon to tell the effect of these changes. A thread on Digitalpoint Forums and WMW so far are the most active. From the forum members, most from my searching have indicated positive results from this update. Its seems like a big update from what I can tell. Some webmasters are seeing upward changes in their sites ranking and some are seeing no changes at all.

If you have been following our reports on Yahoo lately, Barry did an excellent post on websites ranking poorly in Yahoo but doing well in Google. There has been some good discussion lately about the differences between Yahoo and Google algorithms. Lately I have been getting a lot of emails from clients and friends about how to improve their search engine rankings in Yahoo. So what does Yahoo like? Barry profiled at the beginning of May new tags Yahoo has released to let them know about your most important content. There are also some good tips in this WebmasterWorld thread.

Continued discussion at WebmasterWorld, DigitalPoint Forums, SEOChat

A WebmasterWorld thread links to a post at Agerhart.com showing screen shots of Yahoo! Autos cloaking.

Cloaking is when a search bot is given one page of content, while a normal user is given another set of content.

If you go to http://autos.yahoo.com/used-cars/forsale.html and compare it with the Google Cache version, to me they look identical. So possibly, Yahoo! changed it. But in the screen captures, only the Google version had the "used cars" anchor text by every state break down. You can see the before and after at Agerhart.com.

It seems like Yahoo Autos is currently not cloaking at this moment.

Forum discussion at WebmasterWorld.

Update: Tim Mayer of Yahoo! has confirmed on May 22nd edition of The Daily Search Cast that Yahoo Autos has changed the page since this has been reported. So, Yahoo Autos was cloaking. FYI, this wasn't the first time Yahoo! was caught cloaking, there also were spotted using unethical search practices in July 2005.

Gabs spotted that Yahoo Search UK was showing zero results for the term fishing. Here is a screen capture:

But do notice the sponsored results.

But if you do the same search at the UK domain and not add the "countryGB" to the end, you do get results.

Weird, indeed.

Forum discussion at Search Engine Roundtable Forums.

A WebmasterWorld thread notes that if you do a search over at Yahoo! Japan's image search, and then click on an image, it opens the image in a larger view and let's you flip through other images.

For example a search on basketball gives you a dozen or so results. I clicked on the second result which brought up a screen that looked like this:

Notice of the image overlaid on the other results and a little player came up at the bottom. The player lets you close out, go back, go forward, play or get more information about the picture. Will this be coming to Yahoo! Images USA, I doubt it. FYI, Yahoo! Pictures is closing down for Yahoo! owned, Flickr. Flickr images already show up in Yahoo! Search for image related searches. Flickr sports a neat Flash slideshow player, that is something like this, but not exactly.

Forum discussion at WebmasterWorld.

As a big fan of music and especially of singing harmonies, I like to know the words of the songs I sing along to when I'm onstage in the shower car. Lyric search results have never been quite perfect, and people always get some words wrong. It's nice to know that Yahoo! Music just launched its own lyrics website and the search results are being integrated into Yahoo SERPs.

The Yahoo! Search Blog covers the debut in more detail, and I was eager to try it out.

Some of the results are typical. Searching for "[artist] lyrics" showed the new shortcut in the results:

However, searching for "[artist] [songname] lyrics" didn't bring up the shortcut:

I can understand if people feel that the shortcut isn't necessary because there's really only one official result, but I still see a use for it.

Discussion at DigitalPoint Forums.

It seems that a lot of well-known companies have webmasters (or legal departments) who just don't have a clue how to implement a robots.txt file. According to a DigitalPoint Forums thread, the United Kingdom based Daily Telegraph is looking to sue Google and Yahoo for accessing its content.

Their statement, as quoted in the Guardian Unlimited, is that they are concerned that these search engines are accessing content for free and don't give them proper credit.

Our ability to protect content is under consistent attack from those such as Google and Yahoo who wish to access it for free. These companies are seeking to build a business model on the back of our own investment without recognition. All media companies need to be on guard for this. Success in the digital age, as we have seen in our own company, is going to require massive investment... [this needs] effective legal protection for our content, in such a way that allows us to invest for the future.

Apparently, they're clueless about implementing a robots.txt file that will prevent search engines from accessing content "for free." As of this writing, this is its current robots.txt file:

# Robots.txt file # All robots will spider the domain

User-agent: *

Disallow: */ixale/

Not only that, but they have the ability to remove content from the SERPs in Google and in Yahoo.

It is a bit disturbing how many people are concerned about search engines (which ultimately give them more visibility!) The claim that search engines don't respect their rules goes both ways. Daily Telegraph, I imagine you have rules you want Google and Yahoo to respect. Well, the search engines have rules too. Follow them and you'll be fine.

Feel free to add your two cents on the DigitalPoint Forums thread.

Ask.com, Google and Yahoo! all sported logos for Earth Day yesterday. I did not see special logos at Live.com or even Dogpile (but I may have missed them). Here they are:

Ask.com redid their home page:

Google went cold with their logo:

Yahoo! had this cute animation, which I took a screen cast of and put on YouTube. The video is pretty stretched out, so please keep that in mind.

Here is the Flash file from Yahoo (they may move the file off the server in the future):

Last year's Earth Day designs can be found here and 2005 over here, and Google's 2004 here.

Forum discussion at Search Engine Roundtable Forums and DigitalPoint Forums.

An interesting finding was discovered in a DigitalPoint Forums thread that references a study that says that men prefer Google as a search engine and women prefer Yahoo. From the thread:

About men, he states:

Men tend to see it as an office, a library, or a playground--screw the community, this is about function not family.

Men tend to be more intense Internet users than women, being more likely to go online daily (61% of men and 57% of women) and more likely to go online several times a day (44% of men and 39% of women).

About women, he states:

The report found that women are more enthusiastic communicators, using email in a more robust way. Not only sending and receiving more email than men, women are more likely to write to family and friends about a variety of topics, sharing news, joys and worries, planning events, and forwarding jokes and stories.

While both sexes equally appreciate the efficiency and convenience of email, women are more likely than men to value the medium for its positive effects on improving relationships, expanding networks, and encouraging teamwork at the office.

I think this is an interesting observation. I commented from my own personal perspective that I seek out social networks, many of which are Yahoo! properties, but I personally think that the search engine itself does not include those "community elements" that are so heavily emphasized.

Interestingly enough, I discovered another post that showed that female Internet users outnumber males. This is is even more interesting considering that Google is the dominant search engine. I'd love to know what kind of sampling was taken for the study, since I question the accuracy of the data with the information provided.

Still, the psychology behind the study is an interesting one. Perhaps I'm just a bit biased because I'm so immersed in this technology and in social media as well. :)

To further test such hypotheses, Microsoft has an interesting adCenter Labs Demographics Prediction Tool. One can certainly have fun with this. I see that Barry did, but his findings were different.

Forum discussion continues at DigitalPoint Forums.

The Yahoo! Search Blog has announced that Yahoo! Site Explorer is out of beta. Yahoo has also announced that Site Explorer has added functionality for users to submit sitemaps for their mobile device-friendly websites through the control panel. WebmasterWorld has discussion on the mobile-friendly submission and a WebmasterWorld member also notes that authenticated users are also able to report suspicious spam sites sites through the tool as well.

I have included screenshots of both new features.

In Yahoo! Site Explorer, when you view your submitted sites, you can click on Manage to add feeds for mobile browsers:

When you view your site's Inlinks and hover over a specific link, you will see the "Report Spam" button as such:

The spam report feature is being received well, but some question whether this could hurt site rankings if too biased:

The report spam does sound interesting and it was mentioned before who is to say what is not related content?

Webmasters could easily harm their rankings by reporting sites that they do not see as being quality sites or relevant links, however, Yahoo could see that site much differently. Subsequently rankings could suffer as a result.

I sure hope there is a human element to review the spam reports in the backend.

Discussion continues at WebmasterWorld.

Last week, Google, Yahoo, Microsoft & Ask.com To All Support Sitemaps Autodiscovery. So how do you ping these services to notify the search engines of an update to your Sitemaps, if you do not want to wait for them to find it themselves?

Softplus at Cre8asite Forums posted the URLs you can use to ping the various engines. Here they are:

Ask.com: http://submissions.ask.com/ping?sitemap=http%3A//www.domain.com/sitemap.xml
Google: http://www.google.com/webmasters/sitemaps/ping?sitemap=http:%3A//www.domain.com/sitemap.xml
Yahoo: http://search.yahooapis.com/SiteExplorerService/V1/updateNotification?appid=YahooDemo&url=http://www.domain.com/sitemap.xml

I did not test these myself, but they seem accurate.

Note, there is no URL listed for Microsoft's Live search. Why? I suspect they currently do not support Sitemaps. Which brings me back to my lingering question, Is Microsoft's Live Search Ever Going to Add Sitemaps Support? They have been promising it since November 15, 2006.

Forum discussion at Cre8asite Forums.

Great news from yesterday at SES. Danny has a great roundup describing that Search Engines Unite On Sitemaps Autodiscovery at Search Engine Land and I have some more details with my coverage of the Sitemaps & URL Submission session from yesterday.

In short, all four major search engines, Google, Yahoo, Microsoft's Live.com and Ask.com will all support an autodiscovery method for Sitemaps. Sitemaps is an XML protocol that enables you to freely submit a listing of URLs with more meta-data to the search engines, so that the engines can be assisted in their crawl process. It is like a form of paid inclusion without paying.

Sitemaps was first introduced in November 2006 but back then you had to manually go to Google Webmaster Central or Yahoo Site Explorer and inform them about your sitemap. Now, all you need to do is put a little marker in your robots.txt file, telling the search engines the location of your sitemap and presto, the search engines will find it on their own.

Microsoft and Ask.com both promised to support it, but I believe are currently not supporting it yet.

More details on these bot sitemaps (not human sitemaps) at sitemaps.org.

Forum discussion at WebmasterWorld & Cre8asite Forums.

WebmasterWorld picked up on Yahoo's new Alpha Personalized Search Engine, which is currently in beta.

According to the Yahoo Alpha blog, the service adds elements of personalization, RSS functionality, and the ability for users to share their results with friends:

Whilst aggregating feeds on one page is nothing new, we wanted to take a federated search concept one step further. With this beta, we have introduced personalisation elements that not only allow users to customise their view, but also to add their favourite search service (at least those who syndicate their search results via OpenSearch RSS).

We also decided to add a sharing element for logged in users of the site. Now, if you decide to share your personalised url with others, they can view and effectively use the search configuration you have developed.

The screenshot of Yahoo Alpha, which sports an AJAX interface, is below:

Here are what the SERPs look like. Oddly enough, the Sponsored Results are actually hidden unless you click on them!

So far, it's really open to the "Australian community," but that didn't stop me from getting a sneak peek (or you either). Check it out and then join the discussion at WebmasterWorld.

David at Search Engine Roundtable Forums reports that Yahoo! has removed the directory (category) links from within the search results (directly under each search result.

A search on google at Yahoo! Search confirms this to be true. In the past the results looked like:

Notice the category link, which links the search result directly to the category it belongs to within the Yahoo! Directory. Now that seems to be gone for all searches I tested.

Same with a search on search engine roundtable:

Old Result:

New Result:

I tried several different searches that I knew returned the category link within the listings, and all are gone. I do not know if this is on purpose, or if this is just a Yahoo! test.

Forum discussion at Search Engine Roundtable Forums.

A WebmasterWorld thread links to a December 2006 paper at Yahoo! Research named A Reference Collection for Web Spam. The paper can be downloaded as a PDF file, it is not brand new, but relatively new. Here is the abstract:

We describe the WEBSPAM-UK2006 collection, a large set of Web pages that have been manually annotated with labels indicating if the hosts are include Web spam aspects or not. This is the first publicly available Web spam collection that includes page contents and links, and that has been labeled by a large and diverse set of judges.

Gary Price of ResourceShelf linked to an updated paper from Microsoft on Web spam. The 10 page PDF file is named "Spam Double-Funnel: Connecting Web Spammers with Advertisers." Here is the abstract:

Spammers use questionable search engine optimization (SEO) techniques to promote their spam links into top search results. In this paper, we focus on one prevalent type of spam – redirection spam – where one can identify spam pages by the third-party domains that these pages redirect traffic to. We propose a five-layer, double-funnel model for describing end-to-end redirection spam, present a methodology for analyzing the layers, and identify prominent domains on each layer using two sets of commercial keywords – one targeting spammers and the other targeting advertisers. The methodology and findings are useful for search engines to strengthen their ranking algorithms against spam, for legitimate website owners to locate and remove spam doorway pages, and for legitimate advertisers to identify unscrupulous syndicators who serve ads on spam pages.

So here is your weekend reading.

Forum discussion at WebmasterWorld.

There is no doubt that a ton of bot activity on one's sites are from rogue spiders. Spider or bots that pretend to be legit bots but are there to steal your content. We have covered several sessions on this in the past; here are some:

• The Bot Obedience Course - August 8, 2006
• The Bot Obedience Course - December 5, 2006
• Scrape Bots Vs. Search Bots :: Fighting the Battle - September 12, 2006
• Spider and DOS Defense - Rebels, Renegades, and Rogues - November 16, 2006

A new Cre8asite Forums thread asks a question on how does one verify if GoogleBot is really from Google.

Matt Cutts posted a detailed How to verify Googlebot back at the Webmaster Central Blog on 9/20/2006 explaining how to do reverse DNS and then a forward DNS->IP lookup.

Telling webmasters to use DNS to verify on a case-by-case basis seems like the best way to go. I think the recommended technique would be to do a reverse DNS lookup, verify that the name is in the googlebot.com domain, and then do a corresponding forward DNS->IP lookup using that googlebot.com name; eg:

> host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.

> host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1

I don't think just doing a reverse DNS lookup is sufficient, because a spoofer could set up reverse DNS to point to crawl-a-b-c-d.googlebot.com.

Of course there are some ways to automate this. Either code it yourself, buy CrawlWall or implement a solution similar to Ekstreme's PHP Search Engine Bot Authentication.

Rogue spiders are no fun, as we have seen in cases with some forums.

Forum discussion at Cre8asite Forums.

The Compete.com Blog posted results for their data analysis of where do people spend most of their time online. While Yahoo! came in second with 8.5%, Google only captured 2.1%. MySpace came in first place with 11.9%. These figures include all of Google's subdomains. If you add in YouTube.com to Google's mix, you add 0.6% to 2.1% and get 2.7%.

Compete.com has some great tools by the way.

In any event, the forum reaction to this study in WebmasterWorld is "not surprising." Check out some of these quotes:

Well, that makes sense. Yahoo is mostly a portal, while Google is a search engine.

Not at all surprising, but not because of the relative values of the sites. Rather we are seeing the results of strategic placement within a default operating system setup.

When Google launches games.google.com then they could be talked about in the same context as yahoo, but its obviously no comparison now.

Cute.

Forum discussion at WebmasterWorld.

A WebmasterWorld post shows two respected members reporting that they have seen Google AdWords URLs within the Yahoo! Search index.

vicyankees says:

Is anyone else noticing that Yahoo is picking up Google Adwords links? I only know because i append all of my Google and other PPC links with a src=#*$!X and my tracking software also appends similar information.

WebmasterWorld moderator, bill, confirms this saying:

I just noticed this as well. I hadn't checked my Yahoo SERPs in a while and just noticed that they indexed several of my AdWords pages. What's up with that? I guess the first question would be, 'how do we get them to stop it?' Do I have to change all of my AdWords tracking codes?

Forum discussion at WebmasterWorld.

Last week, Yahoo! launched OneSearch a downloaded application for your mobile phone that helps you search quicker on a mobile device.

The Yahoo Search Blog gives you examples of how it works. It currently doesn't work on many phones, you can see the list of carriers and phones that are supported over here. Plus Greg Sterling has a write up at Search Engine Land.

It does seem to be supported in multiple countries, based on the "Choose your country" drop down on the phone selection page.

Yahoo! has added that they are pulling back on the number of people who can download it:

Note: due to overwhelming demand for our cool new Yahoo! Go 2.0 beta, you may be placed on a waitlist of up to 1-2 weeks to get it (depending on your phone model). Register now for the beta — the sooner you do, the sooner you'll get it!

Forum discussion at Cre8asite Forums and WebmasterWorld.

I have no idea what is going on here, but I saw a few instances just a few moments ago myself. This was first reported on WMW where a member noticed that Yahoo was changing the titles of his site's listing in the search results. Yahoo was replacing a shortened lowercase different title for its normal title that was Uppercase longer and the accurate original title used on the site.

At first members wondered if Yahoo was just playing hooky and messing with Dmoz titles or even Yahoo Directory titles. This was not the case, as most of the sites mentioned don't have Dmoz or Y! listings.

Join the discussion and outrage at WMW - Yahoo Replacing Site Titles

Update by Barry: I wrote a long post with Danny Sullivan at Search Engine Land on this, hope to have official word from Yahoo! soon.

Update 2 by Barry: This has been fixed now, Yahoo! Fixes Titles in Search Results Sourced From Internal Anchor Text, Thanks Tim!

It was announced late last night that Yahoo! has bought the blogging community and widget MyBlogLog. Although the agreed amount was not disclosed, a Forbes source claims an amount of over $10m was offered.

There has been surprisingly little reaction to this announcement in the forums, despite being a Highlighted Post at WebmasterWorld.

Rather than letting me ramble on about what MyBlogLog is, Danny has an extremely comprehensive write-up over at Search Engine Land.

What I'm most interested in however, is what value the community and widget site will offer Yahoo! at a very important stage of their battle with Google. Is this simply a case of seeing value in the community aspect of the site, or the fact that their widget for placing on your blog, gives them access to the same (if not more) information as a your web analytics package? If this data was fed into the main Yahoo! beast, it would offer insight into what users do when they reach a specific site; maybe even using the data to manipulate natural search and paid search results. As the widget would only feature on blogs, the data won't necessarily effect individual site, although still provides interesting user behaviour information.

Further discussion over at WebmasterWorld Forums.

The holiday season is here and most of the search engines are already sporting holiday season logos. Tonight is the last night of Chanukah, so we took down our logo for Chanukah and put up the Christmas logo. Here it is:

Google is changing their logo daily, this is the second logo, but make sure to track them here.

Yahoo! has a very cute one, that is flash, they skate around the logo, I took a static image of one frame.

Dogpile is sporting a shopping search engine theme.

Ask.com doesn't have anything yet, but I am sure they will (I'll update the post when it is added). Update: Ask.com does the background change...

View Full Image

Cre8asite Forums sports a holiday logo:

I wanted to wish you all a happy and healthy holidays!

Forum discussion at Search Engine Roundtable Forums and DigitalPoint Forums.

A DigitalPoint Forums thread discusses Yahoo!'s gift for the 2006 holiday season. Yahoo! sent all publishers and advertisers a email holiday card, which you can see at http://brand.yahoo.com/snow/. Yahoo! sent some advertisers a Yahoo! sweatshirt and USB drink cooler. It looks like:

Yahoo! also sent some bigger advertisers a "fancy corkscrew kit from Leeds."

I have more details on Yahoo!'s gift to me at my personal blog, Cartoon Barry.

Forum discussion at DigitalPoint Forums.

There was a lot of buzz yesterday on IBM's co-announcement with Yahoo! on the release of a free enterprise search application named OmniFind. OmniFind is defined as a entry-level enterprise search software solution.

Features Include:

• It will index up to 500,000 documents
  
• Search internal documents as well as web documents
  
• Supports 200+ file types
  
• 30+ languages
  
• REST & XML
  
• Uses open source Apache Lucene
  

Yahoo! and IBM got some good feedback on this release. More details at http://omnifind.ibm.yahoo.net/productinfo.php.

Forum discussion at DigitalPoint Forums & WebmasterWorld.

Barry has just posted on the SEW Blog that Yahoo has now officially integrated Answers into its search results.

We reported back a month and a half ago that Yahoo Tests Enhanced Yahoo Answers Integration In Search Results but now everyone can see it for themselves by conducting a search for vacation ideas.

I personally feel that Yahoo Answers will form an extremely important part of the company's future. Not only will the Q&A platform keep users coming back to Yahoo (or installing the toolbar for instant Q&A notifications), it also offers a way to improve search results for some topics. Unlike algorithms used to position web pages for a specific term; Yahoo Answers offers not only a comprehensive database of topics people are searching for, it also gives a moderated and democratically chosen best answer for those questions. A search for [viagra side effects] which can often return affiliate links, can now show an answer which is voted for by the community in an environment completely controlled and moderated by the engine.

I've gone into more detail on the history, purpose and future of Yahoo Answers in an article which I wrote just before this announcement was made.

Further Discussion At SearchEngineWatch Forums

This is cool, and I never noticed it before. Go to www.yahoo.com and click on the exclamation point. If you have sound on, you should hear the classic, "Yahoooooo!" voice. Pretty cool.

Again, just click on the top portion of the exclamation point, the part I boxed in below.

Forum discussion at DigitalPoint Forums.

SEOdisco has screenshots of Yahoo! asking for feedback on the organic, natural, free search results, asking the user.

How useful are the first two results below?

The survey gives you a 1 to 5 point scale, 1 being not useful and 5 being useful.

We reported something similar in the past, but they seemed to have referred to rating PPC ads at Yahoo!. After looking back at the survey, which asked people to rate the first five results, maybe it wasn't rating the sponsored results, maybe it was rating the organic results.

We know Google constantly asks people to rate the sponsored results with a "Yes" or "No" response to "Was this link useful?"

SEOdisco posted this information at WebmasterWorld message number 3143964.

Most of the search engines are sporting fancy and scary logos for Halloween today. Here is a run down of this years Halloween search engine logos.

http://www.google.com/search?q=halloween

http://events.yahoo.com/halloween06/

http://www.ask.com/web?q=Halloween (sporting a whole background change)

http://www.dogpile.com/info.dogpl/guides/halloween.htm

Forum discussion at DigitalPoint Forums.

Chris Sherman has the ultimate write up on Yahoo Updates Toolbar and Bookmarks. Here is a bullet list of updates, as I see them:

• auto-complete feature
• bookmarking with thumbnail captures
• bookmark folders
• bookmark tags
• bookmarks searched by toolbar
• bookmarks and tags are private in nature with this toolbar

Download the new toolbar at http://beta.toolbar.yahoo.com/.

Forum discussion at DigitalPoint Forums and WebmasterWorld.

Have you guys seen the new Yahoo! commercials hitting your TV? They are pretty good, in my opinion. This one I find to be pretty funny and witty...

There are others, like the Yahoo! raising dead flowers one for Yahoo! Answers. You can also see Yahoo!'s Yahoo! Music commercials . Here is a funny one with a driver. Good stuff Yahoo!

This is all part of Yahoo!'s plan to spend big to compete.

Forum discussion at DigitialPoint Forums.

I pleaded with Yahoo in the past to Please Enable a No Yahoo! Directory Tag, and I am doing it again. It is especially important now, that Yahoo! adds support for this tag. Webmasters are confused, they think that because Google and MSN (Live.com) support a tag to force the engine not to use the ODP directory tag, that Yahoo! would support the same.

A WebmasterWorld thread shows several web site owners complaining that Yahoo! forces the Yahoo! Directory title in the search results.

It really sucks that they do that and the listings are so bland and one simply cannot change anything with them...we have tried to get them to change our listing, but they NEVER answer. That is highly irritating.

It is sad that you pay to get listed in the Yahoo! Directory and it can have a significant difference in your click through rate on your organic results.

More details at Yahoo! Please Enable a No Yahoo! Directory Tag.

Forum discussion at WebmasterWorld.

Update: Yahoo! has now added support for the NOYDIR tag, more details here.

I reported yesterday at the SEW Blog that Yahoo is To Run Huge Global Advertising Campaign.

Yahoo is going to be running a multimillion-dollar global advertising campaign consisting of television, radio, cinema and online advertising.

Yes, they will be even bribing people with coffee and donuts from Dunkin Donuts, with a iced coffee giveaway for Yahoo visitors on Friday, September 22.

The advertising campaign is to begin this Thursday.

Folks in the forums are discussing this advertising strategy. A WebmasterWorld thread has the following quotes;

Yes, a doughnut and coffee would be nice. I still remember the Yahoo commercial with the man living in a trailer in the middle of nowhere, who ordered pillows online to avoid damage from a comet. I don't know why I remember that one, but it doesn't matter, because I rarely search with Yahoo.

Perhaps the advertising money should be spent elsewhere. It wouldn't be a bad idea to learn from other's mistakes. For example GM (and Ford) has tried hard to bribe people into buying their cars, but it hasn't worked. But I bet if they invested their time and money into making better cars, overtime the momentum would shift. Same idea goes for Yahoo!, or does it?

You might hate to admit it, but advertising works. But, you have to have a good product and it's not like throwing a light switch, either. Ask's campaign got them some brand awareness. I think it might have been, longer-term, worthwhile. This should pay off for Yahoo. They are building on a strong brand already. Beside, you got a better idea?

The Yahoo! campaign should be a humorous one, looking back at Yahoo!'s old campaigns and you may even hear some yodeling!

Forum discussion at WebmasterWorld.

Extremely weird, conduct a search at Yahoo! for Google and right now, what comes up is google.com but with the title that reads, "Elisha Morgan Gemologists." The click does take you to Google but still, the title is not Google's title. In fact it belongs to this site listed in the Yahoo! Directory.

Here is a picture for proof.

View Large Image

So weird. You will notice that www.emgemologist.com, the site that's title is displayed, goes to google.com.

Forum discussion at DigitalPoint Forums.

As we state time and time again, vertical creep into the search results are becoming ever more so important each and every day. This time, we see a Threadwatch thread with a picture of Yahoo! Answers being added to the search results in a much bigger way then normal.

Here is the way it looks now for a search on beer:

Here is the way it looks from a screen shot at Threadwatch, both are placed directly under the main organic results:

I personally like the current state of integration, but the second picture does stand out a lot more.

Forum discussion at Threadwatch.

If you go to Yahoo!, I bet you'll have a had time trying to use Yahoo!'s Blog Search feature today. Well, at least until it goes back online. Try doing a search at blog.news.search.yahoo.com, it is gone! Yes, it has also been removed from being embedded in Yahoo! News, and has been replaced by Yahoo! Images, for now. A search on google.com at Yahoo! News would typically show both a Yahoo! News listing and Yahoo! Blog Search listing for this site, now we only see a Yahoo! News listing.

Is it gone forever? Nope. At least that is what Greg Jarboe finds from a Yahoo! spokesman, Brian Nelson.

In an email sent a short while later, he added, “Maybe this goes without saying, (but) blog content remains an important part of our overall news and news search strategy. It’s worth mentioning again because I’ve read speculation in the blogosphere about what Y! might be thinking bigger picture when it comes to blog content.”

What is interesting is that the forums are pretty dead on this topic. I was not able to find a WebmasterWorld thread on this, but I did find a Search Engine Watch Forums thread with some limited discussion.

Why didn't Yahoo! post something about this? Why pull it without telling anyone? Heck, the blog community is almost in an uproar about this, as you can imagine. Take a look at Zawodny's recent links section and how he quotes, Steve Rubel;

To me this is a sure sign that Yahoo is gearing up to launch an integrated feed reader/search engine the way Ask.com/Bloglines did in June. This is the second piece of evidence in the puzzle.

Of course Jeremy can't say anything or he would have. Quoting someone else... Well, let's see.

Forum discussion at Search Engine Watch Forums.

This should be an interesting session, Danny Sullivan is moderting this session. We have Jon Glick, ex-Yahooer now at Become.com, also have Bill Atchison, Dan Thies, Rajat Mukherjee of Yahoo and the new famed Vanessa Fox of Google. Brett Tabke is on my right, talking with Jon now about bad bots. Tim Converse is one row behind me on my left. Danny mentioned Brett's fight with bots and had Brett wave at the crowd.

Jon Glick up first. Robots are good at finding links and pulling that content. Bots pull the content but don't do the analysis. Bots are dumb, finicky and they cannot type. Bot friendly sites includes hypertext navigation, well ordered-hierarchical site and clear instructions in your robots.txt. Robot traps include; dynamic content, excessive parameters, and perpetual calendars. Use a robots.txt file to tell the bots what to or not do on your site. You can also used meta tags on a page by page basis, or you can also use the rel nofollow attribute. Well-behaved bots obey the robots.txt file and metatags, they identify themselves (they dont spoof), they dont crawl too aggressively, they provide FAQs, etc. When bots go bad, the most evil bots don't obey the robots.txt and metatags info. How do you detect these? look at your daily logs, do some real-time analysis. Dealing with misbehaving bots; don't hesitate to block them, sometimes just do a 24 hour block, block at the firewall level. You can also try put up a challenges, such as a text code in an image. Be careful who you block; track who gives you traffic.

Dan Thies from SEO Research Labs. Duplicate content is the same content presented on more than one URL. Most web sites do this to themselves. There is also near duplicate content also. There is a difference between filtered from the index and filtered from the search results. Duping yourself; duplicate URLs, shopping sites and near empty pages. Getting Duped; by screen scrapers, RSS feeds and proxy URLs. The impacts on traffic... 10 - 15% of traffic is organic search. After de-duping the site, 20 - 25% came from organic search. Revenue drop was "feelable." Reverse cloaking vs. scrapers: simple user agent detection, if the user agent is not a major SE spider insert; meta name="robots" content="noindex". Screen scrapers that steal an entries pages's HTML get a page that will not be indexed. Easily thwarted by someone who cares to but reduces duplication by scraping substantially. Links by proxy is an old trick. Hack someone else's site to create a link or redirect to one of your sites - either create a page or credit a URL using XSS attack... then link to it using a proxy URL. There are also public proxies that you can use. Proxy URLs as duplicates; thousands of public proxy servers, every URL on the web can be duplicated by them, proxy based duplicates when link to can affect duplicate content filtering. Public proxies pass along the user agent but proxies use their own IPs. How do you stop them? Spider validation vs. proxies; when you get a request from a search engine spider user agent, check the requesting IP address. This is dangerous so use with caution. But what if they get through? Change and rotate content; testimonials, news and headlines and use brute force. The most important page on your site is probably the home page, yet it is the least likely to get changed often (hmmm). Monitoring Dupes; set up monitoring for a signature SERP text that is unique to your pages, home page duplication is the #1 issue, use a second signature for internal pages and he then lists some tools. You can use the DMCA, digital millennium copyright act. Send the hosting provider or the search engines. I'll leave off the challenging the search engines slide.

Bill Atchison from CrawlWall.com is now up. He calls these bad bots, parasites. He said one day, a scraper took down his server. 10% of his traffic was from bad spiders, these parasites. Bad bots ignore robots.txt, spoof bot names, use multiple IPs. They want to get your data to make money. Motivations include, AdSense, YPN, affiliates. Who are these bots? Intelligence gathering bots, content scrapers, data aggregators, link checkers, privacy checkers, etc. Stealth bots vs. visible bots - visible bots are easy to block, the stealth bots are those masking as humans. How scraper bots use your content? He created the name CrawlWall to easily find pages that were unique to that keyword. He used that to locate sites that stole his content with the term CrawlWall. They took several web sites and scrambled the content together, to serve up Google AdSense. He sometimes feeds them back cookie information, so he can then track them better. He logs all this activity. Scrapers also cloak and hide your content. He shows two active proxies that hijack content, that crawled as Googlebot. How do you stop bots? Opt out bot blocking fails; robots.txt only works for the well behaved bots as the most bad bots ignore robots.txt except when trying to avoid spider traps. He went to an optin strategy. He said, only Google, Yahoo, etc. can come into my web site but that can get you intro trouble. You need to review your traffic prior to doing this. He finds Google Analytics very useful. He created a lot of rules to determine the difference between stealth bots versus a visitor. Some bots use cookies, very few bots execute JS, bots hardly every examine CSS files, rarely do bots download images, monitor speed and duration of site access, observe the quantity of page requests, and so on. He will then serve up a image access code to them. Robots.txt is spider trap because stealth crawlers reading this file expose themselves while trying to avoid spider traps. Also anyone visiting your privacy pages, it is probably not your visitor. Avoid search engine pitfalls; dont allow search engines to archive pages as search engine cache is also scraping target. People are also scraping through translation tools. Ways to rpotect your site: USe a script to dynamically display robots.txt and show proper info to allowed bots and all others see disallow. USer agent filtering and blocking with the rules structured for an OPIN allow list. Block entries IP ranges for web hosts that host or facilitate access for scraper sites. For blocking large lists of IPs, such as proxy lists, use PHP and a database like mySQL.

Rajat Mukherjee from Yahoo! is now up. Yahoo! Search Web Crawler is named Slurp. He has news about the new site explorer features. New features include; you can add your site, you then can authenticate your site (looks so much like Google Sitemaps), to authenticate, you place a file on your site and that will authenticate you. You can manage site feeds, rss feeds, etc. In addition to those standard features they added a subdomains filter, a different view of those results and a way to get those data out of the system via flat file or API.

Rajat Mukherjee from Yahoo! then moves on to bot obedience. Slurp is a very obedient bot he said. Read robotstxt.org. He showed us a photograph of slurp, a joke of course. Make sure you allow content you want Yahoo! to get and disallow content the content you dont want them to index. Yahoo! does honor a crawl delay parameter. http://help.yahoo.com/search is very well organized there, plus some new resources added there. Slurp is new and better, they announced it last week. They show the blog posts from Yahoo Search Blog and Loren Bakers blog from 7.28.2006 - where you should see up to 25% reduced load on your sites. He asks who have seen a reduction of load, and about 1.5 people raised their hands out of hundreds. Yahoo! does have multiple crawlers, but please send feedback to Yahoo about these crawlers.

Vanessa Fox from Google is last up. She put up some funny robots.txt files she found, she had no real slides. She talks about google.com/webmasters they announced last week, a tool to check your robots.txt file. She talks briefly about the www vs. the non www issue, which is now at the google.com/webmasters, that allows you to define which is the proper structure, www vs. non www. Every once and a while a host may block a googlebot IP.

Did you know that when you search at Yahoo! for google.com or google you get a Yahoo! Shortcut telling you to search the web with Yahoo and not Google.

Now, if you do a search at Google for yahoo or yahoo.com you do not have Google telling you to search with Google instead of Yahoo!. Heck, you already are searching at Google.

But maybe Yahoo! has research showing that Yahoo! users don't always know when they are searching. So when they do a search for Google or Google.com at Yahoo! Search, Yahoo! needs to remind them, that they also have a search engine. Sounds kind of dumb, but trust me, it makes sense.

What is interesting is that Yahoo! does it for Google keywords but not for MSN Search, Ask.com, etc.

How did I notice this? Well, I noticed a drop in traffic from Yahoo! News and figured it had to do with Yahoo! News vertical integration on Yahoo! Search. Guess what? It was. Google now doesnt come up in news results always on Yahoo! Search, instead you get Yahoo! telling you not to use Google.

Forum discussion at Search Engine Roundtable Forums.

Yahoo launched a new tool in the Korean domain named Webzari. What it does is use the Site Explorer data to map your inlinks on a map, in the form of planets. A link map, in a sense, with AJAX and more flavor.

You can check it out in action by clicking here.

Pretty cool and here is a screen capture.

View Large Image

Forum discussion at Search Engine Roundtable Forums.

Every now and then I check the directory listings I have at Yahoo! to see what is up with them. I have bookmarked them ages ago and when I have some free time I check them out. I checked last night and noticed that Yahoo! has added the "Save to My Web" button to the top right corner of the pages.

Social search, online bookmarking, is found everywhere with Yahoo!. In Yahoo! Local, where I bookmark phone numbers I use every now and then and now in the directory, which gets less and less attention from Yahoo.

The little icon to me is very useful. If you want to see my listings you can check them out in the New York Consulting list or the generic Yahoo! Web Rank.

I posted a forum thread on if this is a new feature for the Yahoo! Directory or not at Search Engine Watch Forums.

Moderator, engine, posted a new thread at WebmasterWorld noting the news that Database guru to lead Yahoo social search.

Yahoo!, if you did not know, invests heavily in social search. From MyWeb, to desktop search to Flickr and Yahoo! Video - social, tagging and all that fun Web 2.0 stuff is something Yahoo! takes seriously.

Yahoo! hired "Dr. Raghu Ramakrishnan, 45, as vice president and Yahoo research fellow in charge of defining the strategy behind Yahoo's "social search" system, based on his expertise in databases, data mining and privacy-preserving technologies."

Hopefully this will make things more interesting over at Yahoo!

Forum discussion at WebmasterWorld.

A WebmasterWorld thread asks what is that light bulb symbolize on the Yahoo! search results page? You know if comes up for some searches and says "Also try:" keyword phrase here...

Well, besides for it being obviously some sort of search suggestion, I decided to pull up the Yahoo! Search help documentation, which you can find here and you will see it says;

Also Try: When other people have done searches similar to yours, we'll list these queries under the search box. One of these might help you narrow your results. Click on a link to try one of these searches.

Forum discussion at WebmasterWorld.

Chris Sherman has the huge write up on Yahoo's settlement of the click fraud case. He describes;

The terms of the settlement include a cash payment of $4.95 million to plaintiffs' counsel and a provision that will allow advertisers to file a claim for Yahoo to investigate potentially fraudulent clicks back through January 2004. Yahoo will pay refunds to advertisers who file claims if it discovers evidence of fraudulent clicks.

Chris explains that the amount does not come close to the $90M Google Settlement but Yahoo is "offering cash refunds, and there is no ceiling on the amount it will refund if it finds evidence of click fraud," where Google offers advertiser credits.

Forum discussion on this settlement at WebmasterWorld and DigitalPoint Forums.

A SearchDay article by Danny and Chris over at Search Engine Watch named Getting The New York Times More Search Engine Friendly talks about how Marshall Simmonds (first with About.com and then acquired by NY Times) made the NYTimes.com search engine friendly. Part of that process is to allow the search engines, including Google, to access, crawl, index and rank content that would require a username and password by a normal Web user.

Danny and Chris ask the question and answer it; "Isn't this cloaking—serving different pages to a search engine and an individual web browser? Yes, it is." Yes, there is a BUT;

Although both Google and Yahoo warn against cloaking, Marshall says both companies are aware of what the Times is doing, and apparently condone the practice.

"They want the content, and they're very interested in displaying it," says Marshall.

Reviewing the latest from Google on cloaking you see that Matt Cutts makes a clear distinction;

So IP delivery is fine, but don't do anything special for Googlebot. Just treat it like a typical user visiting the site.

NYTimes.com is clearly doing something "special for Googlebot" here and in terms of how Matt Cutts defines "acceptable cloaking," this does not fall within those terms. At other engines like Yahoo!, Ask and MSN, engines that have not taken as strong a stance on cloaking, this most likely would be acceptable. But at Google, I believe, based on Matt Cutts continued campaign against cloaking, this would not fall within Google's webmaster guidelines.

Forum discussion at Search Engine Watch Forums.

There is a thread with a fun name at High Rankings Forum named False Gods. The thread discusses how when searching for some "ego keywords" (keywords a person wants to rank well for) he found himself ranking well. But then he noticed that Google personalized search was turned on.

The results within personalized search, no matter which search engine, are tailored to your liking. So if you want to be number one for "seo" you can be over time. Especially if you use the remove result function until your site is #1 and also if you tend to click on your pages more often than others.

Past related article on this that may be of interest is named Search Engine Optimization is Changing So Quickly.

Forum discussion at High Rankings Forum.

The Yahoo! Design Preview became available earlier this week. Since then folks have been testing it out. One thread at WebmasterWorld notes that if you are testing the new design (more on that here) and if you go to Yahoo! Travel you will notice the search box is already filtered to show segmented results for travel search.

After you conduct a search, the results come from "Yahoo! Travel Guides."

It seems to me, that this was also implemented at the normal, non preview, Yahoo! Travel section. I am not sure if it is exactly new - but it is interesting to point out, nevertheless.

Forum discussion at WebmasterWorld.

We scold Ask.com for not using PPC to promote their brand but we most praise Yahoo! for using Google AdSense for promoting their brand.

A WebmasterWorld thread notes that Yahoo! is using Google's contextual ad program, Google AdSense, to promote its own product. It is important to note, that Yahoo! may not be directly advertising in the Google AdWords program. It may be a Yahoo! affiliate who is doing the leg work here. If that is the case, and it probably is, then I will take away my praise from Yahoo! Nah, I am just kidding. Yahoo! has an affiliate program, that deserves some praise also.

It is also important to note that although I scolded Ask.com for not using PPC to promote their search engine, sem4u posted that he did see Ask.com ads in the Google AdSense program as well. So there you go.

Forum discussion at WebmasterWorld.

This time the new Yahoo! home page that we showed is truly available for beta testings. You can test it out by going to http://www.yahoo.com/preview and following the steps. You can also read a full review of the new design here.

Forum discussion at WebmasterWorld.

Yahoo! Search Blog announced that AllTheWeb is the first of the Yahoo! owned search properties to test out a new search technology called Livesearch. Livesearch is similar to Yahoo! Instant Search in that it shows results as you type, but it also "related queries, spelling suggestions, and enables you to use keyboard shortcuts to help you find the right query faster to get to the results that you want." Let me tell you, it feels incredibly quick and sleek.

Only issue, I went to it on Apple Safari and saw this message.

For an optimal Livesearch experience, we only support the following browsers with javascript at this time: Windows 98/2000/XP - IE 6.0, Firefox 1.5 Mac OSX 10.3 - Firefox 1.5 Mac OSX 10.4 - Firefox 1.5 We plan to support Safari soon. Please let us know if there is interest. Sorry for the inconvenience.

Yahoo!, yes there is interest.

Forum discussion at Search Engine Roundtable Forums and Search Engine Watch Forums.

A Search Engine Watch Forum member posted a thread named Yahoo.it censored for 'preteen'? Basically, if you do a search on the term "preteen" in Yahoo! Italy (image), nothing comes up, but the same search at Yahoo.com (image) brings up results.

Danny Sullivan ventures a guess that "Yahoo got some complaint, perhaps a legal one, about child porn sites in its results in Italy."

Who knows for sure, could be some whacky glitch or something.

Forum discussion at Search Engine Watch Forums.

Mid-February we reported on a new Yahoo home page design. WebmasterWorld shows that you can now test the home page yourself. Go to http://events.yahoo.com/326123171.html and follow the steps.

Forum discussion at WebmasterWorld & DigitalPoint Forums.

To clarify before even beginning, Yahoo! does not necessarily uses these techniques, they are just patent applications issued by Yahoo!

Bill Slawski posted an outstanding blog entry at SEW Blog named In Yahoo We Trust - The Link Spam Patent Application discusses one of Yahoo!'s papers and a patent application on fighting Web search spam.

(1) Combating Web Spam with TrustRank which discusses how non spam pages link to non spam pages, as Bill describes in short.

(2) Link-based spam detection which describes, similar to PageRank, the ability to "manually identifying reputable seed pages" and "separating reputable pages from spam pages."

Forum discussion on these topics at Cre8asite Forums & Search Engine Watch Forums.

DigitalPoint Forums reports that Yahoo! has added, what seems to be a totally irrelevant feature, to the Yahoo! Search results page. It does not work in all browsers, so it seems to be one of those tests, it works in Firefox. Do a search on anything, I did it on yahoo and you will noticed this big box with a yellow/brown header on the right side named "Buzz Popular Searches" and it constantly refreshes with popular searches ranked from 1 though 12.

The Yahoo! Buzz Index "is the percentage of Yahoo! users searching for that subject on a given day, multiplied by a constant to make the number easier to read." But what does the listed searches in the buzz index have to do with my search query on "yahoo?" I have no idea!

View Large Image

Forum discussion at DigitalPoint Forums.

I have been tracking forum discussions for years now and one thing I have been noticing over the past several months is how Yahoo! has been losing a lot of the internal forum buzz. Let me clarify that this does not apply to the Yahoo! Search Marketing or Yahoo! Publisher Network side of things, but rather the Yahoo! Search organic side. The forum buzz, to me, it seems, is at an all time low.

Who is to blame here? I don't know? Is Yahoo! not exciting anymore? Does Yahoo! not send enough qualified organic traffic for it to matter to SEMs? Is the lack of some old and well-known search representatives participation in the community having an impact? Is it that Yahoo! Search results seem to be not as relevant as the competitors? Is it all of them?

One personal result that bothers me is that a search on rustybrick doesn't list rustybrick.com as number one. Google seems to have no problems with it, MSN gets it kinda right, Ask.com does a superb job (I even got binoculars). Some people are blaming the recent Yahoo! Search issues to an RSS handling issue but I see enough evidence to prove that wrong sometimes and sometimes not wrong (depends on when you search - which is bad, really bad.

Forum discussion at Search Engine Roundtable Forums, named Is Yahoo! Web Search Losing Its Fire?

Earth Day was this past Sunday and there was a ton of news and blog buzz on the search engines being creative with their engines on that day however, there were no threads in our forums. Today, I finally noticed a thread at DigitalPoint Forums about the special day. The thread creator notes which engines made an effort for Earth Day and which did not - also asking you to comment on which you find to be your favorite.

Continue reading "Search Engine Commemorate Earth Day 2006"

Some folks have been noticing screen captures of the the advertised sites in Yahoo Sponsored Results, within the Yahoo! News network. AJ has taken a screen capture after he viewed this page, I personally see tax ads at the bottom, but since he is not US based, it is possible they are not showing tax season ads.

Forum discussion at DigitalPoint Forums.

There are two spiders from Yahoo! that you may have been noticing crawling over pages recently. The first, is named Yahoo! Slurp China has reportedly been around since mid-Novmember of last year. The second, is named Yahoo! Search Marketing Crawler and has just been discovered, I believe.

Yahoo! Slurp China found recent activity as reported at DigitalPoint Forums under the User Agent;

Mozilla/5.0 (compatible; Yahoo! Slurp China; Agent: Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)

WebmasterWorld has an older thread on the topic that IP block is 202.160.178. and 202.160.179. and it resolves to *.inktomisearch.com. If you don't like this spider, the folks at WebmasterWorld say you can just block its user agent and it won't affect the main Yahoo! Slurp. They say block; "User-agent: slurp china"

Yahoo! Search Marketing Crawler is a fairly unknown spider that sprung up some recent concern at WebmasterWorld Forums. It has been reported from the IP address 66.35.192.197 and 64.209.232.29, which is resolving to Savvis and not Yahoo! Senior Member, StupidScript, says that this spider is an automated ad-checker. He also said that Yahoo! has never claimed ownership of this bot and that they blocked it only to stop "receiving mail from SBC-Yahoo subscribers." I am not 100% sure about this so I will do a follow up with Yahoo! Search Marketing on this crawler.

The other day we reported the Yahoo! Search Index Update, today word comes via DigitalPoint Forums of a Yahoo! Image Search index update. That means, that Yahoo Image Search has a new stock of fresh images for you to search on. It also means, you may be getting more or less search referrals from Yahoo! Image Search.

Forum discussion at DigitalPoint Forums.

As I covered a few days ago at SEW Blog, Yahoo has updated their search toolbars for IE and for FireFox. The big change is tabbed browsing for IE and adding a button to bookmark pages to del.icio.us, which is a nice addition.

The folks over at WebmasterWorld are discussing this update. Where Marcia says she uses the toolbar heavily for anti spyware, she likes it because it uses less resources then other toolbars (she says).

Forum discussion at WebmasterWorld.

In our continued Ask The Moderators thread, the next question we explored was by Viggen. He asked our roundtable of moderators the following question on what he calls; "private search engines."

what search engines are you using for private searches and why...

We opened up a thread named Private Search Engines Explored for moderator only discussion, and last night opened it up for member discussion as well.

I found it interesting in how each moderator interpreted the phrase "private searches." On one hand, does this mean searches conducted at a password protected search engine? Or maybe it means, searches conducted on a company intranet? Perhaps, he is asking about searches one does each day, but are private in nature? Or maybe he is asking about vertical search engines?

The roundtable of moderator's responses were pretty vast.

Rand Fishkin discussed how he uses Del.icio.us to search on tagged content, Ask.com for "non-search type searches" and Yahoo! for link command searches.

Dazzlindonna explained that if private search engines means vertical search engines, she doesn't use them. Donna is a big fan of major search engines and bookmarking for private searches.

Darrin Ward sticks with Yahoo! Search as his default and Google as his back up, he may also use MSN.

Ben (Phoenix) says he doesn't value Yahoo! Search at all, he sticks with Ask.com and Google. He also is a big user of Google Alerts. For internal private searches he uses Desktop Search, like Google Desktop Search. As now uses Bloglines more and more each day.

I personally use RSS News Searches exhaustedly. I subscribe to searches on a few dozen keyword phrases to be notified via RSS about the latest news and discussions taking place about those keyword phrases.

We would love you to join the conversation at Search Engine Roundtable Forums.

If you visit Ask.com at http://www.ask.com/ today, you will notice that the Ask.com home page is green. They have changed the color from red for St. Patrick's Day. In the past Jeeves dressed up; see March 17, 2004 and March 17, 2005. Clicking on the Ask.com green logo takes you to a St. Patrick's Day search as does with Google and Yahoo! (takes you to a non search page) that are also sporting logos for the special day. However, Ask.com's inner search pages are still red, and so is http://uk.ask.com/, however, they do have a special link to a St. Patrick's Day search.

Forum discussion at Search Engine Roundtable Forums.

All the debate if Yahoo! is a more relevant or better search engine than Google continues. We ran our Search Engine Relevancy Challenge, which showed at the 10,000 result mark that Yahoo! was more relevant than Google at the time of the study. But at some point, Google took the lead at the live results page to have a 0.1764 lead over Yahoo!, not sure when that happened, last time I checked Yahoo! was in the lead.

I found a thread at DigitalPoint Forums named Interesting Google story from my daughter last night. The story shows how an 11-year-old tried to find material for school on Google and was to frustrated that she switched search engines to Yahoo! She discovered what she was looking for, ultimately at Yahoo! Search. Here is the story as documented at the forums;

Daughter says, "So dad I was working on my architecture project and presentation last night for social studies. I ended up using Yahoo to find my answers instead of Google."

Dad says, "Oh? Why is that?"

Daughter says, "I tried Google first and all I was finding was amazon site trying to sell me books on architecture."

Dad says, "So then you tried Yahoo?"

Daughter says, "Yes. I was able to find 2 sites on the first page of Yahoo last night that helped me a lot on my project!"

Dad says, "You couldn't find anything on Google?"

Daughter says, "No. Not on the first page. I gave up."

Dad says, "What words did you use in your search?"

Daughter says, "I don't remember. I was trying to find different blue prints and general architecture information."

Dad says, "Well that is why there is more then one search engine, good for you trying more then just one!"

So which engine is the most relevant? I think the last line sums it up well. "Well that is why there is more then one search engine, good for you trying more then just one!"

Forum discussion at DigitalPoint Forums.

Yahoo! posted a new blog entry last night named Going deeper into the Wikipedia which describes Yahoo!'s approach to utilizing the Wikipedia. If you do a search and a wiki result is found within the SERPs, no matter if it is #1 or 100, it shows "Quick Links" to deeper wiki pages. For example, conduct a search on coffee and scroll down to the wiki result. You will see links to "Etymology and history - The cafe - Coffee bean types" on the site of Quick Links.

Google's approach is a bit different, they place the Wikipedia results, when they deem it appropriate, in the One Box location. Giving them prime real estate, and placing wikipedia results at the top, which may not be as useful as the way Yahoo! does it. So does Google still treat Wiki too well?

Forum discussion at Search Engine Watch Forums.

Danny wrote Yahoo Surveys Search Rewards Idea where he covers a News.com article showing how a group of Yahoo! Mail users were offered "10 different potential reward options" to take a Yahoo! search survey. Kinda funny, I told them they should do this at last years SES San Jose conference - that they don't have to necessarily pay money to get answers. I am sure it wasn't my influence, since it did take almost a year to implement.

What is funny is the diverse feedback on this at WebmasterWorld.

An interesting strategy, and one that I'd never thought might be adopted.

The beginning of the end for Y! Search...

Looks like they're serious about trying to put a dent into AdWords/AdSense and this is just another piece of the puzzle.

Viggen started a thread at our forums named Does Google hire spammers? It is actually a very interesting question. I asked Tim Mayer of Yahoo! this question at the Yahoo! Party at the Palm's Club Rain at WebmasterWorld Pub Con Vegas 2004. This is how the conversation went...

Tim Mayer came over to me, when I was sitting on some sofa, kind of off in the corner. I asked him he they (Yahoo!) hires top notch spammers in an effort to combat spam. You know, like how governments and large companies hire hackers to prevent being hacked. Tim said they have not, they just hire 'engineers'. Which got me thinking, what if the Yahoo! people decided to pass some special gas through the air at this party. The gas contained a drug that turned spammers into the extreme opposite of a spammer (just a note to readers, I am not using the word 'spammer' in a derogatory fashion). I told Tim, that if they had this solution, it might solve a huge chunk of the spam issues they have overnight. Of course I was joking, everyone at the party were clean, white hats.

But if you look at recent patterns, search engines engineers and top folks are "buddies" with so called spammers. As randfish points out in the thread; "MSN certainly gets the opinions of spammers - particularly in last year's search champs." And yes, Matt Cutts from Google goes out for dinners and talks with spammers very often. Not only that at last years Google Dance DaveN and friends spent a whole night talking with Larry or Sergy (I forget which one) and Matt Cutts in the Google Plex. Yahoo! also invited a bunch of people, half consisting of self-proclaimed search engine spammers to Yahoo! headquarters to discuss the search technology.

Has a search engine ever hired a spammer? Gary Price hired by Ask Jeeves is far from Ask Jeeves hiring a spammer. I did hear a rumor of an old WebmasterWorld player who switched sides of the fence, but I do not know much more about that.

As far as I know, a spammer has never been hired by a search engine as an employee. But yes, they do serve up nice tokens to spammers.

Forum discussion at Search Engine Roundtable Forums

Marcia posted a thread at Search Engine Watch forums named Plagiarism, Splogs and Search Engine Spam where she quoted a blog entry at Plagiarism Today named Why Sploggers Splog The snippet Marcia selected was ironic, to say the least.

One of the interesting things that came out of my discussion with the reformed splogger is that Google is not the target of splogs. As odd as it may seem, Yahoo indexes entire sites much more quickly than Google and is even faster at picking up Blogspot blogs because it considers it such an important domain. Thus, even though the service is wholly owned by Google itself, Yahoo is the first to snatch up links contained with it....

The desired end result is that Yahoo searchers will be directed to the junk domains where they will then click on the Google Adsense ads. This arrangement is not only very profitable for the splogger, since they get a sizeable chunk of the revenue from each ad click, but is very beneficial to Google as they are getting money directly from Yahoo’s visitors.

Now I am not sure if there is a study done that shows that:
(1) "Yahoo indexes entire sites much more quickly than Google", I hear that not to be true. That Google is quicker, but there has never been a wide study done on this. I am basing that on forum chatter for the past four years or so.
(2) [Yahoo] "is even faster at picking up Blogspot blogs because it considers it such an important domain." Again, not sure where this is from but I won't argue.

But if those two statements above are true, which I do not think they are a 100% true, then it is funny.

Yahoo! ranks blogspot splogs high, Yahoo users click through, yahoo users are more likely to click on an AdSense ad, Google makes money, sploggers make money.

Of course this can be applied to any search engine, even Google. And it can be applied to any advertising program that is easy to sign up with and pays on a cpc basis, even Yahoo!.

Forum discussion at Search Engine Watch Forums.

Brett Tabke at WebmasterWorld posted a thread named Yahoo Captitulates - Gives Up Goal of Being #1 in Search quoting an article from Seattle Pi.

"We don't think it's reasonable to assume we're going to gain a lot of share from Google," Chief Financial Officer Susan Decker said in an interview. "It's not our goal to be No. 1 in Internet search. We would be very happy to maintain our market share."

Wow! So what are the forum folks saying about this?

they don`t want to set expectations to high, that way they can`t fail! it is a win-win situation.

Yahoo going for the underdog approach?

I'm sure the shareholders will just eat that one up.

Then they need to ditch their search engine, or sell it, and concentrate on being a portal and/or something else.

Forum discussion at WebmasterWorld.

YHOO, Yahoo!'s Stock, took a beating the other day after announcing profits were up almost 40% from the previous year. So in the forums, search forums, we have people discussing wether Yahoo! is at a price where it would be considered a "steal". You have this discussion at WebmasterWorld and at DigitalPoint Forums.

I have nothing to say on this topic. :)

It is true, Yahoo! search went down at new years. Saturday night my fiance does a search and she complains that Yahoo! is not working. I am like, sure its not working (thinking she is doing something wrong). So I mossy over to the computer and punch in a search term into Yahoo! and presto, I get;

Oops!

We ran into a temporary problem while performing your search. Please try your search again.

But it wasn't just seen by the two of us. Many at WebmasterWorld and at Cre8asite Forums and DigitalPoint Forums reported it was down. Heck even GoogleGuy tried it out and said;

Weird. For a second I thought April Fools had come early. :) Working fine for me now..

Too funny. This is not the first time a wide spread report of an outage was reported in 2005. On June 13th there was a reported outage at Yahoo! as well. Maybe this was a Y2K plus 6 years problem?

Brett Tabke posted a featured thread at WebmasterWorld named MSN and Yahoo In Talks? where he quotes an article named Microsoft eyeing deal to rival Google-AOL.

Microsoft may be cooking up a major internet partnership to rival Google's newly bolstered relationship with American Online (AOL), according to a blog posting by a Microsoft manager.

The blog entry that started all these rumors is named Whoa! A Major Player Looks To Take Google Down A Peg.

Forum discussion at WebmasterWorld. There is actually a thread at WebmasterWorld on this topic from December 23rd.

Yahoo, Google and Ask Jeeves all are sporting customized logos for the holiday season. When you go to Yahoo.com and click on the top center logo it takes you to http://events.yahoo.com/holiday05/. When you go to Google.com and click on the middle center logo it takes you to Google's first of many holiday season doodles at http://www.google.com/doodle10.html (more to come). And if you go to Ask.com and click on the Jeeves logo, it takes you to a search results page (as a search engine should, imo) for http://www.ask.com/web?q=Happy+Holidays. Now Ask was sporting a snowman logo yesterday, so they are changing things up, possibly daily for the holiday season, keep and eye on them and Google for logo changes.

Folks are discussing the Google logo at DigitalPoint Forums. And I started a thread for Ask Jeeves at our forums here.

Yahoo! came up with a neat idea, Time Saving Search Shortcuts where you can customize your Yahoo! shortcuts at http://search.yahoo.com/osc/create.

Pretty neat.

Forum discussion at Search Engine Roundtable Forums and WebmasterWorld.

According to a post by Tim Mayer at Yahoo! Search at Search Engine Watch Forums, Yahoo! will have an official response to the debate on Jeremy selling text links on his site. You see, Jeremy is a Yahoo! employee, who specifically works in the search department. But yet, this is his personal blog.

I will keep you posted on any official response from Tim Mayer and Yahoo! on this highly debatable topic.

Mobile is definitely one of those frontiers the search engines are going after. msgraph at Search Engine Watch Forums posted a thread named Yahoo! Go Trademark: go.yahoo.com = Connected Life hookup describing a new trademark Yahoo! filed for; "Yahoo! Go". He believes it has a lot to do with Yahoo Acquiring VerdiSoft Mobile Sync Company.

The URL is at http://go.yahoo.com/.

Last year we had creative logos from all Google, Yahoo and Ask - oh lets not forget gmail's logo. We even had a Turkey Day Google Backlink Update! Today, on Thanksgiving 2005, we have some new logos to share with you.

We have a logo from Yahoo! which links to the Yahoo! Holiday Guide 2005.

We have a logo from Ask Jeeves, which links to one of those nice smart answers on Thanksgiving. I also decided to post a thread on Ask's Turkey day logo at SEW Forums.

We do not yet have an official Google Holiday logo for Thanksgiving yet. But Gmail does have a logo for the day.

Update: Google uploaded it's Thanksgiving logo, which links to a search on thanksgiving.

Happy Thanksgiving All!

Search Engine Watch has the ultimate coverage on this, both Danny, Chris and Gary all have blog entries on the way Yahoo! integrated blog results within the news search engine.

Chris Sherman writes Yahoo Integrates News, Blogs & Flickr Search Results

Danny Sullivan writes Thoughts On & Playing With Yahoo's Blog Search.

Gary Price writes Yahoo Should Let News Search Users Know What a Blog Is; Search Only Blogs From Yahoo News/Blog Integration.

Basically, if you do a news search, for example, on wedding via ask jeeves, on the right hand side, you will find "BLOGS BETA", with blog results followed by a link to More Blog results...

It looks a like the following, but Gary doesn't like the fact they name it blog, and leave it at that. Many news readers might not know what blog means.

View Large Image

Forum discussion at Search Engine Watch Forums.

Thanks to Natasha "That Girl from Marketing" for offering insight into a new feature available for misspellers using Yahoo! Search in this thread at the Search Engine Watch Forum. She described misspelling the word “jewellery” (or “jewelry,” depending on your location –see thread for more) in the search, and not only did she receive a notice of a possible misspelling, but Yahoo! automatically changed the spelling in the search to what it felt was correct, and returned those results. I did a similar search and found the same correction.

Will this put to waste all that hard work stuffing META tags with misspellings that so many SEO’s have performed for clients? (FYI for those that don’t know me that was a joke.) It will be interesting to see if Google keeps its current style of offering a replacement search, or will also begin to substitute the “correct” spelling into the query. Currently, ask.com offers just a suggestion, and MSN offers no such suggestion or action, instead returning almost 700,00 results for my specific spelling faux-pas.

An interesting thread at WebmasterWorld reports Slurp entering site through Overture Ads. The thread creator reports this behavior only on Thursday mornings.

On four seperate days (all Thursday mornings) we had Slurp bombarding our site for about an hour with over 1000 requests, all of these being referals from our Overture Ads

An other member confirms the same activity on his site;

show an abnormal amount of Slurp activity between 07:00 and 14:00 every other Thursday (as well as on a couple of Saturdays). Each time it's about 1,000 visits from the Inktomi IPs.

The interesting portion is that the other member reports that the Inktomi bots are coming through on ads that they pay for but are not being charged to them. BrotherAl says, "Some of the keywords used by Slurp to get to our site have never been clicked according to Overture's stats, so we're pretty sure this activity isn't being charged for."

Can it be some sort of destination page verification process?

Last night Yahoo! blogged a topic they named Fewer clicks, more answers..., which was there way of describing a new feature added to the SERPs page, called "Quick Links".

So for example, you do a search on Walmart at Yahoo! and you see a new line under the main result, named "Quick Links."

Now only that, you get Quick Links to Local results when you do a search in the format of millenium restaurant, but I tried a bunch of restaurants in my area and the local results didn't come up.

I posted a forum thread at Search Engine Watch Forums, I am surprised that other forums are not talking about this. I mean, Google did something a bit similar about a month and a half ago named Google Search Web Categories Revised and the forums were buzzing on it. I'll give the forums more time to catch up on Yahoo!'s late news on this topic, but Google didn't even announce this.

Gary Price, again reports, that Yahoo Cache Now Offers Direct Links to Wayback Machine. He links to the bbcworldwide.com cache link in Yahoo! as an example. The cache header reads;

Below is a cache of http://www.bbcworldwide.com/. It's a snapshot of the page taken as our search engine crawled the Web. We've highlighted the words: bbc The web site itself may have changed. You can check the current page (without highlighting) or check for previous versions at the Internet Archive.

The words "Internet Archive" link to http://web.archive.org/web/*/http://www.bbcworldwide.com/. Nice little addition, IMO.

First forum coverage I found on this was at WebmasterWorld on September 6, 2005. I just posted a new thread at Search Engine Watch Forums.

It's Google's "I'm Feeling Lucky" for the year 2005. You type in a search phrase into http://instant.search.yahoo.com/ and watch this contextual bubble overlay the bottom portion of the page, within that contextual bubble you find the number one result for your query (sometimes). Here is an example of the number one result for the search phrase, "search engine", at Yahoo!.

Danny has a large write up at the Search Engine Watch Blog and I started a forum at the Search Engine Watch Forums.

Reports come way from WebmasterWorld that Yahoo! is indexing new sites via whois data. One member reports;

I registered a new domain mid August and put the in-development site live a week later. I had no reason to look at logs until today - no links in and no submissions. I've just noticed that Slurp came by last week. It followed hot on the heels of a Whois IP. Is this normal of Yahoo! to check out newly registered domains.

We have discussed a bit in the past on speculation on how Google Finds Newly Registered Domain Names.

Ever want a way to quickly check all your .edu backlinks, or backlinks from a specific site? Yahoo! now allows you to do so with the linkdomain command. It is simple, just use the follow syntax linkdomain:www.rustybrick.com inurl:edu to check all your backlinks from .edu sites or linkdomain:www.rustybrick.com site:www.seroundtable.com to find all the backlinks from this site to rustybrick.com. And an other useful feature, is to use the allinanchor to find out who is linking to you with particular keywords, for example linkdomain:www.seroundtable.com allinanchor:blog (doesn't seem to support allinachor after a second look). Outstanding!

I am pretty sure this is new, as is Shawn, who discovered it and posted a thread about this at DigitalPoint Forums. He named the thread Yahoo Allowing Operators With Link Functions. I believe Yahoo! upgraded the command to facilitate the up and coming, Yahoo! Site Explorer product, like they recently did last week with making the link command more accurate. I have not received an official response from Yahoo! about this change, as of yet.

It is very interesting to watch Local Search grow and mature. Today I found a WebmasterWorld thread named Yahoo Local Integrates Editorial Reviews. That is right, instead of the normal user reviews, they have added professional editorial style reviews. One example is of Connie's Pizza in Chicago, IL, with an editorial review.

WebmasterWorld members note that the link to vote if the review is helpful or not, is missing from the editorial reviews and is only an option for the user reviews. Here is one member's concern;

I agree with you about the potential problem of Editorial reviews trumping user generated reviews. A novice user could easily interpret the logo'd Gayot review as the authoritative opinion on the business. On the other hand, when I buy a book from Amazon the first review is always the editorial review, and then I scroll down to read user reviews. I know the difference between the two, but I can't speak for novice users. This would be interesting to explore.

Forum discussion at WebmasterWorld.

Last week we discussed Yahoo! Local twice; (1) Yahoo Local & 360˚ Adds Vertical Reviews and (2) Yahoo! Local Using Wrong Localized English. Today, I want to point you to a thread I started at SEW Forums the other week, named Yahoo! Local Driving Web Leads?

Basically, some are seeing a positive affect from having a listing at Yahoo! Local and also Google Local. I personally have not "noticed any significant referrals from Yahoo! Local in my stats." (I just quoted myself)... Member, MoneyMan said, "It's worth noting that in terms of "click-through" and viewing detailed business data, the results I have seen from YLocal on clients have been fantastic."

It is a bit early to tell, but I would love to get more feedback at the thread or over here.

The other day, when I was reporting on Yahoo Local & 360˚ Adds Vertical Reviews I stumbled across my company's profile also at http://local.yahoo.com/details?id=11262068.

What I noticed was that under the "Payment" type section, it reads "Visa, MasterCard, American Express, Cheque."

Notice anything weird? "Cheque" is not how good American's spell "check." Yahoo! is a US based company, they know that my business is in New York, USA - they should know to spell using American English, right?

I know, I am giving them a hard time. I just thought it was funny so I posted a thread at Search Engine Watch Forums named Yahoo! Local Bad with Localized Language describing this. It lead into people rating my company, which I already got myself in trouble with (see bottom).

Many of you know I am a big fan of Yahoo! 360, I use it, I believe, the way Yahoo! intended. That is, I use it to connect the dots in my life, work and blogosphere. One recent addition was the ability to use the new Yahoo! Local at http://local.yahoo.com/ to conduct vertical reviews. By vertical, we mean, rate a restaurant based on food, ambiance, service, value and a hotel based on amenities, room quality, service, and value.

View Large Image View Large Image

Forum discussion at WebmasterWorld. Also you can view all my yahoo local reviews if you want, I only have four or so.

Yahoo! released its updated user interface and and feature set for Yahoo! Local. John Battelle has a very nice write up on it with inside information from Paul Levine, head of Yahoo Local Search. Danny Sullivan gives a write up of feature sets. Let's not forget to add Jeff Weiner's thoughts on local search and the changing face of media.

Forum discussion on this topic at WebmasterWorld.

Update: Moderator, Chicago's thoughts on this new Yahoo! Local:

- A deeper more broad definition of "local search"
- Fostering a sense of community
- Movement from business lookup to an augmentation of local behavior patterns
- The realization that local search includes events, activities, and other time-based local actions
- Personalization and leveraging user data
- Deeper content integration from disparate local based content within the Y network in a local destination
- Drill down: From city to neighborhood
- Freshness
- Destination: City portal characteristics
- Further emphasis on user generated content
- One step closer to the integration of social networking

I have, for the most part, stayed out of the debate with the recent controversy over if Yahoo!'s index size is inflated or not. All I did was link to forum threads and articles on the topic.

I sat down with Gary Price, News Editor at Search Engine Watch, at the last SES conference and one of the many topics we discussed was this one. Forget the controversy over index size, I have no way of knowing if the 19.2 billion count is accurate or not. That is not the point...

To measure index size by conducting queries, in my humble opinion, is not a valid testing method. Why? Because search engines have an indexer component and a query component. Both of those are pretty much standalone. So for someone to do a query, no matter what it is, at one engine and compare the "total results" to an other search engine, to me, it seems very flawed.

Some of the reasoning on this can be found by a blog entry at sethf.com. The query processor can be limiting the results based on its matching and filter algorithms. One can not measure index size, when it goes through a query processor. In my opinion, it really doesn't matter that Yahoo! claims to be more then twice the size of Google and still doesn't show, for any of those queries, more total results. I know many people argue with me, but this is how I felt from day two - after speaking with Gary about the methodologies being used to measure Yahoo!'s claim.

Update: Jeremy Z just posted a few minutes ago his via on Of Course Size Matters! Got of love the blog-o-sphere.

<