Robots.txt Study Shows Webmasters Favor Google
November 15th, 2007 by
The Pennsylvania State University conducted a study that showed Webmasters favored Google over other search engines. The study looked at which robots or crawlers were listed in a web site’s robots.txt file, and Google was listed more often than any other search engine. The paper is named Determining Bias to Search Engines from Robots.txt (PDF) (it may be slow, so here is a local copy) and showed some interesting details.
This entry was posted on Thursday, November 15th, 2007 at 8:48 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.