How Easily Search Engine Spider Your Web Page

SpiderTest

Besides having good content and search engine optimization (SEO), your web site also must have a good spiderability.

In other words, your web page must be able to be read, interpreted, and indexed by search engine effectively.

SpiderTest is a free web tool to test how easily search engine spiders can spider the content of your web page. I found SpiderTest is useful to crawl all your web pages and it checks if your HTML is valid and if links point to existing pages. To test your web page:

  1. Go to SpiderTest.
  2. Enter your web address and click Check.
  3. SpiderTest will read your web page and run tests on it to tell you what you need to do to improve its spiderability, such as meta tag content, file robots.txt, stylesheet and javascript, top 20 most common words, etc.
October 16, 2007 | Filed Under Testing Tool 

Comments

Leave a Reply

You must be logged in to post a comment.