You are in the: Small Business Computing Channelarrow View Sites +
Small Business Technology
» ECommerce-Guide | Small Business Computing | Webopedia | WinPlanet |  »Close
Enter a word for a definition... ...or choose a computer category.

   Term of the Day
   New Terms
   New Links
   Quick Reference
   Did You Know?
   Tech Support
   Webopedia Jobs
   About Us
   Link to Us

talk to us
   Submit a URL
   Suggest a Term
   Report an Error
Internet Lists
Internet News
Internet Resources
Linux/Open Source
Personal Technology
Small Business
Windows Technology
xSP Resources
Corporate Info
Tech Jobs
E-mail Offers
  Be a Commerce Partner

Last modified: Wednesday, March 03, 2004 

A program that automatically fetches Web pages. Spiders are used to feed pages to search engines. It's called a spider because it crawls over the Web. Another term for these programs is webcrawler.

Because most Web pages contain links to other pages, a spider can start almost anywhere. As soon as it sees a link to another page, it goes off and fetches it. Large search engines, like Alta Vista, have many spiders working in parallel.

Also see How Web Search Engines Work in the Did You Know . . . ? section of Webopedia.

E-mail this definition to a colleague

Sponsored listings

CrownPeak: Web Site Search Functionality - Offers an on-demand web site search solution. Software-as-a-service: completely managed, highly reliable, and very economical.

Hoover's: Web Site Search Functionality - Provides in-depth business and public/private company information, including profiles, industry overviews, financials, officers, competitors and news.

Hoover's: Spider Resources Inc. Company Research - Find information on Spider Resources Inc. with operations and products, financials, officers, competitors and more at Hoover's.

For pages about spider . Also check out the following links!

Related Links

Making Your Pages Easy for Search Engines to Index 
Achieving high rankings for Web sites on search engines has become a black art. An article from the Web Developer's Virtual Library.

Invite Search Engine Spiders Into Your Dynamic Web Site
Increase the traffic to your dynamic site by making your valuable content visible to search engine spiders.

LiveLink Spider home page
Describes LiveLink Spider, the crawler software used in the Open Text Index, available from Open Text Corporation.

Optimization for Dynamic Web Sites
With just a few quick selections by the user visiting your site, a page can be generated on the fly based on nothing more than the user's specific interest or request.

Yahoo!'s web searching robots and spiders page
Yahoo!'s directory of web searching robots and spiders.

related categories

Search Engines

World Wide Web

related terms

Alta Vista


dynamic URL

link farming


query string


search engine



site map

spider trap

Give Us Your

spider Products
Compare Products,Prices and Stores

Shop by Category:
Computer Games
7 Model Matches

1018 Store Offers

203 Store Offers

PlayStation 1 Games
1 Model Matches

Home Furnishings
849 Store Offers

JupiterWeb networks:

Search JupiterWeb:

Jupitermedia Corporation has two divisions:
Jupiterimages andJupiterWeb

Legal Notices, Licensing, Reprints, & Permissions, Privacy Policy.

Jupitermedia Corporate Info | Newsletters | Tech Jobs | Shopping | E-mail Offers