 |
|
|
|
What are 'spiders' and 'robots'?
Search engines compile their databases by employing "spiders" or "robots" ("bots") to crawl through cyberspace from link to link, identifying pages. Spiders are sophisticated programs that record and index the words of your Web pages. Sites with no links to other pages may be missed by spiders altogether. Once the spiders get to a Web site, they typically index most of the words on the publicly available pages at the site. Spiders do not see the beautiful graphics that you have painstakingly prepared - they only see the source code, or the "skeleton" of your Web site.
It is the task of Web designers to optimize each Web design for high Search Engine ranking (See Bells and Whistles).
When a surfer undertakes a search on a keyword or keyword combinations, the search engine applies an algorithm to the data collected by their robots.
The search engines like simplicity. The simpler your Web site is, the easier it is for the engines to determine the nature of your Web site. And, if the search engines can determine exactly what your Web site is about, you have a better chance at top rankings.
Although it is possible for your Web site to be found by a Spider, you should submit your Web site to search engines and directories to enhance your chance of eventual inclusion in their databases.
When you search the web using a search engine, it scans its index of sites to match your keywords and phrases with those in the texts of documents within the engine's database.
Spiders regularly return to the web pages they index to look for changes. When changes occur, the index is updated to reflect the new information. However, the process of updating can take a while, depending upon how often the spiders make their rounds and then, how promptly the information they gather is added to the index. Until a page has been both "spidered" AND "indexed," you won't be able to access the new information.
WHAT ARE THE PROS AND CONS OF SEARCH ENGINES?
Search engines provide access to a fairly large portion of the publicly available pages on the Web, which itself is growing exponentially (see OCLC's statistical page, "How Big Is the Web?") ranking techniques
Search engines are the best means devised yet for searching the web. Stranded in the middle of this global electronic library of information without either a card catalogue or any recognizable structure, how else are you going to find what you're looking for?
On the down side, the sheer number of words indexed by search engines increases the likelihood that they will return hundreds of thousands of irrelevant responses to simple search requests. Remember, they will return lengthy documents in which your keyword appears only once.
The following are SEARCH ENGINES:Your Web pages should be designed to cater for these 'spiders' with the correct sort of information if you are to achieve high ranking. All your pages should be linked so that the spiders can find them all. If the search spiders are happy then the Web surfers they direct to your site are likely to be happy.