The Search Spider

Theoretically, a spider can begin traveling the internet from a single web page and follows networks of links to grab information from your Web pages. The information it grabs is sorted according to specific algorithms or different criteria when determining their ranking system. One problem of search engine positioning is that search engines are always changing their algorithms. This means that each one will sort out the information that is needed for their database. One engine may consider a particular factor such as "popularity" to be important while another may look at "themes" to be the most important information.

The spider will visit the web page and perform the following tasks:
1. confirm the existence of the web page; may index at that time or come back at a later date
2. index page content (using partial or full-text indexing)
3. identify all hyperlinks to other Web resources (this is an indication of popularity and is becoming increasingly important)
4. proceed to follow the hyperlinks, visiting the referenced Web pages and repeating steps 1-3 above

Most spiders "crawl" the Web on a schedule and when Internet network traffic is at its lowest, such as during the middle of the night.It is then there is a change of search results on major seos like google,altavista and etc.