Indexing Web Pages

There are four parts to an engine that you need to know about for optimization purposes:

1. The spider is a program that goes out across the internet, looks for and gathers up web
   pages.
2. The database is where the spider will store the pages that it finds.
3. The search engine website, is where searchers go to pull up information
   from the database.
4. The algorithms are programs that determine which sites will come up when searchers type in
   a query at the search engine website.

Get into the search engines database:

• The spider will automatically find your site from a link on someone else’s site which is the
  path we recommend if you can get 1 or more quality inbound link(s).
• You submit your URL so that the spider will come out and find it.

What Happens When I Submit my URL to a Search Engine?

First, the search engine’s spider visits your URL immediately and schedules your page for inclusion in he search engine’s database.

Second, usually within a few weeks, the spider comes along and places your page(s) into its database. There is no telling how many pages deep the spider will crawl or how many pages it will place in the database. Usually, on the first time around, it will be only a few pages - possibly only the home page.

Third, the spider revisits your page(s) to grab any changes you’ve made. (The old term for this was “automatic update.”) Once a page is in the database, the spider usually revisits every few weeks. The spider will also begin to crawl your site more deeply and place more and more of your pages into the database.