However, while bidding $1,000 on every keyword and ranking #1 for every relevant search sounds nice in theory, most businesses have to play a balancing game between ranking higher and paying too much for clicks. After all, if it costs $17.56 to rank in position #1, but you can only afford to pay $5.00 per click, bidding $1,000 on a keyword to guarantee yourself the #1 position would be a great way to bid yourself out of business.
Webmasters and content providers began optimizing websites for search engines in the mid-1990s, as the first search engines were cataloging the early Web. Initially, all webmasters only needed to submit the address of a page, or URL, to the various engines which would send a "spider" to "crawl" that page, extract links to other pages from it, and return information found on the page to be indexed. The process involves a search engine spider downloading a page and storing it on the search engine's own server. A second program, known as an indexer, extracts information about the page, such as the words it contains, where they are located, and any weight for specific words, as well as all links the page contains. All of this information is then placed into a scheduler for crawling at a later date.