Search Engines before Google

, ,
The History of Technology includes the evolution of search engines before Google.

Google came to dominate the search engine market early in the 21st century, but before that there were many approaches to searching the Internet. Some of them are still around, some are gone, and some exist only as preserved curiosities. Their story makes an interesting part of the history of technology.

The first search engine didn’t use the Web at all. In 1990, Alan Emtage created Archie (playing on “archive”) at McGill University. FTP was still the most popular way of publishing information on the Internet, and Archie indexed public FTP sites for searching. Archie servers appeared at several locations, where users could access them by Telnet. An Archie server still exists at the University of Warsaw.

Early Web search sites

In 1991, Tim Berners-Lee embarked on a project that would be hopeless today: Creating a catalogue of the whole Web! Called the WWW Virtual Library, it still exists today, distributed among many maintainers around the world. Its components vary in approach, and many of the links are broken, but it’s an interesting research resource.

In 1994, the first crawler-based search engine appeared. Crawlers, also known as spiders, search the Web from link to link, trying to find everything. WebCrawler and Lycos were the first to take that approach instead of depending on contributors. Yahoo started about the same time, using human curators to get better relevance. The early crawlers used simple string matching without any attempt to match synonyms or different word forms, so their results weren’t as complete as they might have been. Yahoo’s approach was similar to the Virtual Library.

Ask Jeeves appeared in 1997. It wasn’t a search engine so much as a question-answering service. It claimed to be able to respond to natural-language queries, but results could vary wildly depending on the exact phrasing of the question. Sometimes it produced far better results than regular search engines. In 2006, Ask Jeeves turned into ask.com, which is still around. A search query phrased as a question will sometimes get an answer, but for the most part it responds with search results like its competitors.

The rise and fall of AltaVista, and the rise of Google

AltaVista, launched in 1995, was the most popular search engine of its time. Created by Digital Equipment Corporation, it had the most thorough crawler to date. It covered 20 million pages in 1995, far more than its competitors. In 1996 it began providing search results to Yahoo. Digital, though, was a computer company, not a search engine company, and was unsure how to fit AltaVista into its business plan.

In 1998, Compaq acquired Digital, only making its direction more uncertain. A year later, CMGI bought AltaVista from Compaq. It planned an initial public offering of stock to make AltaVista a public company, but then the “dot-bomb” Internet financial collapse hit. CMGI cancelled the IPO in January 2001.

During 2001, AltaVista’s share of the search engine market rapidly declined. At the same time, a previously obscure search engine called Google rose just as fast. Google had discovered a technique which the earlier search engines didn’t use; it used links as a measure of relevance. The theory was that like links to like. If a site talked about French cooking and other sites about French cooking linked to it, that increased the chances it had something useful to say on the subject, so it got a better rank. Google offered a simpler starting page than its competitors, just a single box to type in a search string.

In 2003, Overture, Inc. acquired AltaVista, and Yahoo acquired Overture soon after. Altavista.com now redirects to Yahoo.

Before long, Google was so dominant that people were giving it free advertising, advising their friends to “Google” for information. Other search engines are still around and have their niches. Bing is for people who like the all-Microsoft experience. DuckDuckGo appeals to people who don’t like Microsoft or Google gathering information about them. Yahoo retains a following. But no one urges people to “duck” or “bing” for information.

Google is the one engine to rule them all, the one engine to find them. But it too may fall when it gets too complacent and someone discovers a better approach. The history of computing is full of companies that seemed invincible but then fell into decline.

 

Sign up here to get a free copy of our special report:

How Outsourced IT Can Save You Thousands

Don’t worry, We will never spam you or share your email with anyone. Your information is safe with us. – We’re IT people, security is what we do.