Search Engines

The Internet without Search Engines is like a large warehouse without a manager, where you find TV's next to vegetables, and where there are no signs or directories to help you find what you are looking for. One could wander indiscriminately and make many useful discoveries. Just as easily, though, hours could pass with nothing of value to show for the effort.*

While it is impossible to sort the Internet itself, e.g. moving all crystallographers in the world to the same place, next to chemists and physicists, it is still possible to put in signs and compile directories.

This idea is challenging enough to give rise to more than just one independent attempt to put it into reality. Actually, it is pretty hard to put together a comprehensive list of Search Engines, and no attempt is made here to do so. Instead, three outstanding services - Yahoo!, AltaVista, and WhoWhere? - are presented.

These three alone are by far sufficient to meet most needs you will ever have when searching for specific information on the Internet. Furthermore, starting with Yahoo! or AltaVista, you will easily discover numerous other ways to get what you want - if present somewhere on the Internet.

• Yahoo! - A Hierarchical Search System
• AltaVista - Non-Fiction Hitchhiker's Guide to the Galaxy
• WhoWhere? - An E-mail Index

Quality of Search Results

The quality of the search results is of course strongly correlated to the way the databases are built. On the one hand, Yahoo! is an edited directory tree constructed with human intervention. This means, the entries are added in a reasonable way and presumably checked for consistency.

However, the Web is a rapidly evolving institution, and it is simply impossible for humans to keep all links up to date. For example, the link titled "Index - Crystallography - WWW Virtual Library" in the Yahoo! subdirectory "Science - Physics - Crystallography" currently points to a page which just tells you that the page has moved. While this is not a catastrophe, you could well have received a message like "The requested URL was not found on this server".

On the other hand, the AltaVista and WhoWhere? databases are automatically built by so-called Web Robots, also known as Web Crawlers, Wanderers, and Spiders. Comprehensive information on Web Robots is available elsewhere on the Web.

The enormous amount of information collected by the Web Robots can not be digested by humans in any way. Instead, the raw data are run through more or less intelligent "data reduction" routines before they are finally stored in the database. Presumably, the stored information is validated at regular time intervals in order to remove outdated references. However, even the best existing Web Robots cannot keep track fast enough to avoid the famous "URL was not found" messages.

When you search and finally find e-mail addresses, you will often experience problems like in one of the previous examples. When searching for "h flack" with WhoWhere?, there were two e-mail addresses, one of which (flack@cui.unige.ch) was more or less accidentally released to the public in Howard's postings to the newsgroup sci.techniques.xtallography (12th Sept. 1995 and 9th May 1996) and should not be used. But how can you know? -- Still, in these cases your best options are to find the personal Home page or to use the telephone!