Tutorial – Guide to Effective Searching of the Internet

[ Previous | Home | Index | Next ]

Part 1: The Size of the Internet

The Internet is a vast place comprised of millions of computers sending information back and forth in packets. It came into being in the early 70s as a U.S. Defense Department network called ARPAnet. This was an experimental network created for military research, initially, for the design and testing of network survival under wartime conditions.

Before long, academic institutions and private companies performing military research were added followed by nonmilitary related communications between other academic facilities. During this period, the net grew slowly and constantly as the need to communicate and share research and technical information increased. In the early 90s, the pace quickened greatly as the personal computer became more affordable and those with access to the net at work also wanted that access from home and smaller offices.

The Mosaic browser developed in 1993 by Marc Andressen at the National Center for Supercomputing Applications at the University of Illinois in Urbana-Champaign was the basis for the graphical web browser commercialized with fantastic success by Netscape. Via the World Wide Web, the Internet became available to the masses and business saw it as the next frontier for commerce.

Though the Internet is comprised of various sections from newsgroups through email services, by far, the most popular and fastest growing is the World Wide Web, cited as most important by two-thirds of the users, followed by electronic mail.

Web Size and Demographics

Table 1 Internet Users by Region

The U.S. accounts for 47% or more of Internet users, though that percentage is dropping.

• Consensus predictions are for 200 million global users by the year 2000. [4]
• Currently, the Yankee Group reports that 25 percent of US households are online.
• Market research by Ovum predicts that, worldwide, the number of people with access to the Internet will quadruple by the year 2005. The firm also predicts that the U.S. market will reach saturation in 2002 and that the Western European and Asian markets will continue to grow, but at a slower pace. [5]
• The top 25 Internet sites, 11 of which are search-related portals, measured in thousands of unique monthly users as of March 1999 [6]:

Table 2 Top 25 Internet Sites

While the educated and affluent have been among the first to use the Internet since the boom began, mainstream America has been joining in over the last two years. Conversely, the rural poor are the least connected. [7]

• Demographics of the Web have changed dramatically as the user community has evolved from the initial innovators and early adopters; today, demographic changes are stabilizing. In the early years, younger white males dominated the Web, most from educational institutions. Yet with growth, the transition is toward the demographics of the society as a whole (as well as more international). As more people have gotten on the net, especially in the U.S., the demographics of the user population are becoming closer to the population as a whole, as we should expect. There has also been a greater diversification of vocations rather than the computer and education emphasis of the past. [8]

Web User Characteristics [8]

• Over 27% of Web users are trained professionals followed by 1.5% in middle management
• Quality of information is the most important consideration for why users visit a particular Web site
• Two-thirds to three-quarters of all users cite finding information as one of their primary uses of the Internet
• Two-thirds to three-quarters of all users cite the inability to find the information they seek as one of their primary frustrations (second only in frustration to slowness of response)
• Perhaps 31% of those on the Internet are 'heavy' users, online each week for 20 hours or more, with another 34% online for 10-20 hours
• Estimates from BrightPlanet.com are that Internet users (in thousands) break into these on-line categories [9]:

Table 3 On-line Categories

For active users, links from other Web sites are four times (54%) more effective in creating awareness for a given Web site than via a search engine (13%); newsgroups (10%), direct email (6%) and word-of-mouth (6%) are other effective means for gaining Web site awareness

• Sites are visited more frequently based on an active users' bookmarks (82%) than for search engines; in fact, the average active Web user now has about 40 bookmarks of frequently visited sites

Web Documents and Searching

• Alexa estimates that approx. 1.5 million Web pages are being added daily, with an overall doubling rate for Web documents every 8 months [10]
• The Internet now contains 800 million documents [11]; this is a significant increase from the 320 million estimated in 1998 [12, 13]
• International Data Corp. estimates there will be more than 1 billion Web documents by the year 2000 [14]
• Of the major search engines, estimated coverage of the documents on the Internet ranges from a high of 34% to a low of 3% in 1998 [12], with the highest percentage dropping to 16%! in 1999 [11]
• The average user enters 1.5 keywords per search engine search
• Combining multiple search engines in a given search can increase the likelihood of finding the information desired by a factor of 3.5 or more [12]
• Different search formats and conventions make it difficult to search multiple engines at one time
• There are more than 2,500 search services presently on the Web [15, 16]. Below, are the twenty or so major Internet search services:

Table 4 Major Internet Search Services

• There are also 'metasearch' services that provide a central access point to multiple of these services. Notable names - again, not suggesting endorsement - are:

Table 5 Major Internet Metasearch Services

• Not all documents on the Web are indexed. Many are not indexed by any search engine or directory at all. Of those indexed, no engine covers even half of the indexed pages (though Alta Vista purportedly comes close). And besides coverage, there is also the question of keeping the links up-to-date. NEC Research Institute tested the degree that links can be out-of-date and, therefore, useless (dead links). Below are the reported coverage statistics [12] and the percent of dead links [17] reported in 1998.

Table 6 Coverage Statistics and Dead Links

An update of that report in 1999 showed that even the most comprehensive Internet search engines — HotBot, Northern Light and AltaVista — individually catalog at most 16% of the Internet's documents [11], a significant erosion in coverage from the 35% to 45% estimated in the previous year.

• The percentage of Internet users citing one or more of these typical search problems were: searches that turn up broken links (50%); not finding information known to exist (30%); inability to organize collected information (29%); and being unable to find pages already visited (12%) [8]
• More than 98% of active Web users rely on the Internet to find reference material, 30% on a daily basis and a further 40% on a weekly basis [8]
• Use of structured, or 'Boolean' queries, while known to help obtain better search results, can be difficult and frustrating for some users to learn.

[ Previous | Home | Index | Next ]

Copyright © 2006. BrightPlanet Corporation – All rights reserved