18 
Searching the Internet 
Andrew Booth
Introduction
The 1990s were characterized by the appearance of thousands of networked information resources available to individuals and institutions with access to the Internet. Kassirer (1995) describes this phenomenon as the ‘next transformation in the delivery of health care’. Initially improved access to healthcare information was heralded with great enthusiasm, but recent years have seen the injection of a more appropriate note of caution (Obst, 1998) – caveat surfer! (Silberg, Lundberg and Musacchio, 1997; Booth, 1998). Attempts now concentrate not simply on making information available via the world wide web but also in providing the tools and skills to evaluate and appraise materials (Jenkins, Grey-Lloyd and Hancock, 1998). 
This chapter discusses approaches to systematic searching of the Internet. The concepts of free-text and index based Internet search engines are introduced and major examples of each discussed. Guidance will be provided on methods of effective searching of the Internet sources. The chapter will conclude with a discussion of how websites might be evaluated and the potential role for the health information professional in exploiting the Internet.
The role of the Internet
Simmons (1997) succinctly summarizes the limitations of the Internet thus:
•	its sheer size
•	the difficulty of finding exactly what you need on an ever-growing number of web pages
•	the sheer amount of information, mesmerizing new users who are tempted to spend a disproportionate time looking
•	the lack of quality information
•	difficulties in identifying authors’ identities.
Notwithstanding this cautionary note, there are three very good reasons to include the Internet in the knowledge base of healthcare. First, the Internet as a delivery mechanism provides access to a range of full-text and summary documents otherwise difficult or impractical to acquire. An example is the area of health technology assessment where evaluations of new or high-cost technologies conducted in Australia, North America or the countries of Europe and previously published as grey literature, are now available to healthcare organizations world-wide. Secondly, the Internet as a source of intelligence acts either as an ‘early warning system’ or, alternatively, as a source of valuable supplementary data. Again, new drugs or biotechnology products can now be identified far in advance of their appearance in the international marketplace. Finally, and most importantly, the Internet as a source of value-added information assists in identification, critical appraisal and synthesis of disparate, and even conflicting, research evidence. 
Getting started
With an estimated 100,000 Internet sites dealing with health (Eysenbach, Ryoung Sa and Diepgen, 1999) it is clear that a systematic approach will need to be adopted. A number of authors attempt to systematize the process. Potter (1995) outlines a five-step approach to answering reference questions over the Internet: gather information and tools, learn the terminology, assemble a manual, write a strategy, and make bookmarks. Such an approach is rooted in mainstream reference work. MacCall (1997) describes a ‘metastrategy for World-Wide Web information retrieval’, analogous to the information protocol of Chapter 16, which makes distinctions between the context of the information need (anticipated versus unanticipated clinical information needs) and also between the available tools (WWW catalogs for retrieving websites and WWW databases for retrieving web pages). Littleton (1998) reviews several strategies for finding health information via the Internet. However, there are very few authors who actually try to describe both content and the sequence of a process to find information. Put very simply, an information-seeking process for locating information might use one or all of the following stages:
•	use of medicine-specific search engines, eg OMNI, MedicalWorld Search
•	use of general search engines or meta-search engines, eg Alta Vista, Hotbot, Lycos or Google, Inference Find, Copernic 2000
•	use of databases or library catalogues, eg PubMed, Medical SUM search, Uncover, BMA Catalogue, BL OPAC
•	use of resource lists, eg Netting the Evidence, Cambridge Public Health, Nursing and Healthcare
•	use of discussion lists and archives, eg Mailbase, Liszt.
There are a number of reasons for using resources in the order listed above. First the search would start with specific, and usually evaluated, sources within the field of medicine. These would offer the prospect of a full-text ‘answer’ to the query in hand. General search engines too may supply a full-text solution but their yield, while being more sensitive (ie less likely to miss something relevant) is usually unevaluated. Meta-search engines broaden the coverage, but may lack the sophisticated search facilities of the individual search engine. Databases and library catalogues, particularly those of a subject-specific collection, may help in the identification of useful materials but will not necessarily help with regard to access. Resource lists, again those of a subject-specific nature, may potentially cover a topic of interest, but will require a large number of hypertext links to be followed. Finally discussion lists and their archives may yield highly relevant materials but their validity is subject to doubt. Notwithstanding this, their interactive function can be incredibly powerful. 
Search engines
Medicine-specific search engines
In this chapter a distinction is made between resources offering direct access to clinical information (eg web pages), designated ‘search engines’ and usually indexed automatically using free-text terms, and those that offer pointers to information (eg books, publications, journal articles etc), designated ‘databases and catalogues’ and typically indexed using a recognized system of subject headings. In practice this distinction is arbitrary, as regards information content (an increasing number of databases offer a hybrid product such as appraised abstracts), resource content (a number of resources search both references and full-text products, eg the Cochrane Library) and indexing (some medical search engines use Medical Subject Headings).
The attraction of Internet resources that provide ready information in health-related topics is self-evident and both public sector and commercial Internet providers have made a foray into this market. OMNI is a UK based gateway to Internet resources in medicine, biomedicine, allied health, health management and related topics. It aims to provide comprehensive coverage of the UK resources in this area and access to the best resources worldwide (Welsh, 1997). The main catalogue, indexed using Medical Subject Headings (MeSH), contains over 4000 records describing quality biomedical Internet based resources. A review of OMNI and other discipline-specific gateways identified a need for a broader subject based approach and in spring 2000, a new gateway BIOME will build on the experiences, skills and content of the OMNI service while expanding coverage to all areas within the health and life sciences. 
Medical World Search (Suarez, Hao and Chang, 1997) is a search engine for medical information on the Internet that uses the National Library of Medicine’s Unified Medical Language System and a carefully selected but large database of medical sites. Other examples include Cliniweb International, which uses the hierarchical structure of MESH allied to the powerful retrieval capability of the Unified Medical Language System, to allow both search and browse functions. 
All the above sites share a number of features: they apply quality and/or relevance criteria, they classify materials according to an established medical terminology, either MeSH or UMLS, and, in contrast to the broad coverage of the web as a whole, their coverage is limited and uneven. This is true too of the otherwise valuable evidence based medicine tools such as Turning Research Into Practice (TRIP) and Medical SUMSearch.
Use of general search engines or meta-search engines
A powerful facility of the Internet is in delivery of full-text reports or other docu ments. UK healthcare managers can sit at their desks and interrogate the Department of Health’s Circulars on the Internet (COIN) database. Having identified a Health Service Guideline, with the aid of free downloaded software, they can then print out its digital image in portable document format (pdf). The document can be stored on their hard disk for retrieval at a later date or, as some health organizations do, the file can be placed on a hospital intranet (Simmons, 1997). However this is a ‘known item search’, deceptive in its ease, and analogous to going to a library to pick up a book known to be on the shelves. A far greater challenge is retrieval of an item answering a specific question or clinical information need. That is analogous to a user identifying books on a broad subject and then scanning through them, or perhaps even checking their index, for the information required. A study looking at advice offered to patients with urinary incontinence found that, although information provided was mostly correct, the most informative site was found only with general search engines, not medical index sites (Sandvik, 1999).
Booth and O’Rourke (1999) identify three information retrieval principles that translate from traditional sources to general search engines: focusing the question (see Chapter 15), constructing the search strategy (see Chapter 17) and filtering the results (Chapter 19). These techniques are specifically appropriate to retrieval of full-text documents although they may also identify other significant items.
Case study
An Alta Vista search
A drug rep has been aggressively marketing his lipid-lowering agent, Atorvastatin, stating that it is better for coronary heart disease than all others. Your postgraduate tutor, a general physician, wishes to assess the evidence so that his junior doctors get an objective picture of treatment options. He prepares a drug seminar on the topic, enlisting your help to find relevant literature.
  Clearly a single term search on Alta Vista for ‘Atorvastatin’ will not be effective, yielding 1,296 hits. Using a focused question (condition and intervention), Boolean logic and truncation the query ‘Atorvastatin AND coronary heart disease*’ yields a more manageable 342 hits. Combining ‘Atorvastatin AND coronary heart disease*’ with a filter (eg controlled trial* or clinical trial*) yields 231 hits. Finally two separate searches combining the focused question with either ‘systematic review*’ (12 hits) or ‘randomi* controlled trial*’ (23 hits) variously yield very useful materials including a drug monograph on Atorvastatin (http://www.rxlist.com/cgi/generic/atorvastatin.htm) and an evidence based article entitled ‘Atorvastatin – Another HMG-CoA Reductase Inhibitor?’ from the free Medscape service. Other restrictions might include limiting to UK mater ials by combining with ‘AND host:uk’.

Comparing search engine performance
It is a very small shift for the health information professional to move from evaluation of traditional bibliographic databases to assessing coverage and quality of Internet search engines. Informal surveys conducted via discussion lists have revealed very little consensus in the general search engines that librarians choose to use. Librarians will have competing priorities that will govern selection; for example, subject coverage may determine Northern Lights, presentation may determine Yahoo, while Alta Vista may be chosen because of its advanced Boolean functions. Search engines use different methods, for example, Google interprets a link from page A to page B as a vote, by page A, for page B and thus assesses a page’s importance by the votes it receives. The specific context in which the engine will be used will also be important. Wu and Li (1999), for example, have conducted an evaluation of seven search engines using representative consumer health topics, assessing them for content relevancy, system features and attributes.
Individual evaluations are not necessarily going to be reproducible across other topics and a librarian will need to keep informed and up to date on the features of all the major search engines. Heidenreich (1997) provides a set of nine questions to assist in comparing search engines.
Meta-search engines
It is increasingly recognized that single search engines are not sufficient for comprehensive coverage of the web. In fact it was calculated that even the most comprehensive search engine (HotBot) covers no more than about 34% of the then estimated 320 million pages of the ‘publicly indexable Web’ (Lawrence and Giles, 1998). The same authors have subsequently revised their estimate to around 800 million pages on 3 million servers (Lawrence and Giles, 1999). It is thus necessary to use meta-search engines to maximize potential retrieval. A number of meta-search engines exist, eg Metacrawler, Inference Find and Dogpile, to search several search engines simultaneously. Products such as Copernic 2000, with its facility to conduct Boolean searches across numerous engines and then store the results locally, are likely to increase the range of client server options. 
Databases and library catalogues
The lead in offering databases free of charge via the web was set by the National Library of Medicine who made MEDLINE, along with other databases such as HealthSTAR and Cancerlit, available via the Internet Grateful Med service with MEDLINE also offered via PubMed (Wilson, 1997; Vine, 1997). A number of sites offer free databases in the UK with the English National Board for Nursing, Midwifery and Health Visiting, the NHS Centre for Reviews and Dissemination, York and the Health Education Board for Scotland being amongst the pioneers for noticeably different types of information (Kiley, 1997b, c).
Increasingly, the development of Z39.50 compatible library catalogues offers improved access via the web. Major institutions that do this include the National Library of Medicine, British Medical Association, the Health Education Authority, the Royal Society of Medicine, the British Library and the Health Service Management Centre in Birmingham. Union catalogues are a particularly attractive proposition for the web. Library networks that offer their union catalogues include those in North Thames, South Thames and Oxford. Consortia of academic libraries doing the same include CARL (in the USA) and the powerful COPAC resource (UK academic libraries). Trawling the Net, a listing of free access databases and catalogues of interest to those working in the NHS, is maintained at http://www.shef.ac.uk/~scharr/ir/trawling.html. 
Resource lists
The resource list is analogous to the arrangement of a physical library; entries are usually classified in a subject order or arranged alphabetically. The least helpful are those where the author compiles them in the order in which they were identified! A resource list can relate to a condition, eg CancerWeb; a broad specialty, eg nursing; a particular group of users, eg the Cambridge Public Health pages; or merely the idiosyncratic interests of its creator. Resource lists usually betray their simple origins by lacking either a search engine or specific inclusion criteria. Specific resources can be found only by typing a text word into the Find window and searching for its occurrences. Extended resource lists need shortcut links to specific sections of the page for speedier navigation. 
Case study
Netting the Evidence: a ScHARR introduction to Evidence Based Practice on the Internet includes items on the methodology of evidence based health care and resources that use these techniques. An alphabetical listing contains over 140 annotated entries; each is linked to the relevant page, with each URL clearly displayed for future reference. The resource list started in 1995 as an in-house resource to answer reference enquiries about EBHC but has since grown to be the resource list used in the Cochrane Library, available in virtually every medical library in the country and in a large number of institutions world-wide. 
A number of methods keep it up to date:

•	A web-monitor site, The Informant, provides weekly updates.
•	A meta-search engine, Copernic2000, is used to search regularly and results are stored for review.
•	The page is run against a URL-checker to identify invalid links.
•	Other EBHC link pages are reviewed regularly. 
About one afternoon a month is spent on revising the resource list. Its shortcomings include not having a search engine nor explicit inclusion criteria for entries. As a recent reviewer wrote:
  It reminds me of a bookstore – it assembles a wide variety of resources in one place, and
  you can visit it to choose a few for your regular use. However it also has a bookstore’s draw-
  backs – you can’t be sure that it contains everything you might need, it takes time to find
  what suits you, and you might have to order some items and pay for them (Lipman, 2000).
Discussion lists and archives
Discussion lists (mailing lists) use an automated server to distribute a single e-mail message to all subscribers to a list. Discussion lists usually have a set purpose or objective and require the list to be used only for these purposes. They have three main functions:
•	as an interactive medium by which other subscribers can be asked for information, advice or opinion (eg Does anyone use the Bookdata system and if so what are its strengths and weaknesses?)
•	as a publishing medium for details of conferences, new publications, reports etc 
•	as an archive of recent debates or postings (eg you could search for the keywords ‘access policies’ to see if anyone has discussed or provided examples of internet access policies, typically over the last two years).
The most relevant discussion lists are: 
•	lis-medical – for medical librarians
•	lis-nursing – for nursing librarians
•	evidence-based-libraries – covering the research base in librarianship 
•	evidence-based-health – covering all issues of evidence based healthcare.
In addition, there are regional network discussion lists and subject-specific lists identifiable via the Liszt service.
File transfer
File transfer, using the comparatively dated ftp (file transfer protocol) facility, is a means of downloading documents, often quite lengthy in nature, from remote sites. In the past it was used as a separate function using purpose-designed software. More recently it can be activated from within a web browser without the user being aware of it. For example, the British Medical Association website has a number of professional statements and reports that can be downloaded in this manner. Most typically a user will only encounter ftp if uploading documents or web pages onto a website.
Evaluating web resources
An interesting phenomenon has been that nearly as many pages have been devoted to the quality of health information in the mainstream medical press as in the information literature. This is undoubtedly because, as Eysenbach and Diepgen (1998a) point out, ‘misinformation could be a matter of life or death’. Studies investigating the quality of health information have looked at a number of Internet contexts including websites (Impicciatore et al, 1997), mailing lists and newsgroups (Culver, Gerr and Frumkin, 1997; Hernandez-Borges, Pareras and Jimenez, 1997) and e-mail communication between patients and doctors. Impicciatore et al (1997) found in a study of the advice given on the web for management of feverish children that, of 41 web pages retrieved, only four web pages adhered closely to recommendations of published guidelines. A case report of acute renal failure as a result of one source of web based information illustrates the dangers (Weisbord, Soule and Kimmel, 1997).
Issues around evaluating websites are not clearcut. For example, which instrument should be used? A systematic review examining 47 rating instruments concluded that ‘many incompletely developed instruments to evaluate health information exist on the internet. It is unclear, however, whether they should exist in the first place, whether they measure what they claim to measure, or whether they lead to more good than harm’ (Jadad and Gagliardi, 1998). Kim et al (1999) reviewed 29 sources of published criteria to identify areas of consensus. Most frequently cited criteria dealt with content, design and aesthetics of site, disclosure of authors, sponsors or developers, currency of information (including frequency of update, freshness, maintenance of site), authority of source, ease of use, and accessibility and availability. They concluded that there was good agreement on key criteria and a need to identify a clear, simple set of consensus criteria that the general public could use. Then there is the issue around ‘who should do the quality assessment?’ Silberg, Lundberg and Musacchio (1997) dismiss the option of quality control at the time of production as neither possible nor desirable. This leaves what Eysenbach and Diepgen (1998a) distinguish as ‘downstream filtering’ (by consumers) and ‘upstream filtering’ (by an intermediary, eg a third-party rating service). Given the number of sites and, indeed, the different contexts in which the information might be used, it seems that downstream filtering is the more promising. Blumberg and Sparks (1999) describe how public health graduate students developed the capacity to evaluate resources over a six-month period with reliability and objectivity of the source evolving as the most used criterion.
The role of the librarian
The Internet has tremendous potential to affect the work of the information professional (Bates, 1997). This includes extending their domain from published materials to ‘intelligence’, including news and ‘grey’ literature resources. Discussion lists and newsgroups enable librarians to expand their resources beyond their collections and selective dissemination of information can also be done via the Internet. Librarians must develop skills in appraising and evaluating Internet sources for their end-users. 
Another important role is in setting up a library bookmarks page documenting sites of local interest and relevance (Carr and Stibravy, 1999). As bookmarks are stored by the Netscape browser as an html file, it is a non-technical process to organize these into folders and then upload the librarian’s bookmarks on to a local intranet. 
Their role in providing training and advice is likely to extend to instructing users in the use of basic network tools such as file transfer, remote log-in, and electronic mail and answering questions concerning network access and information system design (Warling and Stave, 1995). Librarians have always demonstrated an ability to survive and thrive despite increasing disintermediation and changes in information technologies.
A hint of a potential role for the health information provider of the future is provided by the Sheffield-based WISDOM project, a virtual learning community (O’Rourke et al, 1999). Here a qualified information professional has a role in developing a web library, in producing web based and e-mail delivered informatics seminars and in managing and contributing to a discussion list.
It is essential that the health LIS professional maintains good current awareness with regard to Internet developments, new sites and related issues. Two primary sources are the journals Medicine on the Net and He@lth on the Internet while a steady stream of articles can be found in the general medical press. Of course the Internet is itself a valuable source of current awareness for information professionals (McNab and Winship, 1996).
There is little consensus on how best to train librarians in the skills used to exploit the Internet successfully. Isolated examples exist in the literature – the article by Mikita and Drusedum (1993), for example, is now quite dated. It seems likely, therefore, that the main lessons to be learned are from academic sector initiatives such as the NetLinkS project (Levy, 1999). In this way, the Internet can be both a medium for continuing professional development and the subject of it. 
Montague and Tomlin (1999) focus instead on the end-user and describe the syllabus for training primary healthcare professionals as part of the Primary Care Sharing the Evidence (PRISE) project to bring high-quality information to primary care teams. The potential for major library input into networked information development at a regional level is signalled by Project Connect operating out of North Thames (Rees, 1998). 
Conclusion
During the last decade, the Internet has moved from being a specialist channel used by technicians to a mainstream source of information for both health professionals and the public. The healthcare librarian must draw a careful balance between uncritical acceptance of its use, a complete anathema to evidence based practice, and overprescriptive health warnings designed to deter. The way forward is in bringing to bear the same skills and techniques used in a more conventional setting – definition of the question, design of search strategies, and use of filtering mechanisms – together with general principles regarding evaluation and appraisal of resources and the information contained therein. The future of the knowledge base may well be ‘digital’, but traditional skills developed over the past century will be the key to its successful management. 
Key points 
	Although the Internet is sprawling, chaotic and anarchic, it is useful as a delivery medium, a source of intelligence and a source of value-added information.
	To exploit the Internet successfully, a structured approach will be required; this will include selection of resources and focusing of the question to be investigated.
	Medical search engines will usually be the first port of call followed by general search engines; pointers to information such as catalogues will come next before, finally, resource lists and discussion lists.
	Quality of information on the Internet is a major issue that has led to the generation of numerous scales and evaluation tools.
	The librarian will continue to play a major part in exploitation of the world wide web: in identifying and signposting resources, in providing end-user training and in cataloguing and classifying resources of potential use to their customers.
Relevant websites
General
Alta Vista http://altavista.digital.com/ 
	Extensive web and Usenet index (index to electronic bulletin boards with messages linked by threads) with good advanced searching options.
DejaNews http://www.dejanews.com/
	Searches Usenet newsgroup postings.
Excite http://www.excite.com/ 
	Searches web pages and the most recent two weeks of Usenet news.
Google http://www.google.com 
	A search engine that works by ranking pages according to numbers of relevant links.
Liszt http://www.liszt.com/
	Directory of e-mail discussion groups.
Lycos http://www.lycos.com/ 
	Subject guides plus a search tool.
Inference Find http://www.infind.com/
	A powerful meta-search engine that allows time-limited searching.
MetaCrawler http://metacrawler.cs.washington.edu/
	Simultaneous searches of several search engines.
WebCrawler http://webcrawler.com/
	General search tool.
Yahoo http://www.yahoo.com
	Good searchable index, directory classification and people-searching capabilities.
Medical specific
Cliniweb International  http://www.ohsu.edu/cliniweb/
Medical SUMSearch 
	http://SUMSearch.UTHSCSA.edu/cgi-bin/SUMSearch. exe
MedicalWorld Search http://www.mwsearch.poly.edu
OMNI http://www.omni.ac.uk
	A subject-classified search engine using MeSH headings.
Turning Research Into Practice (TRIP) http://www.ceres.uwcm.ac.uk/
Resource lists
Cambridge Public Health http://www.medschl.cam.ac.uk/phealth/phweb.html
CancerWeb http://www.graylab.ac.uk/cancerweb.html 
	A directory listing of sites related to cancer.
Netting the Evidence http://www.shef.ac.uk/~scharr/ir/netting.html
	A comprehensive list of sources for evidence based healthcare.
Nursing and Health Care Resources on the Net http://www.shef.ac.uk/~nhcon/
Trawling the Net http://www.shef.ac.uk/~scharr/ir/trawling.html
	A list of free databases of interest to NHS staff.
Other tools
Copernic2000 http://www.copernic.com
	Downloadable software allowing multiple search engine searching, stored results and Boolean logic.
Informant http://informant.dartmouth.edu/
	A free web-monitoring service allowing specification of keywords or sites.