Evidence Based Medicine

Usability Testing for Effective Web-Based Learning: An Interactive Evidence-Based Medicine (EBM) Web Site

A presentation given at the Online Northwest 2001 conference in Portland, Oregon, January 26, 2001.
Executive Summary
Presentation Slides

Executive Summary

What is Evidence-Based Medicine (EBM)?

Evidence-Based Medicine (EBM) has been defined as the "integration of best research evidence with clinical expertise and patient values" (Sackett 2000, 1). Although most physicians would probably agree that keeping abreast of the latest research findings is a laudable goal, many practicing clinicians base their decisions largely upon what they learned from medical school textbooks or upon the practices of their peers (Sackett 1997, 7-8). The present volume of published medical information is so immense that a medical professional would need to read more than a dozen articles a day every day of the year just to keep up-to-date (Davidoff 1085). The five-part EBM process serves as a guide to help practitioners improve patient care.

History of the Project

In 1998 librarians from the Lamar Soutter Library and a physician from the department of Family Medicine and Community Health at UMass Memorial Health Care began discussions about collaborating to teach EBM to undergraduate medical students. Instruction at that time consisted of a lecture by the physician to second-year medical students in the Physician, Patient and Society II class. Students received a short booklet about EBM and effective MEDLINE search techniques. Their assignment required them to search MEDLINE to locate evidence that answered a clinical question. This traditional instructional format did not encourage interactive learning but rather passive listening. Students often performed the MEDLINE search without utilizing the techniques described in the booklet.

We became interested in the interactive possibilities of web-based instruction, and we began discussions on how to best use the web to teach EBM. It became apparent that we needed funding to develop an instructional EBM web site. The Office of Medical Education at the University of Massachusetts Medical School sponsors an Innovations in Medical Education Grants (IMEG) program. This program awards up to $7500 to provide support to faculty, students and staff for the development and implementation of creative approaches to teaching and learning in the undergraduate medical school program, which then can be permanently incorporated into the curriculum. We submitted a grant application in July 1998 and funds were awarded the following December.

Current Site Description (http://library.umassmed.edu/EBM)

The web site includes several sections that describe the EBM process, link to EBM sites and databases, and present interactive tutorials. The Database Resources page lists several restricted-access sites, including the Cochrane Library and Ovid's Cochrane Database of Systematic Reviews, Best Evidence, and MEDLINE.


Currently the site offers tutorials on how to use MEDLINE to locate best evidence. We decided to develop web-based instruction to accommodate the busy schedules of medical students and residents. Having the tutorials mounted on the web ensures their continual availability. The tutorials require the student to read a case, to formulate a clinical question and then to translate it into a searchable query, to decide on the best study type to address the question, and to successfully complete a literature search. Due to the often hectic pace at which the intended audience functions we took extra care to condense and focus instructions.

Usability Testing

In the spring of 1999 second-year students in the Physician, Patient and Society II class used the first tutorial, which is an exercise in prognosis. The students filled out paper evaluation forms that were quite positive about the tutorial's effectiveness, however, we felt we needed a reliable and more objective method of gaining useful insight on the tutorial's design. Our library director suggested we consider usability testing the tutorial.

Usability testing relies on direct observation of a user's behavior in navigating an interface. This allows the developer to determine where problems exist in the site's design. Usability testing is a relatively easy yet efficient method of obtaining web design feedback. Usually no more than five or six volunteer testers and one or two observers are needed to effectively assess a user interface.

Plans for the future

Currently the EBM site presents two fully functioning tutorials; two more are planned and will be tested during the development phase. In addition to using Ovid MEDLINE for the literature search section, future tutorials will provide guidance in the selection of other resources including the National Library of Medicine's PubMed and the Cochrane Library to answer clinical questions.

An online Journal Club has been developed that will allow UMass family medicine residents to read a journal article and discuss the evidence with their peers in a distributed environment.

Clinical pathways and an online glossary are also planned to enhance the usefulness of the site.


Sackett, David L. et al. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. New York: Churchill Livingstone, 2000.

Sackett, David L. et al. Evidence-Based Medicine: How to Practice and Teach EBM. New York: Churchill Livingstone, 1997.

Davidoff, Frank et al. "Evidence Based Medicine." BMJ, 310 (April 29, 1995): 1085-1086.

URL: http://library.umassmed.edu/EBM/onlinenw
Last Updated: May 21, 2004
Send us comments.

© 1999-2004 The Lamar Soutter Library UMass Medical School
Worcester, MA, USA 01655
Phone: 508-856-2511