The truth behind a plush exterior

Trisha Greenhalgh looks at the down side of high-tech learning on the web of primary health care.

It is digital, it is interactive, it comes with high-tech bells and whistles and it is based on up-to-the-minute pedagogical principles - so it must be good. But when the mists of technological utopianism begin to clear, who will be the first to look hard at computer-assisted learning and question what the emperor is really wearing?

In September 1999, eight postgraduate students began University College London's first fully web-based degree course - an MSc in primary health care.

We promised them, and I hope we delivered, a state-of-the-art virtual campus with all the recommended features of the "rich environment for active learning" so beloved of educationists:

* A problem-based curriculum

* A virtual library with links to a range of high-quality internet resources

* A series of bulletin boards for informal online discussion

* Virtual small-group seminars

* A bank of interactive multi-choice questions for formative self-assessment.

But the "right" computer technology does not itself constitute a high-quality education experience, any more than smart leather bindings guarantee the content of a textbook or plush upholstery and a fancy lectern make a traditional lecture worth listening to. By what criteria, then, should we assess whether our gleaming new course is a genuine success?

When I approached the literature for an answer, I was unimpressed. Publication bias in academic journals and the popular press's hunger for breakthroughs and quick fixes appear to have diverted professionals and public from an informed debate on the potential limitations of new technologies in education.

Most published "research" on computer-assisted learning emanates from computer science departments. Such articles typically report that the new course was an unqualified success, and then launch into pages of technical description.

In-depth independent evaluations of students' experiences on web-based courses are rare and those that exist often have major methodological flaws, such as interviews conducted entirely by the course designer.

A recent qualitative study, by Noriko Hara and Rob Kling from the University of Indiana in the United States, into a new web-based language teaching course (http://www.slis.indiana.edu/CSI/wp99_01.html) is particularly pertinent. The two observed on-line classes and visited students in their homes to try to get a feel for what it was really like to be a member of this virtual class.

Some of the problems they documented are listed in the box below. Had these occurred in a face-to-face course, students would have been able to voice their misunderstandings and perhaps extract better explanations and instructions from their tutor.

This small study should, however, be taken in context. The research concerned a course put together by graduate students, not by experienced practitioners of computer-assisted learning.

The instructor appears to have made elementary errors in preparing self-instructional materials. Students had received no prior training in the use of the technology and helpdesk support was non-existent.

But these "weaknesses" are also the very reasons why we should take this study particularly seriously. The researchers managed to gain access to a hastily as-

sembled course that had been adapted from a previous face-to-face syllabus.

Their project was not part of a mainstream technological study into web-based learning. As such, the findings are relevant to the many ad hoc "experiments" in web-based learning that are springing up haphazardly in the United Kingdom but are not being systematically researched or publicised.

What conclusions can we legitimately draw from this disturbing study? First, those of us who like to think we are at the cutting edge of developments in computerassisted learning should be humbler and more guarded in our claims that our systems can be of general use.

Second, the evaluation of all web-based courses should be formative as well as summative, and, like the study described here, should include in-depth interviews and observation of students attempting to gain remote access and follow online links and instructions.

Finally, we should acknowledge that neither course materials nor teaching skills are directly transferable from the traditional lecture theatre to the virtual campus.

The principles of developing web-based courses are exactly the same as those for any self-instructional or distance-learning materials - and they require skills that most academics do not yet possess. We should recognise, and take steps to guard against, the danger of allowing inadequately trained faculty members to "go virtual".

The students on our web-based MSc in primary care began with a week-long residential summer school in which we provided plenty of opportunities for social interaction and intensive one-to-one training in the use of the course software.

At the beginning of the week, students' main anxieties centred on the technology, but by the end of the week all but one felt this was now a minor issue.

Our preliminary experiences with online tutoring and virtual seminars have been positive, and our students have already taken the lead in suggesting, and systematically evaluating, improvements to the system we designed.

Maybe they will rue the day when they signed up for a course in which they may never again see one another's faces. But if our independent assessors discover big flaws in the experience from the students' point of view, I hope we will not only take those findings to heart, but will also have the integrity to publish them.

Trisha Greenhalgh is director of the world's first fully web-based masters course in primary care at University College London (webmsc@ucl.ac.uk).

PROBLEMS ON THE WEB FRONT

Technology failure

* Multiple dead links

* Inaccessible materials

* Interactive assignments impossible to complete

Inadequate training in technology for students

* General information technology skills, for example, typing and saving documents

* Use of course software

* General internet searching

* Poor pedagogical planning

* Confusing instructions

* Unclear learning objectives

* Too many and too complex activities

Poor delivery of online tutoring

* Lack of individual feedback on performance

* Slow response to students' requests for help

* Perceived requirement to send multiple email contributions for assessment purposes

Inherent difficulties with the medium

* No visual cues from tutor or fellow students

* Distractions of low-quality material found on the internet

* Students "bombarded" with messages in real-time seminars.