A free-standing module from Lancaster University's Advanced Learning Technology (ALT) Programme requiring 60 hours of study time. Some of this time will be spent on-line communicating with other course participants and tutors. Most of the time will be spent reading and preparing a course assignment off-line.
The course will run from 1 February to 30 June 1997 for the MECPOL user trials. The last date for starting study on this course is 30 April 1997.
We will need two weeks notice in order to dispatch course material to you.
A good command of English is required. Much of the work on the module requires on-line discussion and the writing of an assignment. The on-line discussion is NOT real-time, and therefore you will have the chance to prepare your contributions before going on-line.
Centre for Studies in Advanced Learning Technology (C SALT), Department of Educational Research, Lancaster University, UK.
INSTITUTION COURSE CODE: ITL06
The broad aims of this module are to examine ways in which learning and learning materials are evaluated and to consider some of the practical issues involved in formulating effective evaluation plans. We will be concentrating on the evaluation of technology based learning, particularly courseware.
The increasing emphasis on the provision of effective training in industry, and effective learning in education, has raised awareness of the need for evaluation. However evaluation studies are often under-resourced. For example the recent report of the evaluation of the TLT Programme in Higher Education, made the following comments.
"Projects varied widely in the resources devoted to evaluation, the way in which evaluation was organised, the comprehensiveness of the evaluation activities that did take place and the evaluation expertise they were able to draw on. Whilst there were a few good evaluations, the majority were limited in scope to fairly basic user feedback as part of pilot testing. Such variability may be attributed to the ‘mixed messages’ given to projects about the importance of evaluation by TLTP management, leading some to make required budget savings through curtailment of their evaluation plans."
Evaluation of the Teaching and Learning Technology Programme. Final Report - June 1996.
Such accounts of poor evaluation may be familiar to course members. A key goal of this module is to address some of these common pitfalls. Through the recommended readings, activities and discussions we hope that you will be able to: form a rich understanding of the possible approaches to evaluation, gain some practical skills in carrying out an evaluation and develop a vision of how evaluation may be planned and integrated within a wider learning technology project.
The readings for this module cover evaluation of learning technology and the wider issue of educational evaluation. Some readings are very practical, whereas others discuss research issues to do with educational evaluation. Hopefully you will find these readings interesting and relevant to your work or chosen assignment. The readings are organised into three sections:
The Lancaster University Advanced Learning Technology (ALT) Programme has been designed to support the continuing professional development of people working with computer-based learning technologies.
This module is open to people who are currently involved in the design, development, use or evaluation of ALT systems. It is open to those involved in training and to staff working in higher, further or adult education who want to learn more about ALT. It is not oriented towards the needs of those who are primarily concerned with the use of computers in schools.
ALT is broadly defined, to include such things as the use of electronic communications and groupware, multimedia systems, electronic books, on-line help and performance support systems, simulations and exploratory learning tools, as well as more mainstream elements of technology-based training.
You will need access to a computer and the Internet using a World-Wide Web browser such as Netscape (version 2.x or above) or MS Internet Explorer (version 2.x or above). You will be accessing Lotus Notes discussion databases (conferences) via your Web browser.
A participant pack will be provided on request. This pack includes readings for this module. Alternatively, you may choose to find the readings yourself from a reading list which we will supply.
Other learning resources are provided on the module's Web pages, ALTLinks V1 and ALT Links V2.
The assessment task is meant to act as the motor for your learning on the module. It provides a focus for your studying and reflecting on the readings. It is the vehicle through which you can
The exact nature of the assessment task can be negotiated with the tutors. The following suggestions may give you some ideas:
(1) Conduct a small-scale summative evaluation of training or educational courseware. If you aren’t sure if the courseware you have in mind is appropriate, the course tutors can advise. If you do not have access to any appropriate courseware, we can help you find some. Ideally, course members will want to evaluate some courseware in its intended context of use with real users. However not all course members have access to this situation, and you will therefore need to create an evaluation to suit your circumstances. Within this suggested assignment it is envisaged that CM’s will use existing tools of evaluation, such as MEDA, SUMI or the TILT method, as the basis for the evaluation methodology.
We suggest that your assignment report has two parts. The first would be an evaluation report, addressed to the appropriate reader. Your evaluation report should contain the following sections:
The other part of the assignment report will be an opportunity for critical reflection on what you have learned and the linking of some of the readings and your experiences together. This sets the evaluation report into context and should be approximately two pages in length.
(2) One possible variant of suggestion (1) is to carry out a formative evaluation of courseware. You need to have access to a courseware development project or be able to fit this into a courseware development you are already carrying out for another purpose.
(3) Another possible variant of suggestion (1) is to evaluate a training course which uses technology other than courseware.
(4) Develop an evaluation tool, such as a heuristic evaluation checklist. The development of this tool would form the basis of your assignment, as a platform for critical reflection, analysis of some of the problems encountered and description of how these problems were tackled. This contrasts with assignment suggestion (1) were you would focus on using existing evaluation tools with only modest adaptation.
(5) Present a critical review of an evaluation tool or methodology. You can use any of the methods and tools presented here, or others from your own experience. Your analysis should try to suggest the reasons for any shortcomings and how to avoid these pitfalls in suggesting refinements of the tool. One possible comparison would be to compare the MEDA and TILT approaches to courseware evaluation.
(6) Discuss the history and appeal of evaluation as a discipline and suggest ways forward in the evaluation of learning technology.
(7) Design your own assessment or tailor one of the above assignments.
Consult the tutors electronically to discuss ideas that you could explore for your own assessment task.
In assessing your work we will be expecting you:
Assignments should be 2500 -3000 words, or an equivalent.
This module has a 6 CATS points credit
rating in the UK.
(A full masters course = 120 CATS points)
You will be assigned to one tutor who will discuss an appropriate assessment task with you and will provide tutorial support for an agreed period. The normal means of communication will be computer-based conferencing, email or fax, with telephone by arrangement. You will be expected to communicate regularly with the tutor.
The period between your official start date on the module and the submission date for your assignment will not normally be less than four weeks or more than twenty weeks.
Tutors are:
Mr Patrick Hynes and Ms Christine Steeples from Lancaster University, UK.
£100 for the participant pack for MECPOL partner students, otherwise £450.00 (pounds sterling). Please allow up to two weeks to receive your pack from the time of ordering. Packs will be dispatched by surface mail within one week. If you wish to receive your pack by courier there would be an additional charge.
MECPOL partner students may wish to acquire the readings for themselves, in which case no fees will be payable.
If you already have access to the Internet via a Web browser there is no additional cost. If you pay telephone charges related to your use of the Internet, then you will have additional call charges.
If you would like to register for this module, or you have any other queries, please contact:
ALT Programme Organiser- Alice Jesmont
C SALT (Centre for Studies in Advanced Learning Technology)Further details regarding the ALT programme may be found on:
http://www.lancs.ac.uk/users/edres/research/altprog.html