Evaluation to support learning design: Lessons learned in a teacher training MOOC

Jan Nedermeijer

From: Australasian Journal of Educational Technology, 2018, 34(2).
Muriel Garreta-Domingo, Davinia Hernández-Leo Universitat Pompeu Fabra, Barcelona, Spain
Peter B. Sloep Open University of The Netherlands, Heerlen, The Netherlands

University teachers as designers of learning opportunities
Teachers in Higher Education have the task to (re) design their courses.  More and more this task is considered to be a design task. This means the university teachers has to learn how you can design a course in higher education.  The authors have studies ‘how in-service educators perceived and accomplished an (heuristic) evaluation design task as part of a design process to conceptualise a learning activity using information and communication technologies (ICT).’
In the introduction the authors give an interesting description of the teacher as a designer of learning opportunities.

The aims of the study
The authors ‘want to (1) extend and enhance the existing pool of empirical research on how to build on teacher expertise to support them in their design efforts; (2) examine the use of human-centred design methods to empower the designers’ capacity of educators; and (3) inform both practice and research in the fields of learning design.’

Conclusions of the use of heuristic evaluation in the process of learning design
The main conclusions of the authors are: ‘Drawing on the general knowledge that is available about heuristic evaluation, we still advocate HE as part of a design process that covers the entire teaching-learning lifecycle (Goodyear, 2015). But keeping in mind the results we presented here, we suggest that within this lifecycle HE may be of good use if the following practices are followed:

  • Start with educational heuristics only. In the process of designing an ICT-based learning activity, provide educators with heuristics that focus solely on the learning design aspects. This knowledge of powerful design heuristics can also increase the efficiency and effectiveness of educators’ design work (see also McKenney et al., 2015).
  • Include an initial heuristic evaluation task. In our study, educators were asked to define their own heuristics. To reduce the apparent difficulty of this task, educators should first use existing educational heuristics to assess learning activities (scaffolding).
  • Promote a question approach for the de novo creation of heuristics. Similar to the human-computer interaction method of “cognitive walkthrough” (Nielsen & Mack, 1994), educators seemed more comfortable reflecting through questions than through statements. This is aligned with the idea that the formulation of a question is key to the teacher inquiry process (see also Hansen & Wasson, 2016).
  • Bring in user interface/usability heuristics at a later stage only. Once educators have worked on the goals and heuristics of their learning activity, they are more ready to move to the ICT part. Usability heuristics can then become a tool to assess existing technology.

Our study extends and enhances the existing pool of empirical research on how to build on teacher expertise to support teachers in their design efforts; it examines the use of human-centred design methods to empower the design capacity of educators and informs both practice and research in the fields of learning design. In other words, it fits with the idea that “the future progress in learning design R&D, [which] will require more and better research on users, their needs, contexts of use and the affordances of the various tools and resources that are meant to improve their design activity” (Kali et al., 2011, p. 130). If anything, our study supports a call for the creation of more (and better) links between human-centred design and learning design.’