How should we evaluate MOOCs?

MOOC Poster (V3)Education is being flooded by new technologies, from mobile apps and iBooks to webinars and massive open online courses (MOOCs).

Even Bill and Melinda Gates are getting excited by the possibilities of MOOCs, doing research into whether they can be effective learning tools for students. According to the Quality Matters organization, this may be “…the first effort to test whether MOOCs can meet quality design standards, incorporate proven methods of effective online instruction, and be effective for different learners.”

What I’m curious about is: How do we evaluate whether online instruction is effective for anyone at all, never mind different learners? Our educational evaluation methods have been based on classroom teaching for years, with exams and coursework playing the central role. Graduation rates are also seen as important.

But why are we sticking to those same old criteria when evaluating the success of MOOCs?

The lame completion rates of MOOCs – hovering around 7 percent – are often cited (not least by me) as a reason for judging these online courses to be a failure. After all, if a course can’t keep a student engaged to the end, it must be bad.

But how about we forget completion rates?

  • If a student dips in and out of the course, watching the videos she is most interested in and not bothering with the quizzes – but still learns valuable knowledge – is that a failure?
  • If a student in Argentina uses the videos in a MOOC to help him learn English and only completes the first two or three lessons, again is that a failure?
  • If a student is interested only in engaging with others on the discussion board and doesn’t watch the videos at all… is that a failure?
  • If a professor takes some open source material from a MOOC to use in her classroom, or skims through the course to get ideas to improve her lesson preparation, then surely it’s been useful? Surely all of these purposes are useful?

So can we say that a MOOC is a success despite a 7 percent completion rate?

I don’t have any quick or clever answers about how to evaluate the success of new educational technologies like massive open online courses. But we do need a fresh way of thinking about it.

photo by: mathplourde
Tagged , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *