What Are We Measuring?

Learning Analytics is one of the main themes of this conference.  This session was a panel discussion about learning analytics from different perspectives.

The panel:

  • Randall Bass, Georgetown University (Exec Dir, New Designs in Learning & Scholarship)
  • W. Gardner Campbell, Virginia Tech
  • John P. Campbell, Purdue
  • John Fritz, UMBC

How far can we get with learning analytics?  How much can learning analytics help us predict student success, and what kinds of interventions can we devise as a result of using learning analytics ?  Are learning analytics the way forward to improve education and advance student learning?

Bass put himself forward as an educator interested in the Scholarship of Teaching and Learning (SOTL) , which is what he called “slow learning analytics.”  Kind of the thoughtful, reflective approach to understanding interactions between teacher and learner, and how to make those yield the kind of outcomes you are really looking for.

[Note: I’m going to extrapolate a little bit from the speakers’ remarks in order to make their points more clear, I hope.  So some of the examples are mine.]

Bass was cautionary about the hype or “silver bullet” approach that sometimes surrounds learning analytics (LA).  He described educators as traditionally wanting more information about student learning, and now, thanks to corporate interest and grants, we could say ” Great, people with money are finally coming to the party.”   But are LA the fast food of learner-based instructional design?  Learning is not just about mastery of content – it’s about a lot of “moves” or performances that can come from learning. These “performances” are not necessarily being demonstrated within course-based learning management systems (LMS) – maybe we can better see the evidence of student learning through narrative reflection from an internship, for example.

It was brought up in discussion that learning analytics are being built upon an environment (the traditional LMS) that was designed around administrative needs to begin with – is this a significant problem for LA at the outset?

These LMS based learning analytics may be blocking from view much better “learning analytics” such as the kind of correlations that are emerging from studies such as NSSE (National Study of Student Engagement, or something like that.)  NSSE is a more holistic look at how students are engaging with their education and thinking about their learning than the kind of course-based learning analytics that are being tested now as predictive models.  Why are the learning analytics people not talking about things like NSSE?  Bass made another very good criticism in my opinion —   institutional responses as a result of undrstanding the results of  NSSE will be a lot different than institutional responses that result from learning analytics.  (I have a post coming from a session earlier in the day by a guy from Marist that gives me hope that this criticism can be overcome.)

LA movement lacks equipoise.  Well he said that, and I didn;t catch the explanation, so I’m going to look this up later…!

Bass’ advice was to insist on the broadest possible definition of learning analytics (going beyond course-based clicks in the LMS).  Admit we have only crude markers of learning, and that this is not our ultimate vision of what evidence-based design of learning environments could look like.

If you aren’t familiar with Gardner Campbell,  you can check him out at his web site.       www.gardnercampbell.net

G.Campbell drew a comparison between the LMS and our fascination with LA — the LMS was going to give students more time on task, provide all these additional opportunities for learning.  We were excited!  But in fact, if the tech is not used well (and how many students will tell you how bad their course sites are, that the discussion board is just busy work, etc) it doesn’t accomplish much for student learning.  We gravitate toward easy answers, and LA is a seductive approach to “improving” education.  Let the computer do the work, it’s easy, it’s administrative.

The outcomes we want from student learning are about functioning well in a big, messy, changing world.  Campbell quoted from Clay Shirky about these times we’re living in as enabling the largest increase in expressive capability in the history of the human race.  It’s a small, small thing then, to deem student success as being good at being a student, mastering course content in student-like exams, being good at “school.”

Gardner Campbell played a video clip of Chris Dede – we treat learning like sleeping ( a fairly homogenous activity among humans, don’t need to design wildly different environments for sleeping).  But everything we know about learning suggests that it is more like eating or bonding (highly variable activities fraught with preferences, situational factors, even contradictions).  Learning is situational.

Campbell also showed a quote from a graduating high school senior, Erica Goldson, in an essay called Here I stand, June 25, 2010, in which she remarks how worried she is for her own future even though she is best in her class, because she doesn’t feel her schooling has prepared her for the real world.  “I excelled at every subject for the purpose of excelling.”  Being good at school.

Campbell also drew our attention to free range learners.  (he mentioned Reddit.  Not sure what this is, have to look later.)  People going maverick not only about what they want to learn, but how they want to learn.  Institutionalized education needs to pay attention.

We need to get some copies of this book, he quoted from it and it looks terrific:    John Naughton – From Gutenberg to Zuckerberg.

Another thing to look up — Paradox of the Active User, by John Carroll, Mary Beth Rosson.   1987.

My notes are not as good for the other two speakers, John Campbell and John Fritz.  They allowed as how there is a certain amount of hype around LA (but that, of course, is our fault, it doesn’t mean the tool is completely useless.)

A lot of what they talked about was the ethical problem of knowing something about students or having the capability of knowing something about students, and not sharing it.  Suppose there are problems with LA, suppose it’s not a silver bullet.  Yet, it has had an important success in helping students with issues like persistence and getting assistance with coursework before its too late for them to catch up.  It shouldn’t be approached as all or nothing, if LA can’t solve all problems, that kind of analysis has demonstrated it can solve some problems.

One topic that interested me a lot in the discussion was about the way that LA may frame the way we think about student learning. (This is a common theme in the learning science courses I have taken – that assessment drives the way you teach, the way you think about student learning and student success.)  Are we standardizing into a system that we won’t be able to give up?  It becomes like grades or standardized tests — LA will seem like the hard science, the “real” approach as compared with the soft-headed approaches, and that will begin to dictate to us what new forms of education should look like.

As you know from the earlier post, Ancil and I accosted Gardner Campbell after the session – I told him briefly about RIS’ interest in informal learning and he said we could email him after the conference for his recommended books on that topic!  He’s as generous as he is cool!

This entry was posted in Learning Analytics, Shifting Education Paradigms. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s