So, this is my Austin-at-night photo! I see this from my hotel room (Ancil got a view of the parking lot). I’m including the photo because it’s actually quite late at night but I have to finish my notes from today’s sessions because tomorrow will be just as busy!!
This will be much more summary, about the work that is being done at Marist College with learning analytics. Speaker was Joshua Baron.
Marist has a grant to build on the learning analytics (LA) work done at Purdue. They have several goals with their project, which is called Open Academic Analytics Initiative (OAAI). The goals include:
Do they have a good predictive model about student success?
Can the model be ported out for use by other institutions with different educational footprints?
What kinds of interventions are useful? How can we leverage online communities to foster student success?
The idea is similar to Purdue — can we use LA to predict which students are at risk for not succeeding in a course within the first 2-3 weeks, long before it’s too late for them to catch up. What are the student activities in the LMS that can help us predict this; but also what other kinds of data can be gathered and integrated with “LMS clicks” as part of the predictive model (e.g., a student’s GPA from the previous semester, or cumulative GPA, or scads of other kinds of institutional data.)
The intervention discussion was fascinating. For the LA part of their work, they identify at risk students, and then contact them and encourage the student to log into a remedial space. That space is the Online Academic Support Environment (OASE). It takes advantage of open educational resources and other kinds of self-help materials to enable students to better understand and learn the material they are having trouble with. The speaker also indicated that interventions that use support groups are more successful, and so they have as part of the OASE ways for students to connect up with each other — a “lounge” where students can organize a study night, for example.
There was some discussion of technical tools, but the interesting takeaway for me is that they are working to develop an open ecosystem for academic analytics. They are going to release their predictive model under an OS license–I think it was called the Sakai Academic Alert System.
(Tools mentioned: Pentaho, PMML – Predictive Modeling Markup Language from the Data Mining Group as an emerging standard.)
What are the guiding principles for the OASE?
OASE has 3 major design frames or guiding principles:
- Learner/content interactions
- Learner/Facilitator interactions
- Learner/Mentor interactions
(They are releasing their work in developing the OASE as creative commons content, it’s ready to share now).
Self assessment instruments (to help identify areas of weakness).
OER content for remediation (like Khan Academy stuff. But information overload is something to avoid – need to present what’s effective not just a brain dump of all the stuff that could be useful. [Ahh, what does this approach suggest for the utility of library research guides? See Joan Lippincott discussion.)
OER Content for Improving Learning Skills (study habits; time management, etc. Look beyond mastery of content to see all the things that may be causing the student problems in a course – be holistic.)
Baron mentioned Flat World Knowledge – free online college textbooks.
This is the part of the intervention that has a role for an academic learning specialist who would:
—connect learners to people and services
—build a feeling of community and engagement (someone cares). This tactic has been amply demonstrated to help with persistence and success; you can’t just point people to a web site.
—promoting services and special events. For example, a learning specialist might moderates discussions on pertinent topics, e.g. “your first semester at college.”
The way I understood it, this was a peer-to-peer space in the OASE. Peers or a peer mentor might:
—facilitates weekly student perspective discussions
—online student lounge for informal, spur of the moment interactions (wiki) (inviting others to a test study night)
—Blogs for students to reflect on how their earning is going
Polls were also mentioned as a highly useful tool– e,g, how supported do you feel? Asking that thruout the semester. Eventually, a dashboard might show the key data in real time, rather than spreadsheet reports coming out at intervals.
Payoffs of identifying at risk students–
- Getting students to seek help (this correlates with success)
- Increase the engagement of students
- Improve basic study skills that may translate to their other courses
Is it just about the math, the data – or do we care about how the course is designed? Doesn’t the instructional design of a course make a big difference in LA, given that some courses with more activities for students might have far more data points to be folded into a LA framework? Baron said it was a good question, they weren’t really looking at that as part of their research question, and were using huge amounts of data (they are working with 70 plus courses across multiple institutions) to flatten out that kind of issue.
(Kaleidoscope was mentioned as a project looking at instructional design.)
What happens when students realize that they can game the system? Just click on everything as if they are really engaged and then fail the course. Hasn’t really been a problem. Students are not in the dark about the project – they are aware of the goals.
Baron mentioned that they have looked at many problematic issues, including the danger of profiling — knowing some of the data about a student and stereotyping that student as at risk, or vice versa.
A question came up about team or group work, and how LA can be predictive in those instances — again, not part of their research but a good research effort for someone to take up.
I asked how we could stay current with the work, since it appears they are rolling out a lot of stuff — go to a website for the SOCIETY FOR LEARNER ANALYTICS RESEARCH — SIGN UP!! That’s where they report out their findings, relase their stuff.