Questioning the Ethics of Learning Analytics


It's three weeks into the semester, and you're not sure how you're doing in Sociology 101. Course Signals, a predictive analytics software solution, will let you know whether you're in great shape, could use some improvement, or need to consider dropping out.

Created and implemented at Purdue University in 2007, Course Signals integrates with the school's Blackboard-based learning management system (LMS) and displays a traffic light icon whenever students log in. Green indicates that the course is going well; yellow and red follow accordingly. Instructors set the levels and follow up as appropriate.

According to the video, Course Signals bases its prediction on the amount of points earned to date, the number of hours spent on-task (which it tracks via Blackboard), and past performance, which includes all other coursework and perhaps high school transcripts, as well.

Back in Giving Analytics the College Try, we talked about the potential for learning analytics to help steer students toward majors where they're more likely to succeed, identify students whose courseloads are too heavy or light, deliver remedial instruction automatically for students falling behind, and even provide a suggestion engine to surface courses that match a student's background and performance.

Course Signals is a good real-world example of learning analytics in action, and early indicators seem to illustrate its success. A recent feature on NPR reports that some 24,000 students have used Course Signals in various schools, including 20% of recent Purdue undergrads. "It has been shown to increase the number of students earning A's and B's and lower the number of D's and F's, and it significantly raises the chances that students will stick with college for an additional year, from 83% to 97%," according to the article.

More questions than answers
Gathering and using big data is fraught with ethical considerations, and like most every technology, capabilities are advancing far more quickly than legal or organizational safeguards. In fields like marketing, data is gathered and exploited by any means necessary; in academia, there seems to be a stronger push for introspection.

Research scientist Matt Pistilli, who helped create Course Signals for Purdue, wrote a lengthy paper about the ethics of gathering and using student data.

Among the questions raised in the NPR report are:

  • Does the school inform students about the data gathering?
  • Who owns the data: the school, the LMS provider, the student, or some combination?
  • Should the data be used to advance the school's interests, or those of the student?
  • How will the predictions influence student or instructor perceptions?

I agree that some of these are thorny issues, but the first one -- informed consent -- doesn't bother me much. Today's young people live online and have no expectation of privacy. They know their teachers are tracking their coursework via the LMS, so why would additional research bother them, especially if it helps them earn a degree more smoothly?

Pistilli concludes his paper with the assertion:

Institutions will find themselves in the awkward position of trying to balance faculty expectations, various federal privacy laws, and the institution's own philosophy of student development. It is therefore critical that institutions understand the dynamic nature of academic success and retention, provide an environment for open dialogue, and develop practices and policies to address these issues.

In other words, figure it out yourselves.

To that end, a group of academics, lawyers, and scientists convened last month to create a framework for appropriate use of data and technology in learning research. I think their conclusions, if you can call them that, reflect the uncertainty and newness of the field. In a nutshell, the group recommends respect, beneficence, justice, openness, and humanity in considering where and how to use learning data. Why not throw in cleanliness and civic awareness while we're listing vague, positive traits?

So members, I now pose the questions to you: Should learning be more organic or more data-driven? Would a tool like Course Signals have helped when you were in school? What about current students? Share your ideas below.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Related posts:

Michael Steinhart, Contributing Editor

Michael Steinhart has been covering IT and business computing for 15 years, tracking the rising popularity of virtualization, unified fabric, high-performance computing, and cloud infrastructures. He is editor of The Enterprise Cloud Site, which won the Least Imaginative Site Name award in 2012, and he managed TheITPro.com, a community of IT professionals taking their first steps into cloud computing. From 2006 to 2012, Steinhart worked as an executive editor at Ziff Davis Enterprise, writing and managing research reports, whitepapers, case studies, magazine features, e-newsletters, blog posts, online videos, and podcasts. He also moderated and presented in dozens of webinars and virtual tradeshows. He got his start in IT journalism at CMP Media back in 1998, then moved to PC Magazine, managing the popular Solutions section and then covering business technology and consumer software. He holds a Bachelor of Arts degree in communications/journalism from Ramapo College of New Jersey.

Biosensors, Robots Replacing Human Guards

In prisons and other security settings, electronic devices are predicting and detecting anomalous behavior with high accuracy and reducing the need for human personnel.

Questioning the Ethics of Learning Analytics

Schools around the country are weighing the benefits and ethical questions of learning analytics.


Re: Many Questions
  • 7/9/2014 10:57:05 PM
NO RATINGS

It goes without saying that this sort of analytics would only work for homogenized intro courses --- Sociology or Psych 101, for instance --- where professors can create standard lesson plans, multiple choice tests, etc. The more advanced courses I took in college -- history and English, for instance --- required a lot of reading, essay and paper writing ... stuff not easily monitored or quantified.

Re: Many Questions
  • 7/9/2014 8:54:02 AM
NO RATINGS

The program would probably have led me to classes in school that would have given me better grades, but I'm not dissatisfied at all with the old ways of doing things. I did have one freshman philosophy class that I utter failed at but was warned not to take in my first year at college. Analytics may have warned me against it as well, but I probably would have taken it anyway!

Re: Many Questions
  • 7/8/2014 8:40:01 PM
NO RATINGS

@ Michael.


While I don't believe any child should be pigeonhold, I believe career planning should start very earlier.  To get children thinking about what they want to explore and what the requirements are.  The reason I believe that is that so many kids don't understand why they are learning what they are. Having some kind of planning and understanding may help with drop out rates.  Unlike the Soviet Union, such a course of action would not be set in stone and a child could change his or her mind.


I feel the "What do I want to be when I grow up? question is too delayed here in the U.S.  We have people graduating from college still asking the question who have not truly explored any careers . The results, $100,000 school loan debts with no where to go.

Re: Many Questions
  • 7/8/2014 4:34:00 PM
NO RATINGS

At the grad level that kind of situation is even more egregious, I'd say. Yikes!

Re: Many Questions
  • 7/8/2014 4:32:08 PM
NO RATINGS

@Ariella, these examples are horrible, especially in that they teach the students that their work doesn't matter. I would hope that in these cases the administration came down hard on the teachers. Perhaps there ought to be analytics software, if there isn't already, that helps administrators determine whether and when their teachers will fail!

Re: Many Questions
  • 7/8/2014 3:51:27 PM
NO RATINGS

@Beth it would irk me, too. I wouldn't quite call myself a Type A, but I regard deadlines as sacred. I always pushed myself to get students their work back. It really bothers me that in my kids' high school, some teachers keep papers for months. One even lost the whole batch, claiming her cleaning lady threw them out! (Talk about a reversal of "my dog ate it.") Another teacher took a kind of leave b/c of illness in the family but failed to enter records of the tests she marked before doing so. The administration asked the students to bring in their tests, but many said they had already thrown them out. After all, it really is not the students' responsibility to get their scores on record. It really astounds me that the teacher did that -- even with the difficulties at home -- as it is completelyh unprofessional. Every teacher has to keep records of the grades she assigns, and she should be able to send that information in.  

Re: Many Questions
  • 7/8/2014 3:05:43 PM
NO RATINGS

@Ariella, I think one of my stumbling blocks is my envisioning of classes for freshmen, which would potentially be in most need of some friendly handholding from a professor. At a large university, you're talking too many students, I think, to make this idea feasible. It'd be more doable for professors with smaller class sizes and/or smaller universities. I'd like to hear more about how Purdue managed this, if in fact professors used Course Signals to facilitate student reachout, since it's got close to 30,000 students. 

Re: Many Questions
  • 7/8/2014 3:04:39 PM
NO RATINGS

@Beth. I hear you. My daughter is in a grad program with the same issues.

Re: Many Questions
  • 7/8/2014 2:58:30 PM
NO RATINGS

And shame on educators who don't know that the best way to reach out and engage today's students is via the digital platforms they most use and that they're failure to grade papers, etc., in a timely fashion is all too readily apparent given today's electronic access to school systems. I can't tell you how many times I heard my daughter griping earlier this summer because one of her professors hadn't posted final exam or semester grades as quickly as she had expected them to be posted. This prof, in fact, posted final grades after the university's set date -- so that really irked my daughter (who you might say falls into the Type A category, at least regarding school). 

Re: Many Questions
  • 7/8/2014 10:18:29 AM
NO RATINGS

My mind is split on this one. Yes, the student should take responsibility. And, yes, the professor should take some ownership to, if only to determine whether what they are preaching is getting through to the kids. Heck, it's a way for the professor to see if they are effective in their own role.

One thing that this system won't help with is something that goes back to when I was in school (ahem) years ago, and that I've seen as my own kids have gone through college. That is a disconnect between the educator and the student.  Often there is little real interaction between professor and student, even to the point of teachers taking weeks to grade and comment on student work. Without that type of data in a timely manner, this red flag system won't work (no data, nothing to work with until it's too late for the student).

Note, as much as I love the idea of online and blended courses, which should make feedback easier, I'm note sure interaction is happening often enough.

Page 1 / 2   >   >>
INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +