HOME COLUMNS EDITORS' PICKS ARTICLES BLOGS MULTIMEDIA

James E. Willis, III Monday, April 7, 2014 Editors' Pick

James E. Willis III is an educational assessment specialist

SHARE

Ethical Discourse: Guiding the Future ofLearning Analytics

VISIT EDUCAUSE

Search

Learning analytics holdsincreasing potential forstudent agency and autonomy,highlighting a need for ethicaldiscourse at all levels of highereducation institutions. Topicscentral to this dialogue includestudent awareness of analytics,the future of algorithms andlearning analytics, and theredefinition of failure.

Question 1 of 3 or fewer:

What is your overall satisfaction with thenew EDUCAUSE Review site?

VeryDissatisfied  

VerySatisfied

Help us improve this website

James E. Willis III is an educational assessment specialistand Matthew D. Pistilli is a research scientist at PurdueUniversity.

In a previous article published by EDUCAUSE Review Online,"Ethics, Big Data, and Analytics: A Model for Application,"we, along with John P. Campbell, argued for a Potter's Boxframework to address pressing ethical concerns in learninganalytics. As a follow-up to the central concerns raised in thatarticle, we delivered an interactive presentation, "Ethics andAnalytics: The Limits of Knowledge and a Horizon ofOpportunity," at the EDUCAUSE 2013 Annual Conference.That presentation was guided by four major questions:

1. What are the current projects going on in learninganalytics today? What are the potential ethical pitfallsthat surround these developments? Why are theypotentially harmful? Are these things always wrong, orare they contextually wrong?

2. What is the role of "knowing" a predictive analytic —once something is known, what are the ethicalramifications of action or inaction? What are the roles ofstudent autonomy, information confidentiality, andpredictive modeling in terms of ethical development ofnew systems, software, and analytics?

3. How might we affect the development of future analyticssystems by having ethical discussions? What are thepossible inventions and innovations that could comefrom these discussions?

4. What are the frontiers of ethical discussions andtechnology? Are there new frameworks? How cantoday's discussions affect the future?

What emerged were a series of concerns that helped shape thethesis of preparing for the future of learning analytics withethical dialogue, particularly as learning analytics' computingpower becomes more sophisticated and ubiquitous. Here, wediscuss session participants' ideas, which are helping shape thefuture of learning analytics research and innovation. Among the

questions raised in our conference session were: What do the

1

2

3

questions raised in our conference session were: What do theethical models look like? How are these models deployed rapidly— at the speed of technology? How are these models refinedwith time?

We distilled the group discussions into a series of topics,including student awareness (or lack of awareness) of analytics,future algorithmic science, and the future of learning analyticsas defined by business practices, student and faculty access tothe data, and a redefinition of failure.

The arguments put forward here often take the form ofrhetorical questions; the methodological purpose in presentingthe argument in this way is to frame how ethical questioningmight guide future developments.

Student Awareness of AnalyticsToday's college students are often digitally connected and oftenown multiple Internet-ready devices. From identifiable Internethistories to semantic analysis to databases full of demographicinformation, today's students also stand at the juxtaposition ofpowerful regression analyses that can predict success withstartling accuracy and the tutelage of administrators who arepolitically pressured to secure high retention and graduationrates amidst declining budgets. Are students even aware of whatis known about them — whether "known" is defined asdisparate pieces of data or in powerful analytic systems poisedto offer them help? Better yet, should students have inputregarding what data is stored and how it is used? As a means toan end, is it unethical for administrators to do whatever possibleto help ensure student success, even if it means stretching themeaning of privacy?

Broadly speaking, student information is still kept private; at thegranular level, it is kept confidential. The Family EducationalRights and Privacy ACT (FERPA) allows for the use of data on aneed-to-know basis, and then defines the parties who are privyto the information and for what purposes. Perhaps the tensionhere is that students are paying the institution for an education,and institutions in return provide said education. Is it unethical

for an institution not to readily offer support when it can identify

4

5

for an institution not to readily offer support when it can identifystudents who might benefit from various resources?

Disclosures of what data are stored and how they are used hasbecome troublesome in technology in general, not just in highereducation. The nearly incomprehensible legal jargon employedto protect software firms and brokers is difficult to understandat best and potentially litigious at worst. A question commonduring the EDUCAUSE 2013 presentation — and in manydiscussions since — includes the extent to which studentsshould be notified of what is going on with their data in a waythat echoes the legal statements requiring compliance.Furthermore, some institutions remain unprepared to managedata or analytics systems. Perhaps the fear is that administrativebodies will bury their disclosures to avoid explaining to studentsthe fate of their data.

The ethical dimensions of selecting data points, whether inaggregate or specific students, should come under the scrutinyof disclosure, due diligence, and good faith. That is, studentsshould be made aware, in plain language, of how their data isbeing used, how it is being protected, and what the possibleoutcomes might be. It also means that, despite the appealingpossibility of original research in learning analytics and despitebudget-conscious administrators, at all times of developmentthe question of what should be done versus what can be doneought to serve as a prescient warning.

The Future of AlgorithmsOne of the effects of computing so-called big data is theexponential growth and complexity of statistical modeling,including the regression analysis often employed in learninganalytics. Will future algorithms, massaged with sophisticateddata, lead to a far more powerful way of predicting studentsuccess? What approaches unforeseen at this point couldperhaps replace regression models? Because the emergence ofnew data patterns might well create entirely new methods. Thequestion of what this data really tells us will remain. Will thedata affect student motivation in any quantifiable way? Canstatistical models lead to better outcomes with student

motivation? What measures can be developed to assess the

6

7

motivation? What measures can be developed to assess thechanges affected through the use of predictive algorithms?

A discussion perpetually arising at the intersection of studentmotivation and learning analytics is that of the self-fulfillingprophecy. That is, when students receive feedback indicatingthat they need to improve their performance to achieve asatisfactory outcome, they might assume that it is already afruitless labor and give up on the course rather than actconstructively to better their situation. So, instead of being aconstructive tool, feedback becomes a prophet of failure. Willfuture highly sophisticated and validated algorithms onlyintensify the inverse of the intended outcome? Administratorsmight wrongly assume that any and all feedback is beneficial tothe student; if this assumption is, indeed, false, then whodecides what feedback is valid and how often it should bedelivered?

Predictive analytics have a dark side, too. If we know studentstruly stand at the threshold of shortcoming, are we ethicallyresponsible to prevent academic failure? As measurements oflifestyle choices — such as how often students go to the gym,seek out tutoring, or play video games — become morepervasive and quantifiable, the question of failure predictionmight sharpen acutely. At what point are college students to betreated as independent agents who are, as adults, responsiblefor their own successes and failures? More importantly, whogets to decide where that threshold is? The question of in locoparentis lingers large in these questions because, although theyare answerable, who has the authority to provide guidance isunclear. In other words, who determines what constitutes asuccessful outcome in a student's career?

The Future of Learning AnalyticsFrom its inception, learning analytics developers took their cuesfrom business intelligence and analytics systems. The journeyof a struggling student who is able to succeed with feedbackfrom a faculty member, facilitated by an analytics system, couldbe cast in the same light as the individual who similarly journeysfrom potential customer to repeat patron thanks to targeted

marketing and coupons. The question here, though, is how

8

9

marketing and coupons. The question here, though, is howmuch learning analytics should continue to take cues from thebusiness world? Are the systems now divergently different, ordo they share potentially symbiotic developments? Is itappropriate to glean what is possible from business analytics,especially as increasing amounts of data become available forresearch into human motivations? If the purpose is to helpstruggling students, does this justify the possibility of usingalgorithms like those typically aimed at customers?

Perhaps central to questions about using business analytics ishow educators view data. Is there an inherent difference in howa business analyst and an educational analyst view data sets? Itis probably fair to extrapolate a key difference: portability.Whereas business intelligence is often held in the strictestconfidence, educational institutions have great potential forsharing de-identified, aggregate data to further analyticsdevelopment. Even more difficult, yet potentially valuable, is thepossibility of individualized information that might becomeportable and follow students from a young age through theiradvanced education; this might well help us identify individualmetrics, such as motivators for success.

Redefining FailureHigher education's proliferation in the past few decades has ledcommentators to question a college degree's value. Arguably,learning analytics provide, with increasing breadth and depth, ascience to measure and implement student success initiatives.The ethical question, then, becomes one of failure in relation topaternalism: what constitutes failure — a poor quiz grade or alow GPA? Who determines what failure is, a professor or anadministrator? How long should a school allow students to failbefore they intervene? And, if schools intervene once failure hasbeen identified, how much help is offered, and by whom?

There is much to be learned in failure. Few people haveachieved a college degree without some modicum of failure;with a troubling grade or lack of understanding comesadditional effort to compensate for the difference. Studentsuccess has been at the forefront of learning analytics

development, but it may soon emerge that some of the best

10

11

development, but it may soon emerge that some of the bestdata sources come from student failure. Far from hamperingfuture development, the encouragement to learn from failure orencouragement of resilience could provide potent motivators foranalytics systems.

ConclusionThe ethical questions involved in learning analytics developmentand implementation are multifaceted, difficult, and involvenumerous stakeholders in the educational process. Althoughlearning analytics might provide a pathway to efficiently helpingstudents, they also involve critical decisions with far-reachingconsequences. The most pertinent questions focus on the topicsdiscussed here; from these, it is possible to generate questionsthat help us determine what really helps students and how wemight efficiently implement interventions.

AcknowledgmentsWe thank the following conference contributors, whose ideasinformed our work here: Jean-Paul Behrnes, Jeffrey Belliston,Shaun Boyd, Aaron Coburn, Jody Couch, Zach Heath, KatieHimmelrick-Bruce, Amy Irvin, Shiro Kashimura, Sander Latour,Hans Peter L'Orange, Karen Meelker, Jack Neill, Joyce Nijkamp,Phil O'Hara, Paul Schantz, Tyler Schlagel, Marianne Schroeder,Mike Sharkey, Randy Stiles, John Tong, and James Williamson.

Notes1. James E. Willis III, Matthew D. Pistilli, and John P.

Campbell, "Ethics, Big Data, and Analytics: A Modelfor Application," EDUCAUSE Review Online, 6 May 2013.

2. James E. Willis III and Matthew D. Pistilli, "Ethics andAnalytics: The Limits of Knowledge and a Horizon ofOpportunity," presentation, 2013 EDUCAUSE AnnualConference, 17 October 2013.

3. The outcomes from this "flipped" presentation includeda series of group notes that grapple with thesequestions. Although not everything discussed is included

in this article, it is worth expanding on some of the key

in this article, it is worth expanding on some of the keyideas with the express intent of continuing the dialogueon the role of ethics within learning analytics.

4. Eszter Hargittai, "Digital Na(t)ives? Variation inInternet Skills and Uses among Members of the 'NetGeneration,'" Sociological Inquiry, vol. 80, no. 1 (2010),pp. 92–113; and Sue Bennett, Karl Maton, and LisaKervin, "The 'Digital Natives' Debate: A CriticalReview of the Evidence," British Journal of EducationalTechnology, vol. 39, no. 5, 2008.

5. Alfred Essa and Ayad Hanan, "Student Success System:Risk Analytics and Data Visualization UsingEnsembles of Predictive Models," Proceedings of the2nd International Conference on Learning Analytics andKnowledge, ACM, 2012, pp. 158–161.

6. Fred Stutzman, Robert Capra, and Jamila Thompson,"Factors Mediating Disclosure in Social NetworkSites," Computers in Human Behavior, vol. 27, no. 1(2011), pp. 590–598.

7. Vicente-Arturo Romero-Zaldivar, Daniel Burgos, AbelardoPardo, Daniel Burgos, and Carlos Delgado Kloos,"Monitoring Student Progress Using Virtual Appliances: ACase Study," Computers & Education, vol. 58, no. 4(2012), pp. 1058–1067.

8. Beth Dietz-Uhler and Janet Hurn, "Using LearningAnalytics to Predict (and Improve) Student Success: AFaculty Perspective," Journal of Interactive OnlineLearning, vol. 12, no. 1 (2013); and Barton Keller Pursel,"Learning Analytics: Tread Carefully," SchreyerInstitute for Teaching Excellence: Thoughts on Teachingand Learning at Penn State, 10 April 2012.

9. George Siemens and Phil Long, "Penetrating the Fog:Analytics in Learning and Education," EDUCAUSEReview, vol. 46, no. 5 (2011), pp. 30–32.

10. James Rosenbaum, Kennan Cepa, and Janet Rosenbaum,"Beyond the One-Size-Fits-All College Degree,"Contexts, vol. 12, no. 1 (2013), pp. 48–52; and Min Zhan

and Michael Sherraden, "Assets and Liabilities,

and Michael Sherraden, "Assets and Liabilities,Educational Expectations, and Children's CollegeDegree Attainment," Children and Youth ServicesReview, vol. 33, no. 6 (2011), pp. 846–854.

11. Kimberly Arnold, "Signals: Applying AcademicAnalytics," EDUCAUSE Quarterly, vol. 33, no. 1 (2010);and Paul Baepler and Cynthia James Murdoch,"Academic Analytics and Data Mining in HigherEducation," International Journal for the Scholarship ofTeaching and Learning, vol. 2, no. 2 (2010).

0 Comments EDUCAUSE Login1

Share⤤ Sort by Best

Start the discussion…

Be the first to comment.

Subscribe✉ Add Disqus to your sited Privacy�

Recommend

© 2014 James E. Willis III and Matthew D. Pistilli. The text ofthis EDUCAUSE Review online article is licensed under theCreative Commons Attribution-Noncommercial 4.0license.

Keep up with higher education IT trends

Sign up for free EDUCAUSE Review weekly e-mails to hear about new

SUBSCRIBE NOW

ExploreColumns

Editors' Picks

Articles

Blogs

Multimedia

EDUCAUSE Review Archives

About EDUCAUSE Review

ContactContact our editors:editors@educause.edu

For advertising information:advertising@educause.edu

Research & Publications Conferences & Events Career Development Focus Areas & Initiatives

Connect & Contribute About EDUCAUSE

Copyright © 2015 EDUCAUSE | All rights reserved.

Sign up for free EDUCAUSE Review weekly e-mails to hear about newcontent.

Stay Connected Find us on YouTube

Follow us on Twitter

Subscribe to thenewsletter