Communications of the ACM

Computing ethics

The Temptation of Data-Enabled Surveillance

By Alan Rubel, Kyle M. L. Jones
Communications of the ACM, April 2020, Vol. 63 No. 4, Pages 22-24
10.1145/3382741
Comments

There is increasing concern about "surveillance capitalism," whereby for-profit companies generate value from data, while individuals are unable to resist.⁹ Non-profits using data-enabled surveillance receive less attention. Higher education institutions (HEIs) have embraced data analytics, but the wide latitude that private, profit-oriented enterprises have to collect data is inappropriate. HEIs have a fiduciary relationship to students, not a narrowly transactional one (see Jones et al.⁷). They are responsible for facets of student life beyond education. In addition to classrooms, learning management systems, and libraries, HEIs manage dormitories, gyms, dining halls, health facilities, career advising, police departments, and student employment.

HEIs collect and use student data in all of these domains, ostensibly to understand learner behaviors and contexts, improve learning outcomes, and increase institutional efficiency through "learning analytics" (LA). ID card swipes and Wi-Fi log-ins can track student location, class attendance, use of campus facilities, eating habits, and friend groups. Course management systems capture how students interact with readings, video lectures, and discussion boards. Application materials provide demographic information. This data is used to identify students needing support, predict enrollment demands, and target recruiting efforts.

---

Current learning analytics practices may be inconsistent with higher education institutions' fiduciary responsibilities.

---

These are laudable aims. However, current LA practices may be inconsistent with HEIs' fiduciary responsibilities. HEIs often justify LA as advancing student interests, but some projects advance primarily organizational welfare and institutional interests. Moreover, LA advances a narrow conception of student interests while discounting privacy and autonomy. Students are generally unaware of the information collected, do not provide meaningful consent, and express discomfort and resigned acceptance about HEI data practices, especially for non-academic data (see Jones et al.⁷).

The breadth and depth of student information available, combined with their fiduciary responsibility, create a duty that HEIs exercise substantial restraint and rigorous evaluation in data collection and use. Consider several recent examples.

Back to Top

Three Cases

Movement tracking. Based on student ID card swipes, a university researcher mapped student movements and social networks and built student retention models. It plans to use Wi-Fi router data to form even more detailed understandings and to share this information with advisors.¹ It is unclear whether students are aware their data is collected, have opted in, or provided informed consent. To act in students' interests, this HEI could have provided substantial information before the study started (including its rationale), the ability for students to easily opt-out, and a clear policy about collection and use of movement-tracking data. This university is one of many engaging in intensive student tracking, either by dedicated beacons, Wi-Fi check-ins, or phone apps.⁴

Third parties. In April 2018, researchers from Pearson publishing revealed they had conducted an experiment by incorporating encouraging "growth mindset" messages into a learning software interface and testing (without students' knowledge or consent) whether they affected students' performance.⁶ This demonstrates the value of student information to third parties for non-educational goals, including corporate profit. Similarly, Piazza, maker of a popular question-and-answer app required by many instructors, has sold student data based on students "opting-in" through a pre-checked box on the app sign-up page. Acting in students'—rather than vendors'— interests demands that HEIs develop stronger controls to protect students and avoid learner data becoming part of surveillance capitalism. At the beginning of the relationship with vendors, HEIs should require that edTech companies make opting-in difficult. If the data collection involves interventions or data sales, HEIs should re-evaluate the relationship, and perhaps require the companies compensate students for their data.

Intensive advising. Like many HEIs, Georgia State University (GSU) has struggled to ensure students (in particular those from underrepresented backgrounds) complete their degrees. In 2011, GSU developed a system tracking academic and financial information that alerts advisors about risk factors (for example, an unsatisfactory grade in a key course). GSU's six-year graduation rate rose from 48% in 2011 to 55% in 2018.⁵ Moreover, students of color, Pell-eligible, and first-generation students now graduate at higher rates than the student body overall.³ The GSU case is often described as a LA success. However, GSU simultaneously hired dozens of new advisors and substantially increased student advising. Hence, it is not just an analytics program; it is an advising program informed by data.

Certainly, the improved student outcomes are important, but it is not clear how much is due to LA, exactly what GSU's advising interventions actually are, and whether they provide the best outcomes for each student or just for GSU. Do they steer students away from challenging courses, reducing agency and potential for excellence, or provide tutoring services for at-risk students—increasing their agency and capabilities? It is unclear the increased student surveillance improved social good, or that GSU considered all of the relevant moral trade-offs. It is crucial to ensure we do not learn the wrong lesson and retain the analytics at the expense of the advising. Acting in students' best interests would require a robust, ongoing evaluation.

Back to Top

Doing It Differently

The cases mentioned in this column illustrate a range of issues in LA, and each demonstrates how HEIs can better fulfill their responsibilities to advance student interests. HEIs have responsibilities before developing LA programs, while student data is collected and analyzed (especially by third parties), and after analytics have been incorporated. Here, we make explicit some of the specific responsibilities of HEIs have as information fiduciaries that can guide their actions.

Diverging interests. Fiduciaries have a responsibility to act in the best interests of their clients, although not necessarily to act only to advance client interest. Data analytics may provide insights for HEIs and help them fulfill their responsibilities to educate students and marshal resources effectively. However, it is an open question whether LA will live up to that promise or that primarily student interests motivate LA research. The tracking and third-party data use cases do not advance student interests.

Moreover, universities collect and use data somewhat indiscriminately because it is potentially relevant to their educational and custodial missions. Yet this "relevance condition" is insufficient to justify data collection, analysis, and use.⁸ Any student data is potentially relevant to educational objectives and it is impossible to tell a priori what data will actually be useful. Hence, a collection principle based solely on potential relevance is no limitation at all.

Fostering trust and trustworthiness. Students attend HEIs believing they are trustworthy, will respect students as individuals, and will not implement systems that subordinate student rights and interests for the sake of institutional or third-party goals. Yet systems currently being built and deployed create opportunities for greater privacy intrusions (the tracking case) and for institutional benefit (movement tracking, third-party use). Students have little knowledge of how they are surveilled, typically have no ability to opt out, are uneasy about data collection by HEIs, and cannot control use of their data. It is worth asking whether their trust is misplaced, or whether they must simply acquiesce because of the social and economic value of a college education.⁴

The right benefits. The benefits of LA may be quite limited. The exemplary LA system (GSU) rests substantially on advising resources, not data, with small increases in student success. It is unclear whether success stems from funneling students into easier courses or from collecting and analyzing troves of student data.

HEIs do have obligations to advance the educational interests of at-risk students. However, it does not follow that HEIs should subordinate students' privacy and autonomy interests for the sake of (speculative) retention and achievement rates. Student support (social, advising, tutoring, financial, mental health) should come first, and long before impinging on other interests.

---

Institutional interests and student interests are not identical, and we should not assume they align.

---

Full account of student interests. We should also be wary of narrowly construing student interests. The aforementioned cases focus on student academic achievement, but students have other interests as well, including privacy. They should not have to forego that interest for a marginal (and speculative) return in academic achievement. This should seem familiar to professionals who bristle at overweening surveillance by supervisors.

Full range of higher education's aims. Higher education has a number of aims, including developing communication, critical thinking, understanding and appreciation of diversity, and development of rewarding employment and careers.² These are fostered by helping students develop their autonomy. Yet close monitoring of student movement, social networks, and daily habits is an imposition on student privacy, a key element in developing and exercising individual autonomy. If we value student autonomy, we ought to curtail student surveillance.⁸

Third parties. Students' social networks, their travel around campus, their health, political, and religious activities are inferable from movements. If that information is valuable for HEIs, it will be of interest to others: potential employers, the FBI (pursuant to business records requests under Patriot Act section 215), software vendors, and more. Student privacy protections under FERPA are limited. Moreover, the more student data collected the greater the risk for data breaches. Hence, LA carries risks that we should consider in determining whether it advances student interests overall.

Back to Top

Conclusion

To be clear, we are not opposed to LA tout court. Some student data collection will advance legitimate educational and custodial goals while respecting student privacy and fostering autonomy. However, LA and data-enabled surveillance can begin as tools for social good but slide into morally suspect territory, especially in immersive institutions with fiduciary responsibilities like HEIs. Institutional interests and student interests are not identical, and we should not assume they align. Governments, corporations, and HEIs should avoid data collection and analysis as a matter of convenience and for specious reasons. When HEIs aim to advance student educational interests, they should primarily do so in ways that are consistent with interests in privacy and autonomy. In short, HEIs should conduct LA in ways that justify the substantial trust that students place in them. That will require restraint and evaluation at the beginning, during, and at the end of LA projects.

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2020 ACM, Inc.

---

No entries found