المفاهيم الأساسية
Developing a human-in-the-loop AI system to detect and deter cheating rings in online exams, ensuring integrity and fairness.
الملخص
Abstract:
Concerns about security in online exams due to professional cheating services.
Introduction of a human-in-the-loop AI system to detect cheating rings.
Emphasis on Responsible AI standards for ethical considerations.
Introduction:
Challenges of maintaining integrity in online assessments.
Development of an AI system to automatically detect potential cheating rings.
Escalation of suspicious tests to human proctors for further scrutiny.
System Overview:
Integration of the cheating ring detection system into existing online examination platforms.
Collection of data including video recordings, machine-specific information, and keystroke/mouse movements.
Focus on detecting professional cheaters controlling test takers' devices.
Keystroke and Mouse Movement Pattern Analysis:
Comparison of historical sessions with current test sessions for similarity detection.
Proctoring interface presentation for suspicious tests and historical sessions with similar patterns.
System Evaluation:
Experiment details using keystroke and mouse movement pattern analysis methods.
Dataset sampling from Duolingo English Test sessions in 2022 and 2023.
Performance comparison across different methods using AUROC, FPR, FNR metrics.
Discussion:
Limitations in the study and opportunities for further research.
Importance of integrating AI signals as evidence, not sole determinants in cheating accusations.
Adherence to responsible AI standards for privacy protection and bias prevention.
الإحصائيات
"Table 1: Performance across different groups for deep-keystroke+mouse method."
"Table 2: Performance for different methods."