Dissertation Abstract

While evaluating user task performance with eye tracking has been examined within the field of human-computer interaction (HCI), previous research has generally utilized eye tracking to understand how users perform a task. This dissertation defines a new paradigm by which eye-tracking research can be used in HCI: to predict whether users will be successful at a task, using a pattern classification model trained on their eye-tracking data.

This dissertation describes the experimental framework and it demonstrates (through laboratory experiments and machine-learning modeling) the feasibility and limits of predicting user outcomes based solely on eye movement patterns. Utilizing an online learning scenario as a proof-of-concept application of this technique, several rounds of eye-tracking data collection studies were conducted in which participants viewed multiple windows of simultaneous information content: a video of an instructor, lecture slides, a transcript of the speech, etc. After viewing the lesson while being recorded with an eye tracker, students responded to a test in which their understanding of the information content was measured. A variety of machine-learning approaches and feature selection techniques were used to explore the relationship between the eye movements of students and their success on the final test, and classification models that outperformed a majority-label baseline were successfully trained.

To test the limits of this approach, additional studies were conducted with modified user-interfaces that relaxed some of the homogeneity of information content and visual presentation that were rigorously maintained in the initial experiment. In addition, a follow-up study was conducted in which the strict temporal segmentation of the experimental session was randomized, to measure the robustness of the modeling approach to such perturbations of the data-collection and analysis methodology.

The results of this study can assist HCI eye-tracking researchers in developing new techniques for evaluating systems, e.g., by predicting users' task performance based on eye-tracking data. Further, this dissertation lays the conceptual and methodological groundwork for the design of intelligent systems for predicting which users may be struggling with a task, based upon an automatic classification of users into groups of high- or low-performers based upon an examination of their eye movements alone.

Publications

Book Chapters

Hernisa Kacorri, Allen Harper, Matt Huenerfauth. 2014. Measuring the Perception of Facial Expressions in American Sign Language Animations with Eye Tracking. Universal Access in Human-Computer Interaction. Lecture Notes in Computer Science, Volume 8516, pp. 549-559. Switzerland: Springer International Publishing.
[Available from Springer]

Conference Proceedings

Hernisa Kacorri, Allen Harper, Matt Huenerfauth. 2013. Comparing Native Signers Perception of American Sign Language Animations and Videos via Eye Tracking. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). ACM, New York, NY, USA, Article 9, 8 pages. DOI=10.1145/2513383.2513441
[Available on ACM Digital Library.]

Theses/Other Publications

Harper, Allen V.R., (2015). "Eye Tracking and Performance Evaluation: Automatic Detection of User Outcomes". Doctoral Dissertation. CUNY Academic Works.
[Available from City University of New York]

Harper, Allen V.R. (2011). Eye Tracking and User Performance Evaluation. Literature Survey, Computer Science, Graduate Center, City University of New York.
[Download]