Eye-Movement Indicators of Reading Comprehension Outcomes: A Meta-analysis

Recent work on the relationship between eye movements and reading comprehension suggests that while eye movements can predict comprehension outcomes, this relationship is influenced by aspects of the reader, text, and task. To date, eye-movement indicators of reading comprehension outcomes have not been clearly identified. We explored this issue by running a meta-analysis on the relationship between eye-movement and reading comprehension. We ran a search across major databases with keywords limiting the search to eye-tracking studies (e.g., eye-tracking, eye movements) and reading comprehension (reading, learning, text comprehension). The initial search yielded 15022 results after duplicate removal, and a total of 135 articles were included after abstract and full text screening. Correlations between eye-movement measures and comprehension outcomes were extracted directly from published reports, calculated from open-source data, or provided directly by the authors upon request. As many of the studies identified did not directly test or report the relationship between eye movements and comprehension outcomes, data was only available from a limited number of studies. From the original 135 studies, 104 datasets from 37 studies/corpus were available for analysis. However, due to limited data availability across samples, text types, and tasks formats, we focused the analysis on data from adults reading expository texts followed by multiple choice questions (~54% and ~78% of the data for L1 and L2 readers respectively). Random-effects meta-analysis models were conducted separately for eye movement measures for L1 and L2 speakers for individual eye-movement measures, and examined effects of language characteristics (e.g., transparency) on the relationship between eye movements and comprehension. For L1 readers, results suggest that making shorter fixations, particularly during first-pass, and making more fixations is associated with better comprehension outcomes. For L2 readers the pattern was similar with shorter fixations across both early and late measures associated with better comprehension outcomes, although fixation count was not associated with comprehension. These results will be further examined with regards to language characteristics. Limitations due to the lack of data from children and data using varied types of text, comprehension tasks, as well as the possible mediating role of these characteristics will also be discussed.