Springer Nature retracts widely cited ChatGPT-in-education meta-analysis

Springer Nature has pulled a May 2025 paper in Humanities & Social Sciences Communications that claimed ChatGPT delivered a large positive effect on student learning performance and moderate gains in learning perception and higher-order thinking. The publisher cited discrepancies in the analysis and a loss of confidence in the conclusions. Before retraction, the paper accumulated 504 citations, nearly 500,000 readers, and a 99th-percentile attention score.

The meta-analysis pooled 51 prior studies comparing ChatGPT-using experimental groups to control groups. Critics, including University of Edinburgh researcher Ben Williamson, argue the underlying studies were of poor quality and methodologically incompatible — different populations, samples, and designs being mashed together to produce a headline effect size. Williamson also notes the timeline is implausible: ChatGPT launched in November 2022, leaving too little runway for dozens of rigorous peer-reviewed evaluations to exist by the paper’s submission.

The broader problem is citation contamination. The paper has already been referenced 262 times within Springer Nature’s own journals and circulated widely on social media as supposed gold-standard evidence for generative AI in classrooms. Retraction does not unwind that downstream literature, meaning a flawed effect estimate will continue to prop up edtech claims and policy arguments long after the source has been withdrawn.