Language Log — Garbage In, Garbage Out
Summary: Victor Mair uses the GIGO principle to frame a Northwestern University study on the industrialization of scientific publishing fraud — paper mills, fabricated data, hijacked journals — and how AI is accelerating it.
Sources: Raw/Language Log » Garbage in garbage out.md
Last updated: 2026-05-07
The GIGO principle
Garbage in, garbage out: flawed inputs produce flawed outputs regardless of how correct the logic is. In formal terms, validity does not imply soundness. The principle has long applied to computing; Mair’s point is that it now applies to the entire scientific literature pipeline, with AI amplifying both the production of garbage and its spread.
The scale of the problem
The Northwestern study (Amaral et al., PNAS 2026) found:
- Fraudulent papers are doubling every 1.5 years
- Legitimate publications double every 15 years — ten times slower
- In 2023 alone, publishers retracted over 10,000 papers; Hindawi retracted 8,000+ after paper mill infiltration, costing Wiley an estimated $40 million
- A survey of medical residents in southwest China found ~47% reported buying, selling, or commissioning papers
How the fraud industry works
Paper mills operate as production lines:
- Products sold: authorship slots, citations, entire manuscripts (fabricated data, plagiarized or stolen images, impossible claims)
- Price: hundreds to thousands of dollars per paper, with guaranteed publication
- Infrastructure: brokers connect mills to compromised journals; defunct journals’ domains are bought and repurposed for high-volume fraudulent publication
The incentive structure runs from individual researchers (publish-or-perish) through institutions (international rankings reward publication counts) — making fraud structurally rewarded, not merely tolerated.
AI as accelerant
Mair’s framing: the dangers of GIGO “have only been magnified with the advent of AI.” AI tools lower the cost of generating plausible-sounding manuscripts, manipulating images, and evading plagiarism detectors. The same tools that help legitimate researchers are weaponized by the fraud industry.
Why it matters beyond academia
Peer-reviewed literature underpins medical treatments, pharmaceutical approvals, public health guidance, and policy. If the literature is systematically corrupted, the downstream effects extend to clinical decisions, regulatory frameworks, and public trust in science.
Connections
The GIGO logic is a special case of algorithmic-fairness’s core insight: flawed inputs (biased training data, fraudulent studies) produce outputs that appear valid but aren’t — and auditing the output alone cannot detect the problem.
The paper mill phenomenon is structurally similar to the fake-expert identity problem in language-log-grammarly-expert: both involve the production of plausible-looking content that has no authentic relationship to the claimed source.
narrative-bias (WYSIATI) explains why the fraud scales: once a study is published in a peer-reviewed journal, readers construct a coherent story around it and stop asking whether the underlying data exists.