In this article
I write about this case with the unease of someone who knows they're treading a minefield. In December 2020, Timnit Gebru, technical co-lead of Google's AI ethics team, left the company in the middle of a conflict over a paper. She says they fired her; Google said it accepted her resignation. The paper warned of the risks of certain language models that two years later would carry names like GPT-3.5, ChatGPT, Bard or Claude. On the how of her departure there are two versions that have never been reconciled. On the what of the paper, time has already spoken.
Why this article goes carefully
The case was public, legally sensitive and heated in the media, and it's still being discussed five years on. There's no final ruling, and there are statements that contradict each other among Gebru herself, Jeff Dean — then head of Google AI — and Sundar Pichai, Alphabet's chief executive.
Hence it's worth separating three planes before getting to the substance. What's documented by checkable sources — MIT Technology Review, Axios, CNBC, CNN, Google's official statements, the tweets of those involved and the text of the paper — forms the firm ground. Above that there's another plane, of what the parties declared, where Gebru and Google don't agree and no outsider can arbitrate. And lastly there's the interpretable, which I offer as my own reading and not as fact.
Where the dispute is factual and unresolved, you'll read "according to Gebru", "according to Google", "the official version says". The primary sources are at the end, for anyone who wants to check for themselves.
The paper, in brief
It's titled On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? and carries a parrot emoji after the question. Four authors sign it: Emily M. Bender, a linguist at the University of Washington; Timnit Gebru, then at Google; Angelina McMillan-Major, also of the University of Washington; and Margaret Mitchell, co-director of Google's ethics team. Mitchell appears in the published version as "Shmargaret Shmitchell", a pseudonym she adopted after Google demanded she remove her name.
The text rests on four warnings. The environmental one: training these models consumes electricity at industrial scale and emits CO₂ in quantities the authors try to quantify. The data one: when you feed a model text scraped en masse from the internet, the model inherits the worldviews travelling in that text, including those that harm minority groups, and the authors illustrate it with concrete examples. The disinformation one, given how easy it is to produce believable falsehoods at high speed.
And the fourth, which is the one that gives the paper its title. The authors argue that these models don't understand the language they process, that they manipulate statistical correlations between words with no access to meaning. Hence the parrot, which repeats without comprehending. Hence the adjective, because the choice of each word obeys a learned probability distribution.
Four warnings, published at FAccT in March 2021. Today all four are in the headlines.
The documented chronology
What follows is reconstructed with MIT Technology Review, Jeff Dean's internal memo that leaked to the press, Gebru's public statements on Twitter and the coverage from CNN Business and CNBC.
In October 2020, Gebru and her co-authors submit the paper to the FAccT conference and put it through Google's internal review process, the so-called PubApprove, mandatory for the company's publications. In late November, Google asks her to withdraw the paper or remove from it the names of the co-authors tied to the company. Google's version, conveyed afterwards by Dean in his memo, holds that the paper was submitted late to the process and that it ignored too much relevant research. Gebru's is another: the objection wasn't academic but corporate, because the text criticised exactly the models Google was about to build products on.
Gebru replies by email. She offers to take herself off the paper in exchange for Google explaining who reviewed it internally and by what criteria, and committing to a more transparent process. She sets a condition: without that information, she can't keep working there.
On 2 December she announces on Twitter that Google has fired her. Google replies that it received an email accepting her resignation. The difference isn't a nuance of wording: dismissal and resignation drag along different practical and legal consequences, and the two versions still haven't met.
On 9 December, Sundar Pichai sends staff a memo — obtained by Axios and reproduced by CNBC — in which he apologises for how the departure was handled and acknowledges that "a prominent Black female leader, with enormous talent, left Google unhappily". He promises to investigate how it came to that. What he doesn't say is that Google was wrong to push her out. Gebru rejects the text as insufficient and publicly calls it another attempt to make her doubt her own version.
Around those same days, an open letter gathers the signatures of around 2,700 Google employees and more than 4,300 academics and researchers from outside, protesting the departure and demanding clarity on the internal editorial process. In February 2021, Margaret Mitchell — the team's other co-lead, co-author of the paper — is fired. Google alleges multiple violations of its code of conduct and security policies, among them the exfiltration of confidential documents and other employees' private data. Mitchell holds on Twitter that the firing is retaliation for having defended Gebru.
A year later, in December 2021, Gebru announces DAIR, the Distributed AI Research Institute, with 3.7 million dollars of initial funding from the Ford, MacArthur, Open Society and Rockefeller foundations, alongside the Kapor Center. Critical research on AI, outside the corporate orbit.
What the chronology suggests and what it doesn't
From the documented facts a fair amount can be stated. There was a conflict over the publication of a paper criticising models Google was going to build products on. Gebru left under disputed conditions. The ethics team lost both its co-leads in three months, hundreds of employees protested and thousands of academics signed in support, and in time Gebru set up an independent institute.
What can't be settled from outside is the motivation. Whether Google acted over the paper's quality, as Dean held, or out of fear of reputational damage, as Gebru held, is something the documented facts don't decide: they fit both readings. My suspicion, and I declare it as a suspicion, is that the truth mixes the two. There's no court ruling that resolves it nor any recording of the internal meetings that clarifies it.
That's why this article doesn't write the sentence "Google fired Gebru over the paper". It tells an uglier and more exact story. Google tried to halt the publication, Gebru refused, the subsequent rupture was told in two incompatible ways, and the public effect was enormous.
Why the case still matters
I'll start with what stings most, which is that the paper got it right. The environmental cost of training frontier models has been calculated and published. The biases inherited from the training data drag a substantial literature behind them by now. Disinformation generated by language models stopped being a lab hypothesis and became a front-page matter. And the question of whether these systems understand or only correlate is still open, debated today in almost the same terms the text posed. Four warnings, four bull's-eyes.
There's a second reason, less commented on, that has to do with how much research freedom is worth inside a big corporation. When someone on the company payroll warns of a risk that threatens a product launch, what the Gebru episode teaches is twofold: pushing her out can cost the company dearly in reputation, but pushing her out is perfectly within its reach. That imbalance hasn't been corrected in the years since.
And there remains the outcome, which changed the geography of the problem. DAIR, the AI Now Institute, the community around Hugging Face, collectives like EleutherAI: part of the critical research has moved outside the companies' perimeter. The move matters because it drags the questions with it. What's researched inside Google rarely clashes head-on with Google's business model. What's researched at DAIR answers to no shareholder.
The honest balance on Google
I don't want to close handing out medals or condemnations, because the case doesn't allow it. Google held up neural networks through a decade when almost no one was betting on them, published Attention Is All You Need and gave it to the world, sheltered Geoffrey Hinton until his departure in 2023 and produced in its Brain lab half the technology beating today inside any model. Painting it as an enemy of critical research would simply be false.
And at the same time it's a publicly listed company with a fiduciary responsibility to its shareholders, with a product that needs a competitive edge and a legal department watching the reputational risks. When an in-house researcher signs a text questioning the commercial foundations of products about to ship, the friction isn't an accident. It's built into the structure.
That same friction shows up in any big tech company, and that's why whoever asks whether Google behaved worse than the rest has the wrong question. Another one stands, with a worse answer. What institutional mechanism, if any exists, protects critical research when it conflicts with the business that pays for it. The intervening decade suggests none holds up entirely. Gebru didn't wait for one to appear: she went off and founded her own.
Definitions
Large language model: a system trained on enormous quantities of text to predict the next word from context. It's the technology underneath today's conversational assistants.
Stochastic parrot: an image coined in the paper to describe a model that reproduces language sequences without understanding them. "Stochastic" means it follows a probability distribution: the words are chosen by learned probabilities, not by an understood meaning.
PubApprove: the name of Google's internal process that reviewed its employees' academic publications before they went out.
FAccT: the ACM Conference on Fairness, Accountability, and Transparency, one of the reference conferences in algorithmic ethics, where the paper was published in 2021.
DAIR: Distributed AI Research Institute, founded by Gebru in December 2021 to do critical research on AI away from the tech companies.
References
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and "Shmargaret Shmitchell" (Margaret Mitchell), published in the proceedings of FAccT 2021 (ACM, DOI 10.1145/3442188.3445922). The text in dispute.
Karen Hao, "We read the paper that forced Timnit Gebru out of Google. Here's what it says", MIT Technology Review, 4 December 2020. Summarises the paper's content and the initial conflict, and records that Google told Gebru it "accepted her resignation".
Axios, "Google CEO Sundar Pichai pledges to investigate exit of top AI ethicist Timnit Gebru", 9 December 2020, where Pichai's internal memo was published. CNBC reproduced the text of the memo that same day.
CNBC, "Thousands petition Google for answers on Timnit Gebru departure", 4 December 2020, on the open letter signed by around 2,700 employees and more than 4,300 people from outside.
TechCrunch, "Google fires top AI ethics researcher Margaret Mitchell", 19 February 2021, on Mitchell's firing and the code-of-conduct violations alleged by Google.
Press Release: Announcing DAIR, official site of the Distributed AI Research Institute (dair-institute.org), December 2021, with the funding figure of 3.7 million dollars and the list of funders (the Ford, MacArthur, Open Society and Rockefeller foundations, and the Kapor Center).
The Washington Post, "Geoffrey Hinton leaves Google, warns about the dangers of AI", 2 May 2023, on Hinton's departure from the company.
Wikipedia entries on Timnit Gebru, Stochastic Parrot and the Distributed Artificial Intelligence Research Institute, useful as an index of primary sources.
You might also like
- Invisible biases. The declared bias seeps through, the underlying one stays intact
- The dataset as ideology. The sector's most silent political choice
- Cathy O'Neil and the weapons of math destruction
- Geoffrey Hinton and his non-regret about AI
Comments0
No comments yet.
Leave a comment