In this article
When AI stops being a tool and starts having a voice of its own in a decision, what is it? The word "collaborator" sounds nice and hides the problem: a collaborator has a voice, context and responsibility. AI has zero of the three, yet it already signs medical diagnoses, drafts rulings, generates code that ships to production. We've created a new figure with no name and we're using it as if we knew what it was.
The word the sector chose, and why
There's a detail of vocabulary that slips by and is worth looking at slowly. When Microsoft launched GitHub Copilot in 2021, the lexical choice was deliberate. It didn't call it autocomplete. It didn't call it an assistant. It called it a copilot. The word came with its baggage: the copilot is the second in command, shares the cockpit, has an operational voice, can take control in critical moments. It's an active figure in the decision, not a passive helper. Accepting the copilot metaphor was, even then, accepting that the system stops being a tool and enters the category of those who collaborate.
The metaphor spread. Cursor, Devin, Replit Agent, Claude Code and other products followed the trail. The word "agent" came in strong around 2024-2025. Ethan Mollick, in Co-Intelligence (Portfolio, 2024), explicitly defended the collaborator metaphor: he argues that the analogies people understand best are the anthropomorphic ones, and that treating AI as a coworker yields better results than treating it as software. It's a coherent, much-cited position, and worth reading even if you end up disagreeing. The question worth putting to the metaphor isn't whether it's useful. It's who it benefits.
The three conditions the word drags along
It's worth unpacking what "collaborator" means in human use before applying it to a system. A human collaborator has three properties without which the word doesn't work.
Voice
They have a voice, in the strong sense: a stance of their own they can hold when it contradicts the boss, the team or the client. Voice isn't output. It's the capacity to disagree at a cost to oneself.
Situated context
They have situated context: a life that precedes the work, a field of shared but not identical references, a position in the world from which they look. Situated context isn't a context window. It's having been through something the other hasn't been through and that enters silently into every judgment.
Responsibility
They have responsibility: the possibility of being singled out for a decision, of paying consequences for an error, of getting credit for a success. Responsibility isn't log traceability. It's personal exposure to consequences.
Zero of three
An LLM has no voice, because it can't hold a stance against the incentive of its training; RLHF fine-tunes it precisely to align with the user's preference. It has no situated context, because its context is the prompt's token window; nothing before, nothing of its own. It has no responsibility, because it can't be sanctioned, fired, sued, or even reprimanded in the sense that verb applies to a person. Zero of three.
Calling it a collaborator, strictly speaking, is a category error. The word sounds nice and is, at the same time, operationally false.
Linguistic inflation and who it serves
So why has it caught on? The less cynical answer is that the inertia of language pushes: humans anthropomorphize any system that answers them in their own language, and we've known that since ELIZA. The less naive answer is that lexical inflation has a specific commercial usefulness worth naming.
If the product is a tool, product law applies: product liability, warranties, defects, recalls, damage claims. If the product is a professional, professional regulation applies: licensing, accreditation, codes of conduct, professional liability. If the product is a worker, labor law applies: contract, wages, rights, employer obligations. If the product is a person, the law of persons applies.
The figure of the "AI collaborator," which is in the air but in no legal category, escapes all four. It's not a product in the sense of the EU product safety regulation. It's not a professional with accreditation-based liability. It's not a worker with employer obligations. It's not a person with rights and duties. It's an entity with no status, and the makers thrive in that gray zone because it saves them substantial regulatory costs while letting them sell the metaphor of natural integration into the team.
Crawford, in Atlas of AI (Yale UP, 2021), already described this pattern as strategic ontological ambiguity. Bender, Gebru, McMillan-Major and Shmitchell contributed the other half of the diagnosis in On the Dangers of Stochastic Parrots (2021): lexical inflation with human vocabulary ("collaborator," "understands," "remembers") is what sustains that strategic ambiguity in the public space. It's not a new phenomenon. The gig-economy platforms did the same with their workers: neither employees nor classic freelancers, an intermediate figure handy for not paying what employers pay and not assuming what clients assume. AI as a collaborator is the synthetic version of the same move.
The silent transfer of authorship
Cui and others, in The Effects of Generative AI on High-Skilled Work. Evidence from Three Field Experiments with Software Developers (Management Science, 2025; SSRN 4945566), ran three randomized experiments on 4,867 developers at Microsoft, Accenture and another Fortune 100 company. They found a 26% increase in completed tasks among developers with access to GitHub Copilot, with a larger effect among junior developers. The figure is notable and worth holding on to.
What the figure doesn't capture, and what's worth asking in any serious conversation about assisted productivity, is who is the author of the code produced. The junior developer who accepts 80% of Copilot's suggestions and closes the task faster, did they write the code? The literal answer is ambiguous. The operational answer, the one that matters for the chain of responsibility, is more uncomfortable. If the code fails in production, the developer is responsible. If the code works and gets credited as productivity, the developer collects it. If a copyright dispute arises because the model reproduced a fragment of GPL code, responsibility returns to the developer. The model appears in none of the three cases.
The operation is elegant. The model contributes production without assuming risk. The developer assumes risk and shares production. The company that supplies the model extracts rent without assuming either product risk or professional risk. It's a silent redistribution of the chain of authorship that the collaborative discourse facilitates: if we're all collaborators, no one is responsible for the whole. The word "collaborator" dissolves the question "who's signing this?" and the practical answer ends up being "the weakest human in the chain."
The courtroom case
There was an episode in 2025 worth naming because it puts the problem in black and white. The Second Collegiate Court in Civil Matters of the Second Circuit, based in the State of Mexico, with the opinion delivered by Justice Juan Jaime González Varas, resolved Civil Complaint 212/2025 setting a bond with the support of ChatGPT, Grok and Gemini; the criterion was published in the Semanario Judicial de la Federación on August 22, 2025. The ruling didn't hide the use. It declared it, and the higher courts published transparency guidelines off the back of the case, in what's been cited as the first isolated Mexican thesis on judicial use of LLMs. The relevant detail: the models didn't sign the ruling. The justices signed it. The models contributed to the judgment the signature legitimizes.
This reopens the question. If the model materially took part in the deliberation but doesn't sign, what exactly was it? It wasn't a tool, because it had an argumentative voice documented in the file. It wasn't a professional collaborator, because it has no accreditation, no licensing, no responsibility. It wasn't an expert witness, because it neither appeared nor was cross-examined. It was a fourth thing the judicial system hadn't named until then, and that now appears unlabeled and with real effects.
EU Regulation 2024/1689, the AI Act, in its Annex III §8 on the administration of justice, classifies AI systems used by a judicial authority as high-risk. What the text doesn't establish, because there's no clear legal language for it, is the status of the model within the process. It's high-risk, yes. But what is it? The law regulates the effects. It doesn't name the thing.
Co-Intelligence, the optimistic version, and where it falls short
Mollick, in Co-Intelligence (2024), argues that the collaborator metaphor is not only useful but correct in a strong sense. His thesis is that LLMs exhibit enough fluency, enough capacity for initiative and enough domain specialization for the optimal pattern of use to be treating them as unreliable but capable coworkers. The book is readable and offers operational advice that works in many cases.
What the thesis doesn't resolve is the problem of responsibility. Mollick may be right that treating AI as a collaborator maximizes productivity. That doesn't touch the fact that the rhetorical collaborator isn't a legal collaborator. When something goes wrong, there's no one on the machine's side to settle accounts with. When something goes right, there's no one on the machine's side claiming rights. The asymmetry stays intact under the friendly metaphor.
Russell, in Human Compatible (Viking, 2019), framed the problem from another angle: the expected usefulness of a capable but unaccountable assistant depends on how much agency is delegated to it. If the delegation is limited — suggestions reviewed by a competent human — the problem is manageable. If the delegation widens — outputs accepted wholesale by a tired human — the problem shifts from the model to the human who accepted, with no added protection. The form of use the collaborative metaphor encourages is the second, not the first, because talking about a collaborator dissolves the cognitive friction of review.
The figure with no name, what's left to discuss
The current situation is an institutional oddity. There are millions of daily interactions in companies, hospitals, offices, newsrooms, courts, where a human and a model jointly produce an output whose authorship is diffuse, whose responsibility falls unilaterally on the human, and whose public description uses the word "collaborator" or its equivalents. The operation is stable as long as no one pulls the thread. When someone pulls — a lawsuit, an adverse ruling, a high-profile case — the thread reveals that there's no legal figure on the model's side to appeal to.
What should be done? The question exceeds the blog and exceeds what any serious observer can claim to resolve in a paragraph. What can be said is what hasn't been done. The figure hasn't been legally defined. No specific liability regime has been established that distributes the burden among model provider, integrator and user. No provenance transparency has been required on professional outputs. The status of this fourth thing hasn't been publicly discussed, with broad political voice. Ambiguity remains, in 2026, the sector's preferred option. And as long as it is, "AI collaborator" will keep being an accounting layer that reduces responsibility without reducing the decision.
Definitions
Copilot / agent / assistant. Commercial labels used interchangeably to describe AI systems that intervene actively in a professional workflow. Each term suggests a different degree of initiative, but none has a stable legal definition.
Strategic ontological ambiguity. A pattern documented by Crawford and others: deliberately placing a new kind of economic actor outside the existing legal categories (product, professional, worker, person) to avoid the regulatory burden of each.
Annex III of the AI Act. The section of EU Regulation 2024/1689 that lists high-risk AI systems. It explicitly includes the administration of justice and employee evaluation, without defining the status of the model within those processes.
Authorship transfer. The implicit operation by which an output co-produced between human and model is legally attributed to the human, who assumes all associated responsibility, while the model provider keeps the rent of the use.
Isolated thesis (Mexican law). A jurisprudential pronouncement by a collegiate court that sets a criterion without yet generating binding jurisprudence. The one from the Second Collegiate Court in Civil Matters of the Second Circuit (published Aug. 22, 2025) is the first Mexican thesis that explicitly addresses the judicial use of LLMs.
References
Bender, E., Gebru, T., McMillan-Major, A. & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots. FAccT 2021.
Crawford, K. (2021). Atlas of AI. Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press. Analysis of the ontological-ambiguity pattern as a strategy to escape the regulatory burden.
Cui, K. Z., Demirer, M., Jaffe, S., Musolff, L., Peng, S. & Salz, T. (2025). The Effects of Generative AI on High-Skilled Work. Evidence from Three Field Experiments with Software Developers. Management Science, accepted. A 26% increase in completed tasks among 4,867 developers with access to GitHub Copilot.
Mollick, E. (2024). Co-Intelligence. Living and Working with AI. Portfolio. Explicit defense of the collaborator metaphor as the optimal mode of using LLMs.
Regulation (EU) 2024/1689. AI Act. Annex III §8 "Administration of justice and democratic processes" and §4 "employment and performance evaluation." Classifies the relevant systems as high-risk.
Russell, S. (2019). Human Compatible. Artificial Intelligence and the Problem of Control. Viking. A framework on the delegation of agency to assistants and the risks associated with the tacit shift of responsibility.
Second Collegiate Court in Civil Matters of the Second Circuit (Mexico) (2025). Civil Complaint 212/2025, opinion by Juan Jaime González Varas; use of ChatGPT, Grok and Gemini to recalculate a bond. Criterion published in the Semanario Judicial de la Federación (registry 2031010), August 22, 2025. Coverage and analysis in Nexos ("El juego de la Corte"), Hogan Lovells and Santamarina+Steta; the first isolated Mexican thesis on judicial use of LLMs.
You may also like
- AI as a defective tool
- Diffuse responsibility
- Can an AI be responsible?
- Responsibility as a construct
Comments0
No comments yet.
Leave a comment