In this article
I've spent months reading lawsuits, opinions and terms of service behind a question that looks simple: when an AI system causes harm, who pays? The short, depressing answer is that it's almost never whoever designed the system. Twentieth-century product law was built on two certainties —that the defect can be pointed to, that the chain between manufacturer, distributor and user can be traced backwards—, and AI dissolves both at once. The defect turns stochastic: the same instruction produces different outputs. The cause turns opaque: not even whoever trained the model knows why it answered that. When the chain breaks, responsibility doesn't evaporate. It falls to the floor. And the floor is picked up by whoever wants to, which is almost always nobody.
The classic chain
The old scheme was orderly, almost tidy. The manufacturer answered for the manufacturing defect, the distributor for the marketing one, the user for misuse. If a car failed, the claimant could trace the way back until reaching a specific node: a design flaw, a part out of tolerance, a botched repair, a use outside the conditions. Each link was separable from the next, and on that separation the case law built doctrines that worked reasonably well: strict liability of the manufacturer in the United States, presumptions of defect in Europe.
That whole architecture rested on two conditions nobody bothered to state because they were taken for granted. One, that the defect be identifiable: a part that fails reproducibly, a design with an error a technical expert can describe precisely. Two, that causation be traceable: the defect causes the harm in a sequence that can be followed —defect, failure, harm— without leaps. Design a braking system badly and the two conditions hold on their own. That's why strict liability, the kind that obliges the manufacturer to answer without the claimant having to prove negligence, could be applied for decades without the building creaking too much.
What AI breaks
An AI system isn't a part. It's a trained model, plus the infrastructure that serves it, plus the use it's put to. And it breaks both conditions at once.
Let's start with the defect, which stops being locatable. A language model's hallucination doesn't come from a broken line of code a technician can point to; it comes from the statistical distribution the model learned during training, the product of millions of samples and billions of parameters that not even its own developers can audit one by one. Selbst and Barocas separated that into two problems that the public conversation tends to mix: inscrutability, which is being unable to access the model's internal reasoning, and non-intuitiveness, which is not understanding why the rules are what they are even if you managed to access them. In the systems we use today both opacities overlap, and that leaves the expert with nothing to measure.
The cause, moreover, branches before reaching the harm. A problematic output is a function of the input, of the weights, of the sampling temperature, of the prior instruction chain, of the session context, of the calls to external tools, of the accumulated fine-tuning. Deciding which of all those factors produced the disaster is, very often, technically impossible. And use widens the fan even further: a general model is sold as a conversational assistant and ends up employed in thousands of contexts the manufacturer didn't anticipate. Does the manufacturer answer when someone uses it for medical diagnosis even though it was sold for chatting? The integrator who plugged it into the clinical record? The doctor who signed off? The chain opens into a fan precisely at the point where classic law needed a single thread to pull on.
How responsibility gets dumped on the user today
The dominant answer in 2026 doesn't arrive via law but via contract. It gets dumped on the end user through terms of service that exclude almost any critical use: OpenAI, Anthropic, Google and Microsoft prohibit in their conditions medical, legal, financial or military use without human supervision. Whoever skips the clause and suffers harm carries it. Clean on paper, and that's why it's worth looking at where it gets dirty.
It gets dirty when the use isn't free. The doctor who uses the system because the hospital imposes it didn't choose to use it; nor did the civil servant processing benefits with a mandatory scoring tool. The transfer of responsibility presupposes a decision that often hasn't existed, so it ends up resting on a fiction of choice. It gets dirty too when there's a product chain in between: the maker of the base model licenses it to an integrator who embeds it in a vertical product —a legal chatbot, a diagnostic module, a credit-scoring system— and sells it downstream. Pinning it on the end user ignores that the integrator and the manufacturer took technical and commercial decisions that weigh on the harm.
And it gets dirty, above all, when the model's output is indistinguishable from the real thing to a non-expert. Mata v. Avianca (case 22-cv-1461, Southern District of New York, 2023) is the example already circulating through every law school: some lawyers filed a brief with six judgments invented by ChatGPT, and Judge Castel imposed a sanction of five thousand dollars on finding bad faith. The blame fell on the lawyers, and applying the professional-care doctrine there seems reasonable: nobody forced them to cite case law without checking it. But the case teaches something the sanction doesn't touch. The product was designed so that its output was stylistically identical to genuine case law, and the manufacturer knew that risk. That the lawyer pays in full doesn't prove the manufacturer did the reasonable thing; it proves they were the easiest link to find.
The flip side of the coin comes from Moffatt v. Air Canada (2024 BCCRT 149). The British Columbia civil resolution tribunal held Air Canada liable for the incorrect information its chatbot gave and flatly rejected the argument that the bot was a "separate entity" the company wasn't answerable for. The decision matters because it shuts the door on the integrator hiding behind automation. What it doesn't touch, once again, is the provider of the model operating behind it.
The European attempt
Europe has responded with two pieces best not confused, because they do different things. The Regulation (EU) 2024/1689, the so-called AI Act, came into force in August 2024 and rolls out its obligations in phases through August 2026 for high-risk systems: technical documentation, conformity assessment, registration, transparency. It imposes duties on the manufacturer, but it doesn't resolve civil liability; for that it refers to the general regime. The second piece, the Directive (EU) 2024/2853 on liability for defective products, in force since 9 December 2024 and with transposition pending in the member states until 9 December 2026, does get into the substance: it puts software and AI squarely inside the concept of a product, and provides presumptions of causation and a reversal of the burden of proof for complex cases. It is, as of today, the most citable rule of the period.
Both move in the right direction, and neither closes the three holes that hurt most. Attribution within the chain when there are several players remains unresolved. Harms that don't fit the mould of the classic technical defect —bias, stochastic hallucination— remain without a clear home. And cross-border cases, when the manufacturer is outside the Union, turn any claim into an exercise of patience and money that few individual claimants can afford.
Kaminski and the four models
If you want to understand why the holes persist despite so much regulatory activity, the map Kaminski draws in Regulating the Risks of AI (Boston University Law Review 103, 2023) helps. She identifies four ways of managing risk, each with its bill. Reactive liability determines who pays after the harm —the classic civil model, with its expensive proof and its already-consumed harm—. Ex ante regulation, made of prohibitions, authorisations and requirements, which is the logic of the European Regulation, is expensive to administer and doesn't guarantee the harm won't arrive anyway. Procedural regulation imposes requirements on how the system is developed, in the style of the NIST risk-management framework or the ISO/IEC 42001 standard: flexible, but it leaves flimsy evidence of compliance. And algorithmic accountability —external audit, a right to explanation, a right to contest— remains little more than a project.
Kaminski's argument isn't that one model beats the others. It's that none suffices alone and that the combination of all of them is producing overlaps and gaps at the same time. Responsibility slips through those gaps, in the in-between space no model quite covers.
The human who signs but doesn't decide
There's a point where the theory turns uncomfortably everyday. The doctrine of negligence assumes the human "in the loop" exercises judgement of their own. Selbst, in Negligence and AI's Human Users (Boston University Law Review 100, 2020), shows what happens when that assumption is a lie: when the human merely confirms what the system proposes —because they have no time, because they lack the technical competence to argue with it, because the process penalises them for deviating, because their instinct is to trust the machine—, the doctrine caves in. The human stops contributing judgement and starts functioning as a rubber stamp. Legal responsibility falls on them. The real control was elsewhere.
The civil servant who approves or denies benefits according to a scoring tool is the textbook case. If the system errs, they answer because they signed. If they rebel against the system, they're disciplined internally for deviating from procedure. Between the two threats there's no room left, and the signature ends up transferring formal responsibility without transferring effective control. Into that gap falls the citizen whose benefit was denied, the only one who signed nothing.
The industry knows how to operate in the diffuse zone
What comes now is my reading, and I mark it as such, though I find it hard to fault: the AI industry learned early that legal ambiguity is comfortable terrain to move in. The terms of service offload. The diversity of jurisdictions lets you choose where to litigate. Technical speed outruns the regulatory pace effortlessly. The opacity of the model hampers expert evidence almost by design. And the gulf in resources between a manufacturer with a huge legal department and an individual claimant drags out any lawsuit to the exhaustion of the latter.
As long as responsibility isn't assigned clearly and with real economic consequences, the incentive points more towards obscuring the chain than towards designing with care. The motto "move fast and break things", which Facebook popularised in the late 2000s and early 2010s before retiring it in 2014, keeps operating in AI with even less friction, because here the harm is more diffuse than on a social network and spreads worse among those who didn't cause it.
The discussion that isn't held
AI policy in 2026 talks about existential risk, bias, intellectual property and employment. On concrete civil liability —who pays when the machine harms— there's an ever-growing academic output and, facing it, a stunted political conversation. The reason is the usual one in these matters: the costs are spread among many and the benefits concentrated in a few, so only one of the two parties has both motives and instruments to mobilise, and it isn't the one taking the hit.
As long as that asymmetry holds, the harm will keep falling. The floor that receives it has specific names: the taxpayer, the end user, the worker with no contract to protect them, the citizen with no administrative remedy that works. It accumulates, quietly, everything the classic chain was designed to pick up and no longer picks up.
Definitions
Strict liability. The doctrine by which the manufacturer answers for its product's defect without the victim having to prove negligence. It consolidated in the United States in the sixties and seventies for manufactured products, out of the case Greenman v. Yuba Power Products (1963) and its reception in Section 402A of the Restatement (Second) of Torts (1965).
Stochastic defect. The erratic behaviour of a system that doesn't reproduce consistently; characteristic of AI models with random sampling, where the same input can generate different outputs.
Inscrutability and non-intuitiveness. Selbst and Barocas's distinction between opacity from lack of access to the model's internal reasoning and opacity from human inability to understand it even when accessed.
Doctrine of professional care. The standard of diligence required of a professional —doctor, lawyer, engineer— in the exercise of their function, used to measure whether they acted negligently.
Product chain. The sequence of players running from the original manufacturer to the end user —base-model maker, integrator, distributor—, each potentially responsible for the defect in their stretch.
AI Act (Regulation EU 2024/1689). The European regulatory framework for AI based on risk levels, in force since August 2024 and with gradual application through 2026-2027.
Directive 2024/2853. The European directive on liability for defective products that expressly includes software and AI in the concept of a product.
References
Regulation (EU) 2024/1689 (AI Act). In force since 1 August 2024; obligations for high-risk systems applicable from August 2026. Cited as the European regulatory framework of reference.
Directive (EU) 2024/2853 on liability for defective products. In force since 9 December 2024; transposition and application to products placed on the market from 9 December 2026. Central piece of the European section.
NIST AI Risk Management Framework (NIST AI 100-1, 2023). Cited as an example of procedural regulation.
Selbst, A. D. and Barocas, S., "The Intuitive Appeal of Explainable Machines", Fordham Law Review 87(3), 2018, pp. 1085-1139. Origin of the distinction between inscrutability and non-intuitiveness.
Selbst, A. D., "Negligence and AI's Human Users", Boston University Law Review 100, 2020, pp. 1315-1376. Basis of the section on the human who signs but doesn't decide.
Kaminski, M. E., "Regulating the Risks of AI", Boston University Law Review 103, 2023, pp. 1347 ff. Source of the four regulatory models.
Mata v. Avianca, Inc., 22-cv-1461 (Southern District of New York, 2023). Case of the judgments invented by ChatGPT and the five-thousand-dollar sanction on the lawyer, imposed by Judge Castel.
Moffatt v. Air Canada, 2024 BCCRT 149. British Columbia tribunal ruling that attributes the chatbot's erroneous information to the integrator and rejects the "separate entity" argument.
Pasquale, F., New Laws of Robotics. Defending Human Expertise in the Age of AI, Belknap Press, 2020. Cited in the debate on the assignment of responsibility.
Wachter, S., Mittelstadt, B. and Floridi, L., "Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation", International Data Privacy Law 7(2), 2017, pp. 76-99. A critique of the GDPR's "right to explanation".
Mulligan, D. K. and Bamberger, K. A., "Procurement as Policy. Administrative Process for Machine Learning", Berkeley Technology Law Journal 34(3), 2019. Regulation of public AI via the procurement route.
Hartzog, W., Privacy's Blueprint. The Battle to Control the Design of New Technologies, Harvard University Press, 2018. Cited in the debate on product design.
Crootof, R., "AI and the Actual IHL Accountability Gap" (manuscript available on SSRN, 2022). Analysis of the responsibility gaps in military-use AI systems, transferable to other high-risk contexts.
You may also be interested in
- Function vs tool. No alternative term is settled
- The drone's human operator. What has no witness has no account
- Can an AI be responsible
- The algorithmic judge already wins
Comments0
No comments yet.
Leave a comment