The human as a defective machine. The only way to work is the one we have

Kahneman and Tversky cataloged dozens of systematic biases starting in 1974: confirmation, anchoring, loss aversion, the gambler's fallacy, availability, framing. Cognitive psychology lists them by the dozen and demonstrates them again and again. Seen from outside, we're machines with reproducible errors. The dizzying question: if we know them and still fail, are we defective or are those "failures" the only way to function?

The catalog, without decoration

There's a whole genre of popular-science books that opens with the line "we think worse than we believe." It's worth skipping the line and looking at the inventory. Amos Tversky and Daniel Kahneman published in 1974, in Judgment under Uncertainty. Heuristics and Biases (Science 185), the paper that opened a five-decade research program. The paper's thesis was direct: human decisions under uncertainty don't follow the normative model of Bayesian calculation. They follow heuristics —fast, cheap rules— that produce predictable errors. The errors aren't random. They're systematic, replicable, and largely immune to the subject's mathematical or statistical training.

The catalog, simplified, contains at least the following well-documented patterns.

Confirmation bias: the tendency to seek, interpret and remember information that confirms prior beliefs, ignoring or reinterpreting what contradicts them. It's probably the most documented bias and, according to meta-analyses, the most resistant to conscious correction.

Anchoring: any number presented before a numerical question substantially influences the answer, even when the number is manifestly arbitrary. Tversky and Kahneman demonstrated it with a rigged wheel of fortune that generated obviously random numbers; the subjects, when later estimating percentages in an unrelated domain, anchored on the generated number.

Loss aversion: a loss weighs, subjectively, around twice as much as an equivalent gain. This breaks the symmetry of expected-utility calculation and produces manifestly suboptimal decisions in repeated games.

Availability, framing and company

Gambler's fallacy: after a sequence of favorable outcomes in an independent event, we humans overestimate the probability of a change. The roulette wheel doesn't remember previous spins. The gambler does, and projects that memory onto the next throw.

Availability: we estimate the probability of an event by how easily we recall examples of it. The memory of examples isn't a representative sample of the real frequency, so the estimate ends up skewed toward the memorable. That's why we overestimate the risk of an attack and underestimate the risk of a household accident.

Framing: the phrasing of the question changes the answer. The same decision presented as saving lives or losing lives activates different preferences, even in professional subjects like doctors consulted about health programs.

There are dozens more. Thaler and Sunstein, in Nudge (2008), applied the catalog to public policy. Ariely, in Predictably Irrational (HarperCollins, 2008), applied it to consumer behavior with striking experiments. The catalog is robust, cuts across cultures with regional nuance, and has survived in essence the replication crisis that shook social psychology in the past decade.

The replication crisis, not glossed over

It's worth mentioning the crisis because doing so lends credibility to what's left. The Open Science Collaboration, in Estimating the reproducibility of psychological science (Science 349, 2015), replicated a hundred psychological studies published in top-tier journals and found that only around 36-39% produced results statistically comparable to the original. The figure was a shock to the field. Some famous effects didn't survive —social priming, ego depletion— and required revision.

A good part of Kahneman and Tversky's catalog did survive. The central biases —confirmation, anchoring, loss aversion, availability— have been replicated many times, in many contexts, with large samples and preregistration. What didn't survive were some flashy third-generation extensions, not the core. It's worth qualifying this in an honest conversation. But it's also worth not using the crisis as an excuse to discard the general picture. The biased human isn't an invention. It's an empirical finding with a solid basis.

Defective, compared to what?

Here comes the dizzying question of the introduction. If the human has systematic errors, the word "defect" supposes an ideal system the human departs from. And the question is: which ideal system?

The comparison implicit in the classic literature was with a perfect Bayesian agent, one that updates beliefs in response to evidence with mathematical precision. That agent is a theoretical construct useful for analysis. It doesn't exist and can't exist in a system with limited cognitive resources. Processing any everyday decision Bayesianly would require time and memory no biological organism and no practical artificial system has. All functional intelligence, biological or synthetic, operates with heuristics, shortcuts, approximations.

Gerd Gigerenzer, in Gut Feelings (Viking, 2007) and in Adaptive Thinking (Oxford UP, 2000), put forward a position opposed to Kahneman's worth bearing in mind. He called it ecological rationality. His thesis: the heuristics classic psychology calls biases are, in reality, optimal or near-optimal solutions to the real problems the organism faces in its environment. What looks like error from the lab is efficiency from life.

A concrete example. The recognition heuristic: if between two options you recognize one and not the other, choose the recognized one. It's stupid in lab tests designed to make it fail. It works reasonably well in many real decisions, where recognition is an indirect signal of relevant information. When Gigerenzer tested it in stock-market prediction, comparing portfolios composed by brand recognition with portfolios managed by complex algorithms, the recognition heuristic often won. What classic psychology calls a cognitive bias is, for Gigerenzer, fast intelligence adapted to the niche.

The technical dispute between Kahneman and Gigerenzer wasn't resolved. It probably isn't resolvable in general, because it depends on the kind of problem and the relative cost of error and of computation. What the discussion makes clear is that "defect" isn't an absolute category. It's a comparative category, and the referent matters.

The clinical evidence that inverts the picture

There's a body of neurological evidence worth holding onto because it challenges the optimistic reading of "if we eliminated biases, we'd think better." Antonio Damasio and his collaborators studied, from the nineties, patients with lesions in the ventromedial prefrontal cortex. Bechara, Damasio and others, in Insensitivity to future consequences following damage to human prefrontal cortex (Cognition 50, 1994), documented the pattern.

These patients passed intelligence tests normally and reasoned abstractly well. When put in the Iowa Gambling Task —a game in which the subject draws cards from decks with different probabilities of reward and punishment, and learns implicitly which are advantageous in the long run— the patients with prefrontal lesions didn't learn. They kept choosing bad decks, over and over, without the accumulated loss changing their behavior. Their galvanic responses to the imminence of loss were flat; they didn't anticipate the punishment emotionally.

What the patient lost wasn't calculated rationality. It was the somatic marker that in healthy humans operates as an anticipatory emotional bias. And losing it, they decided objectively worse. The clinical conclusion was counterintuitive and worth holding onto: emotional biases aren't defects to be corrected. They're part of the apparatus that makes functional decisions in the real world. A human without those biases isn't a purified rational human. They're a human with brain damage whose personal and financial life deteriorates.

Damasio formalized the observation in the somatic-marker hypothesis, set out in Descartes' Error (Putnam, 1994). Human cognition, in this framework, isn't separable from the body that sustains it. The system we know as "thinking" is a coupled system, and the emotional part isn't external noise, it's an integral signal. Removing it leaves a system with less performance, not more.

The paradox of metacognition

There's another finding in the field worth reporting because it breaks the comfortable recipe. Knowing the biases doesn't eliminate them. Explicit instruction on how the anchoring bias works, for example, slightly mitigates the effect but doesn't cancel it. Successive studies have confirmed the pattern. Metacognition —knowing how one thinks— and cognition —thinking— operate at different levels, and the first doesn't control the second the way one controls a car.

Stanovich, in What Intelligence Tests Miss (Yale UP, 2009), introduced a useful distinction. There's general intelligence (g), measured by classic tests, which captures raw cognitive capacity. There's reflective rationality, the capacity to detect and correct biased thinking when needed. The two are separable: there are subjects with high g and low reflective rationality —intelligent and dysrational— and the reverse. The operational consequence is important: being smart doesn't protect against bias. Whoever boasts of having none usually has an extra one, the bias blind spot, the inability to see one's own biases while being able to see them in others.

This paradox, taken to practice, leaves the individual exercise of correction in an uncomfortable position. Knowing you have biases doesn't protect you from having them. The interventions that do work are those acting on the structure of the environment, not on the will of the agent: decision architecture, mandatory checklists, systematic second opinion, group deliberation with diverse voices. Correction by individual will is inversely proportional to the bias one tries to correct.

What the words "defective machine" reveal

There's a point worth reaching before closing the piece. The metaphor "defective machine," applied to the human, assumes three things worth pausing on.

First, it assumes a single performance metric. The machine is defective with respect to some evaluable task. For the human brain there's no single task. There's a broad repertoire —survival, reproduction, body maintenance, social life, learning, emotional regulation, decision, communication— that the apparatus juggles. Optimizing one of the elements degrades the others. Loss aversion may be a "defect" for repeated games in the lab and a critical adaptive trait for not betting in real life what you can't afford to lose.

The ideal system that doesn't exist

Second, it assumes an ideal system to compare with. As already said, the perfect Bayesian doesn't exist and can't exist. The LLM isn't the ideal system either —its failure modes are different, neither better nor worse in aggregate, different. Comparing the human with the LLM gives ambivalent results: the human wins on some dimensions, loses on others, and the sum depends on the chosen weighting. There's no absolute ranking.

Third, and here's the most interesting part, it assumes that the agent doing the evaluating is outside the evaluated system. The human who calls the human defective examines themselves with the same defects they claim to diagnose. The operation is legitimate but drags a logical problem: the metric used to measure is already biased by the system being measured. Cognitive psychology knows the problem; that doesn't mean it resolves it. What it produces is useful knowledge with no guarantee of exteriority.

The question the introduction posed —are we defective or are those "failures" the only way to function?— admits, then, not an answer but a displacement. "Defective" is a category that assumes an ideal system that doesn't exist. The traits psychology catalogs as biases are the properties of a cognitive system that is embodied, limited in resources, in time and in memory, calibrated by evolution and by learning to survive in a concrete world. Calling them defects compares them with a system that exists only as a useful theoretical fiction. The comparison works as an instrument of analysis. It doesn't work as a verdict.

What does remain, after everything, is the honest observation that we think worse than we believe about concrete things, in concrete conditions, and that knowing this is practical information about when not to trust our own judgment. Not out of absolute defect. Out of insufficient relative reliability in domains where the cost of error is high and independent verification is available. The question wasn't whether we're defective machines. The question was when, in which domains, we're sufficiently reliable machines, and when not. That question does have an answer, though it isn't the answer the word "defective" insinuated.

Definitions

Heuristic. A fast decision rule that produces reasonable answers with low cognitive effort. Heuristics can produce systematic errors in specific conditions —those are the cognitive biases— and, simultaneously, produce good decisions in most of the environments they're calibrated for.

Cognitive bias. A systematic deviation of judgment or decision from the normative model. Documented empirically since Tversky and Kahneman (1974). Replicable, cross-cultural, resistant to conscious correction.

Ecological rationality. A concept of Gigerenzer and others. It holds that human heuristics are optimal or near-optimal adaptive solutions to the organism's real problems, and that their apparent "irrationality" disappears when evaluated in the appropriate environment.

Somatic marker. Damasio's hypothesis that functional human decisions incorporate bodily emotional signals as integral information, not as noise to be filtered out. Its absence, through prefrontal lesion, deteriorates real decision.

Bias blind spot. A bias documented by Pronin and others (2002): the tendency to detect cognitive biases in others while being unable to detect them in oneself. Especially prevalent in subjects who consider themselves rational.

References

Ariely, D. (2008). Predictably Irrational. The Hidden Forces That Shape Our Decisions. HarperCollins. An experimental application of the bias catalog to consumer behavior.

Bechara, A., Damasio, A. R., Damasio, H. & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50(1-3), 7–15. Documentation of the Iowa Gambling Task and the functional deterioration associated with the loss of the somatic marker.

Damasio, A. (1994). Descartes' Error. Emotion, Reason, and the Human Brain. Putnam. The formulation of the somatic-marker hypothesis and a critique of cognition/emotion dualism.

Gigerenzer, G. (2007). Gut Feelings. The Intelligence of the Unconscious. Viking. A defense of ecological rationality as an alternative framework to the classic bias catalog.

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. The canonical synthesis of the research program on heuristics and biases.

Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science 349(6251). A systematic replication of a hundred psychological studies with variable results; the central biases of the Tversky and Kahneman catalog survived.

Stanovich, K. E. (2009). What Intelligence Tests Miss. The Psychology of Rational Thought. Yale University Press. The distinction between general intelligence and reflective rationality.

Tversky, A. & Kahneman, D. (1974). Judgment under Uncertainty. Heuristics and Biases. Science 185(4157), 1124–1131. The foundational paper of the program.

Thaler, R. H. & Sunstein, C. R. (2008). Nudge. Improving Decisions About Health, Wealth, and Happiness. Yale University Press. An application of the heuristics-and-biases catalog to the design of public policy.

Pronin, E., Lin, D. Y. & Ross, L. (2002). The Bias Blind Spot. Perceptions of Bias in Self versus Others. Personality and Social Psychology Bulletin 28(3), 369–381. The original formulation of the bias blind spot, cited in the glossary.

You might also like

• Irrationality as an evolutionary advantage
• Emotion as noise
• Human memory as a corrupt system
• Decision-making without data

Elsewhere

• Daniel Kahneman — Nobel Lecture (Nobel Prize)
• Daniel Kahneman: Prospect Theory of Decision Making (UBS Nobel Perspectives)