Decision-making without data. The radical uncertainty AI doesn't replicate

Once I had to decide something big without half the information I'd have wanted. I didn't get it in time, so I decided anyway. It turned out reasonably well —though I only know that now; back then there was no way to know, and that's exactly what interests me. Almost every decision that really shapes a life looks like that one. And there's a machine that advertises itself as a help for making them which, precisely on that ground, can't do what a human does. Frank Knight separated risk from uncertainty in 1921. AI is tuned for the first and jams on the second.

The distinction few make and almost everyone needs

Frank Knight published in 1921 Risk, Uncertainty and Profit, an economics treatise that left embedded in the discipline a distinction that's still working a hundred years later. Public discourse on AI forgets it with a regularity that's getting hard to put down to chance.

Risk, in Knight, describes a situation with known probabilities. You know what can happen and how often, you compute expected values, you optimize. Rolling a die is risk in the strict sense: the distribution of outcomes is closed and in plain view. In there live insurance, which computes the probability of a claim over large populations and sets premiums accordingly, and much of portfolio management that leans on historical series.

Uncertainty is something else, and the difference isn't one of degree. It's the situation in which the probabilities aren't known and can't be known a priori. You know things can happen; you don't know which or how often; you have no distribution to optimize over. You can't compute expected values without inventing in advance the numbers that go into the calculation, which is no longer calculating. Launching a product into a market that doesn't yet exist falls here. Accepting a job in an emerging sector, moving to an unknown country, marrying someone, also, however much the culture of marriage insists on selling it as a viability calculation.

The distinction isn't scholastic, because it changes the tools. For risk, classic probability serves. For uncertainty it doesn't serve, simply because there's no distribution to apply it to. And most contemporary artificial intelligence is built for the first ground: its predictive models estimate probabilities over learned distributions. When the situation falls outside that distribution, what they return isn't a calibrated «I don't know». It's a projection from what they do know, served without warning that it's a projection.

What happens in the human head when there's no data

There's a poorly documented human capacity, partly because it's hard to fit into a lab, worth naming before going on: that of deciding under radical uncertainty. Moving to action without waiting to have enough information. Choosing a course knowing it might be the wrong one, and without even being able to compute the probability of being wrong.

Seen from classic rationalism, that capacity has a bad press. It sounds like faith, like impulse, like a dinner-table hunch. Cognitive psychology has tracked it under various labels that each catch a piece of the same operation: expert intuition, fast judgment, that visceral hunch Gigerenzer calls gut feeling, the decision guided by Damasio's somatic markers. When the data don't reach, the human apparatus integrates the tacit —the similar prior experience, the situated context, the values, the emotion, the pressure of the clock— and produces an output. It isn't an optimal output, nor can it be in a normative sense, because the situation doesn't allow optimization. It's functional in the only sense that counts here: the agent moves forward instead of staying nailed in place.

Damasio formulated in Descartes' Error (Putnam, 1994) the somatic marker hypothesis as the neural mechanism behind all this. His patients with damage to the ventromedial prefrontal cortex kept intact the capacity to calculate in the abstract and lost the capacity to close a decision when the data were ambiguous. They stayed turning the option over, weighing pros and cons indefinitely, incapable of stopping. Bechara, Damasio and their colleagues documented that paralysis experimentally in the Iowa Gambling Task (Cognition, 1994): intelligence preserved, decision broken. Without the somatic marker, the choice under uncertainty doesn't arrive. What arrives is a calculation that drags on to infinity.

Deciding without data, seen this way, isn't a breakdown of rational thought, but a central functionality of the apparatus. Remove it and what's left is a cognitively impeccable system useless for living.

Three ways of falling silent before the void

It's worth describing, without caricature, what an AI does when asked to decide under radical uncertainty, because the observable isn't a single behavior. There are at least three outputs, and none is the human's.

Sometimes it declines. «I don't have enough information to answer that.» It's a legitimate response, calibrated by RLHF (reinforcement learning from human feedback) so the system doesn't assert what it can't sustain, and often it's the desirable one. The problem isn't in the sentence, but in what comes after: when the user insists or rephrases, the system usually abandons the honest stance and hands over something anyway, almost always projected from the distribution it does master.

Other times it averages. It returns the median of the corpus for similar cases —«people facing this usually consider such-and-such factors»— and that serves if what you want is to know how the median thinks. It doesn't serve, or it flatly deceives, if what you need to know is what's good for you, because your case may be far from that median and the average has no way to detect it.

And sometimes it hallucinates: it produces plausible claims nothing supports and drops them with the usual fluency. It's the option the public conversation flags over and over. Under radical uncertainty its frequency tends to rise, because the model is being pushed to answer about a territory where its training is poor, and the statistical machinery fills the gap with local coherence anchored in nothing.

What a person does in the same fix doesn't resemble any of the three. The person decides, carries the risk of failing, commits to a course and manages the consequences afterward. The AI doesn't take on the commitment because it has no commitments to take on; its «decision» is text on a screen and the consequences don't land on it. On the user they do. That asymmetry changes everything.

Hayek and the scattered fragments

There's another, more macro angle worth incorporating. Friedrich Hayek raised in The Use of Knowledge in Society (American Economic Review 35, 1945) an observation that reads better as an epistemic description than as a political manifesto, even though it was born of a political polemic. The knowledge that matters for deciding in economics doesn't exist as an aggregate available somewhere. It lives as scattered fragments in thousands or millions of heads, each with its context, its circumstances, its local information. No central planner can gather them without losing along the way what made them useful, which was precisely their anchoring to the place and the agent that held them.

The consequence reaches any system that claims to decide well about the specific starting from the aggregate. In aggregating, the local is lost, and the decisions that come out of there turn out sensible on average and wrong precisely in the particular cases where the local was the decisive factor. Contemporary AI, trained on massive corpora of aggregate information, works within that same limitation: it knows a lot about many and little about the concrete individual consulting it.

When that individual, rational within their window of time, delegates the decision to it assuming the machine knows more, the asymmetry flips on them without their perceiving it. The machine knows more in aggregate and less about the concrete, which was exactly what they needed.

Simon's approximation. Enough instead of optimal

Herbert Simon, in Rational Choice and the Structure of the Environment (Psychological Review 63, 1956), developed satisficing —a contraction of satisfy and suffice— to describe how real agents decide when they can't optimize. An agent with limited resources, set in an environment of incomplete information, doesn't comb for the best possible option. It combs for the first one that crosses an acceptable threshold, and as soon as it appears it takes it and abandons the search. It doesn't compare itself with all the alternatives because the alternatives are too many and there's no time or head to examine them.

From classic rationality that looks suboptimal. From Gigerenzer and Goldstein's ecological rationality (Psychological Review 103, 1996), and from the framework of Risk Savvy (Gigerenzer, 2014), it's the only viable thing. Taleb, in The Black Swan (Random House, 2007), picked up Knight's distinction and pushed it to the extremes: what really weighs in a life isn't the quantifiable risks, but the black swans that blow up the distribution we'd taken for good. And there's a nuance rarely told. In many contexts, settling for the acceptable produces better aggregate results than forcing the optimization, because searching beyond the threshold costs, on average, more than finding something barely better is worth.

In the big decisions we're satisficers by default, even if it's hard to admit. We choose a partner among the people we know, not among all the theoretically possible ones. We accept a job when a reasonable offer arrives, not when we've audited the whole market. We buy one of the houses we've seen. The decision is locally good, not globally optimal, and the wisdom we admire in someone who decides well is more in knowing when to stop searching than in searching more.

AI, trained to optimize over explicit metrics, doesn't settle that way naturally. It tends to return the median of the corpus or to unfold the whole fan of alternatives with their pros and cons, which is an informative operation, not a decisional one. When the user, already saturated with information, still doesn't know what to do, the system has fulfilled its technical task without having touched the problem that person had in front of them.

Partner, profession, move, breakup, children

It's worth looking closely at the decisions that really mark a biography, and at how they're made in reality and not in the brochure.

Whether or not to marry someone is decided with incomplete information about who the other really is in the long run, about how the relationship will change, about how the two will change along the way. The couples who last forty years had, at year zero, no data on year forty. They decided without what they'd have needed to optimize, and there was no way to have it.

Choosing a profession is the same thing displaced in time. The consequences of accepting this career, this offer, this change of sector, show up decades later; the information available at the moment of deciding covers, at best, the first few years. Whatever comes after is signed now, blind.

Moving country, city or neighborhood is decided with partial and almost always biased data, because what you think you know about a place from outside doesn't resemble what you discover living it from inside. The breakup has, on top of that, a trap of its own: when a marriage dissolves, a job is left or a friendship ends, both the consequences of cutting and those of not cutting are uncertain, so there's rarely a solid basis for preferring one path over the other. You choose in the dark in both directions.

And then there are children, which are the limit case. The consequences will last the rest of the decider's life, and there's no data whatsoever about the future agent that parenthood will turn them into, because that agent doesn't yet exist. It's an act of operational faith, in the most literal and least devout sense of the expression.

None of these decisions is made with complete data. None admits optimization. They're all made anyway, and most of those who make them survive the consequences and build reasonable lives. Deciding under radical uncertainty is what allows doing all this: not despite having no data, but by recognizing there's none and moving forward with that recognition on your back.

The delegation that doesn't complete

Here the practical problem the spread of AI brings with it peeks out. Popular intuition says a machine with so much aggregate knowledge has to help decide these things better. The intuition is half correct and half a trap, and it's worth separating the two halves.

Correct, because AI contributes structured information, comparable cases, frameworks of analysis, variables we might have been missing. Whoever decides after consulting it knows more, in aggregate, than whoever decides without doing so, and all else being equal more information tends to give a better decision. Denying it would be foolish.

A trap, because the AI doesn't make the decision. It hands over material, not commitment. And when the user, worn out from processing, asks it directly to decide for them, what they get looks like a decision and materially isn't: it's advice said in the cadence of a decision. In acting on that advice it's they who decide, carrying the whole weight, while the system, in legal and operational terms, carries nothing.

The finest consequence arrives slowly, and it's the one really worth holding onto. If a growing portion of radical-uncertainty decisions starts being made after digesting the AI's aggregate material, the human agent stops training the muscle of deciding without data. Even the subjective experience of the act changes shape: it's no longer «I choose assuming I don't know», but «I choose after having processed all the information the system gave me». The shift is false epistemically, because the information still isn't enough for that concrete decision, and true psychologically, because the user feels they've done their homework and that's enough for them to plant their foot and move forward.

What erodes with prolonged use is the internal tolerance for uncertainty. The part of the apparatus that knew how to operate without knowing becomes rarer, less accessible, more uncomfortable to inhabit. And when the next genuinely radical decision arrives —the illness no one saw coming, the unprecedented life upheaval, the choice that resembles no earlier one—, the agent runs to the AI hoping for the data the AI can't have, and ends up more paralyzed than if they hadn't opened it.

The capacity we don't know how to rebuild

The practical question is hard and I prefer to leave it open. If the capacity to decide under radical uncertainty is trained by deciding under radical uncertainty, and if that practice shrinks as delegation to the aggregate system grows, what happens to the human agent who has spent fifteen years consulting the AI before every choice.

Two readings fit. A pessimistic one, by which they lose calibration. An optimistic one, by which they keep it latent and recoverable when needed. There's no data to prefer either one yet, because the mass experiment has been running only a few years. The only thing assertable is that this capacity doesn't sustain itself by inertia: it sustains itself with use, and the use is dropping.

There's a way to slow the erosion without giving up the tool, and it runs through deliberately reserving the decisions where uncertainty is radical and where what's missing isn't aggregate information. Not consulting the system before choosing a partner. Not consulting it before accepting or rejecting the life upheaval. Not going looking in it for the alibi of what the body already senses. Those decisions were trained by putting the whole person in front of the problem, with no shield, and they only keep being trained if the person agrees to put themselves there again.

We aren't better than AI for deciding better with complete data. With complete data whoever has a prefrontal cortex instead of a graphics card almost always loses. We're something else: we're the ones who decide something when the data don't reach, and that capacity, if reserved, stays there, whereas if delegated it goes. Almost every important decision of a life, told honestly, depends on that capacity and not on the other.

Definitions

Risk (Knightian). A situation with known probabilities, over which expected value can be computed and optimized. Classic probabilistic techniques work within this regime.

Uncertainty (Knightian). A situation with probabilities neither known nor estimable a priori. Classic probabilistic techniques don't apply, and the decision is made with other cognitive resources.

Satisficing. A concept associated with Herbert Simon. A strategy consisting of choosing the first option that crosses an acceptable threshold, instead of the best among all the alternatives. Adaptive when the cost of continuing to search exceeds what, on average, finding something better is worth.

Fragmented knowledge. Hayek's thesis (1945). The knowledge relevant to specific decisions doesn't exist in aggregate, but scattered in individual heads with their local context; in aggregating it, what made it useful is lost.

Somatic marker. Damasio's hypothesis (1994). A bodily signal tied to prior experiences that the brain reactivates before similar situations and that works as an emotional prefilter of options. Without it, the decision under radical uncertainty doesn't get closed.

RLHF. The tuning of a model by reinforcement from human feedback, which among other things calibrates when the system should abstain from asserting what it can't sustain.

References

Bechara, A., Damasio, A. R., Damasio, H. & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50(1-3), 7–15. Experimental documentation of decisional paralysis in patients with prefrontal lesions despite preserved intelligence (Iowa Gambling Task). Confirmed on PubMed (PMID 8039375).

Damasio, A. (1994). Descartes' Error. Emotion, Reason, and the Human Brain. G. P. Putnam's Sons, New York. The somatic marker hypothesis as the neural basis of decision under uncertainty.

Gigerenzer, G. (2014). Risk Savvy. How to Make Good Decisions. Viking. The operative distinction between situations of risk and uncertainty and the tools appropriate to each.

Gigerenzer, G. & Goldstein, D. G. (1996). Reasoning the Fast and Frugal Way. Models of Bounded Rationality. Psychological Review 103(4), 650–669. Empirical evaluation of simple heuristics under uncertainty.

Hayek, F. A. (1945). The Use of Knowledge in Society. American Economic Review 35(4), 519–530. A foundational analysis of the fragmented and local nature of the knowledge relevant to deciding.

Knight, F. H. (1921). Risk, Uncertainty and Profit. Houghton Mifflin, Boston. The foundational distinction between risk (known probabilities) and uncertainty (probabilities not estimable a priori).

Simon, H. A. (1956). Rational Choice and the Structure of the Environment. Psychological Review 63(2), 129–138. The development of the concept of satisficing as a realistic model of decision in environments of incomplete information.

Taleb, N. N. (2007). The Black Swan. The Impact of the Highly Improbable. Random House. A contemporary reformulation of the Knightian distinction with an emphasis on extreme events.

You might also like

• The human as a defective machine
• Irrationality as an evolutionary advantage
• Emotion as noise
• The moral delegation

Elsewhere

• Explained: Knightian uncertainty (MIT News)
• Knightian uncertainty (Wikipedia)