When to use a chat and when not

You already know how the chat works inside, how to talk to it well and that it sometimes makes things up with poise. With all that on board, here comes the question that saved me the most grief: not how to use it, but when. Because an AI chat isn't good for everything, and realising that is what separates the person who makes the most of it from the one who gets into trouble.

It took me a while to see it. At first I'd ask it for anything, like someone breaking in a new tool and trying it against everything. I want you to skip that phase.

It's not a universal tool

The temptation is to treat the chat as if it did everything equally well. And it doesn't. It's excellent at one kind of task and frankly bad at another, and the border between the two isn't arbitrary: it comes from what you already saw in the first steps. An AI chat predicts likely text; it doesn't compute, it doesn't consult an official register, it doesn't guarantee a fact.

So the underlying rule is simple. Where the task is about language and you're going to review the result, the chat shines. Where you need an exact fact or a decision with consequences, the chat is, at best, a starting point you'll have to confirm somewhere else.

Where it shines

This is where I use it without reservations. Drafting and rewriting: asking it for a first draft, changing the tone of a text, shortening it, ordering it. Summarising a long document down to the essentials. Translating. Brainstorming when I've got a blank page and need something, anything, to start thinking against. Explaining a concept I don't understand, and at whatever level I ask, from "like to a child" to "I know the basics, get to the point".

What do all these have in common? That they're language tasks and that the final judge is you. If the summary leaves something out, you see it on reading it. If the rewrite doesn't convince you, you reject it. The chat does the heavy lifting of putting words down, and you put the judgement. There the probabilistic machine plays on its home ground, because you're not asking it for a closed truth, but for material you're going to mould.

Where it's a bad idea

And this is where I got burned. Exact calculation, to start with. A chat predicts text, it doesn't compute like a calculator; if you ask it for a sum with several steps it can give you a number with all the confidence in the world and be wrong. That's what a calculator or a spreadsheet are for, which are deterministic: same operation, same result, always.

Nor for official data and concrete numbers: a price, the text of a law, a date, a medication dose. That you look up in the official source, not in a model that estimates the most likely. And certainly not for critical decisions about health, money or legal matters without a professional behind them. The chat can help you understand the terrain, arrive with better questions for the doctor or the lawyer, but it's not the one who decides. The bill for getting this wrong isn't paid by the machine; you pay it.

Fluency isn't reliability

This is the misunderstanding that confuses people most, and that's why I set it apart. When the chat answers fast, confident and well written, the brain reads it as "this is to be trusted". It's a trap. Fluency isn't reliability. That something is written with ease says absolutely nothing about whether it's true.

The maddening part is that it sounds just as convincing when it's right as when it makes something up. It doesn't lower its tone on reaching the dubious part, it doesn't hesitate, it doesn't warn you. Remember the first step: inside there's no sensor that tells "I know this" from "I'm making this up". In both cases it does the same, chaining the most likely token, and it does it with the same air of certainty. OpenAI itself says it of its models: they can produce incorrect or misleading information and you should approach them with a critical eye and verify what matters in reliable sources.

The question beforehand

Out of all this I came away with a gesture I make almost without thinking before hitting enter. I ask myself one single thing: does this thing I'm about to ask need a verifiable fact or a decision with consequences?

If the answer is no —I want it to rewrite an email, to summarise a report, to give me ideas—, go ahead without fear. If the answer is yes —a number that'll end up on an invoice, a date something depends on, a decision that hurts if it goes wrong—, then the chat can help me understand it, but the final answer I confirm somewhere else. I don't hand it the decision; I ask it to get me ready to make it.

That half-second question is, deep down, the whole criterion of use boiled down. You don't need more not to trip over the big stuff.

To think with, not to arbitrate

If I had to leave you one single sentence from this step, it'd be this: use it to think, draft and explore; don't use it as the final arbiter of facts, numbers or decisions that hurt if they fail. Thinking and drafting is its ground, what it really knows how to do. Arbitrating the truth isn't, however much it looks like it.

You'll see it isn't giving anything up. It's using the tool where it pays off and leaning on another where it doesn't. And notice I've left a loose end on purpose: I've said "confirm it somewhere else" without telling you how. Nor have I said that not all chats are alike and that it's worth choosing which one you talk to. Those are the steps coming up now, the ones that fine-tune this criterion: which model suits you to work with, how to verify what it tells you and what things it won't tell you even if you ask.

Definitions

- Deterministic system: the one that, given the same input, always gives the same output because it follows a fixed rule. A calculator or a spreadsheet: the same sum gives the same number today and always. - Language task: one where the work is to produce or transform text —draft, rewrite, summarise, translate, explain—. It's the ground where the chat pays off, especially if you review the result. - Fluency: the ease with which the chat writes. It sounds confident and well written both when it's right and when it's wrong, so it's no use as a signal that something is true. - Reliability: that you can trust a fact to be correct. It doesn't depend on how it's written, but on whether it's true; that's why fluency doesn't guarantee it. - Official source: the place where a fact is the right one by definition —the bulletin of a law, the website of a body, the leaflet of a medication—. It's where you go to confirm what the chat only estimates.

Further reading

- Maldita.es, Por qué ChatGPT y otros chatbots de IA no son fuentes fiables de información — explains with examples why a language model "is never the last source of anything". https://maldita.es/malditatecnologia/20240410/chatbot-ia-chatgpt-no-fiable-datos-actualizados/ - OpenAI, ¿ChatGPT dice la verdad? — OpenAI itself admits the model can give incorrect or misleading information and recommends verifying what matters. https://help.openai.com/es-419/articles/8313428-chatgpt-dice-la-verdad