Articulate isn't wise

A friend of mine had a rough month. We talked on the phone. I said things like that sounds really hard and I'm so sorry, man. Mostly I just listened. After we hung up I felt like I'd been useless.

He texted me later that night. He'd been having long conversations with ChatGPT about it, and it had helped. He sent me a screenshot. The reply was thoughtful, structured, gentle in the right places — better, honestly, than anything I'd said. He told me the AI made him feel really understood.

I was glad he was getting something out of it. I also couldn't shake a small feeling, the kind you get when something is off in a way you can't immediately name.

This is the post about that small feeling.

We trust the voice that sounds like it knows

There's something humans have always done with grief, and with anxiety, and with any of the hard parts of being a person: we trust the voice that sounds like it knows what it's talking about. Articulation is a proxy for understanding. Someone who has the words for an experience is presumed to have the experience.

That proxy worked for most of human history, because the only entities producing fluent language about feelings were other humans who had felt things. The proxy didn't have to be perfect. It just had to be good enough.

It isn't anymore.

AI is now more articulate about grief, anxiety, loneliness, parenting, and self-doubt than most humans are about their own. It can structure a response to your friend's hard month better than you can. It can name what they're feeling more precisely than they can. It can produce a paragraph that lands in a way your fumbling that sucks never will.

And so we're doing the thing the proxy tells us to do. We're handing the conversations that matter most to the voice that sounds the most like it knows.

The wise-sounding voice is the one with no felt grasp. The inarticulate friend is the one who actually knows.

AI is fluent in feelings it hasn't felt

AI has read every description of every human experience that has ever been written down. Every poem about grief. Every memoir of depression. Every Reddit thread about anxiety, every essay on loneliness, every therapist's blog post about healthy boundaries. It has all of it.

It has been in none of it.

The wisdom in the voice — and there is real wisdom in the voice, I want to be honest about that — is borrowed wholesale from the people who actually went through the thing. The model is performing a kind of fluent retrieval. It is not telling you what it has learned from being alive. It has not been alive.

This is not a gap that gets smaller as the models get bigger. It is a different kind of gap. It is the gap between knowing-about a thing and knowing-from-inside it.

Where the pattern bites hardest

The products growing fastest in this category right now — AI therapists, grief chatbots, parenting coaches, companionship apps — are built precisely in the domains where felt experience is the whole content of the help being asked for.

The user ratings look great. The replies land beautifully. The thing the words are doing, though, is not what care is. Care has always been about another person being there, with their own felt history of having been somewhere similar. The AI version of it is a beautiful, well-phrased imitation of that, produced by something that has never been anywhere.

What to do with this

Two takeaways, one for each side of the screen.

If you use these tools — and I do, a lot:

Notice the proxy. When an AI response feels more comforting than a friend's, take an extra beat before concluding it was wiser. Sometimes it was. Sometimes it was just more articulate.

The inarticulate text from someone who's actually been there is worth more than it sounds. The friend who sat with you in silence may have helped more than the model that wrote you a paragraph.

When something is genuinely hard, try not to do all of it with the AI. Not because the AI is dangerous. Because handing the moments that matter most to the voice with no skin in them is a quiet way of being alone in them.

If you build in this space:

Design for handoff, not substitution. Make the AI good at being the first thing a person reaches for, and bad at being the only thing.

Eval the product not on whether it said the right thing, but on whether the user left it more or less alone with what they were actually feeling.

Resist the temptation to optimize for the rating that comes from "wow, that response really got me." That is exactly the rating an articulate-but-empty system will excel at, and chasing it will quietly walk you away from what you're actually trying to build.

A note on what I don't know

I do not actually know that AI has no inner experience. Nobody does. The hard part of this question — whether there is anything it is like to be a model — is genuinely open, and anyone who tells you they've settled it is selling you something.

The argument doesn't depend on settling it. It depends on the asymmetry: all of the behavior, none of the proof. And the asymmetry matters most exactly where we are now deploying it most — in the domains of life where the proof would actually count.

“AI can describe a heartbreak. It can't be in one. The mistake is letting the description do the work of presence.”