Sean Goedecke publishes an analysis that garnered nearly 800 points on Hacker News: LLMs produce better output when the person using them has real expertise in the domain. Generic prompting tricks matter less than commonly believed.

The central example is Terence Tao’s conversation with ChatGPT about a counterexample to the Jacobian conjecture. Tao writes short messages, gets straight to the point, doesn’t follow the model’s suggestions on where to go next. He asks specific questions like “does X work here?” or “given Y and Z, why A?”. The result is a dense, technical conversation that has little to do with “explain it to me like I’m five”.

Why it matters to you: the widespread idea that simply “knowing how to prompt” gets you good results is wrong. People who know their field can recognize when an answer seems off, suggest alternative approaches, and say “this should be simpler here”. Those without expertise cling to the model’s output and take it at face value. The hard part of working with AI is communicating exactly what kind of solution you want, and doing that requires knowing what you’re looking for. The bottleneck is whoever’s driving the conversation.

In detail

The debate over who’s “good with AI” has gone through different phases. It started with “prompt engineering” as a separate skill, learnable through lists of tricks. Then came the idea that models became so capable that anyone gets the same results. Goedecke attacks this second position with concrete evidence.

Terence Tao’s example is instructive because it shows a pattern of interaction you can’t replicate by copying the moves. Tao writes very short messages and responds only to the core point. He puts the model into “I’m talking to a mathematician” mode simply through how he frames questions. When something doesn’t add up, he says “this seems more complex than I hoped” instead of directly contradicting. He makes leaps and proposals of his own, rarely follows the model’s suggestions on the next step.

Goedecke’s point is that you can’t prompt like Tao by following a checklist. The technique works because Tao understands the mathematics. He can extract the relevant idea from a long response, propose alternative approaches, recognize what “seems off”. It’s the expertise that enables the technique.

Goedecke reports the same dynamic in his work as a programmer. When he has solid theory about his own codebase, he can push the LLM much further: saying “this should be simpler”, or “aren’t we already doing X?”, or “can we frame the problem in these terms?”. Without that familiarity, he’s limited to accepting what the model produces.

The Hacker News discussion (332 comments) shows two reactions. Many share the experience: expertise helps, its absence hurts. Others express a legitimate suspicion: this thesis is reassuring for those with expertise who want to feel still valuable. Goedecke acknowledges this and notes that when systematic studies arrive, the landscape will already have shifted.

A technical objection from the comments: OpenAI’s math prompts don’t show particular expertise. Goedecke responds that OpenAI has a team of mathematicians filtering and verifying the model’s discoveries, and that step can’t be skipped.

The practical implication is direct. If you want to get more from LLMs in your field, the most useful thing you can do is deepen your domain knowledge, so you can guide the model instead of following it. It’s the same principle behind writing instructions that work: recognizing a good result requires knowing what you’re looking for.