Andrej Karpathy published a short video on YouTube tackling the 64 sugar cube puzzle, a visual problem-solving challenge that requires imagining and combining objects in space. The discussion on Hacker News went beyond the anecdote, with substantive contributions on what visual puzzles reveal about the reasoning and planning capabilities of models.

Language models have become skilled at reasoning about text and code, but spatial and visual reasoning remains less explored territory. A puzzle that a person solves by looking at and manipulating physical objects is a good test bed for understanding where AI manages on its own and where it needs external representations, iterative feedback, or supplementary tools.

The question running through the HN discussion is the same one that resurfaces in every agent project: understanding whether the model can plan a sequence of steps toward a goal or if it gets lost halfway. It’s the difference between asking an assistant to solve an equation and asking it to organize a physical space while accounting for geometric constraints.

In detail

Frontier model reasoning has been tested primarily on text and mathematics. GPT-5.6 Sol proved open mathematical conjectures from decades past, and public discussion has focused on how well models can “think” in symbols. The sugar cube puzzle shifts attention to a different kind of reasoning: spatial and visual reasoning, where the problem must be solved by imagining or manipulating objects in three-dimensional space.

What makes this case interesting.

Karpathy is no ordinary observer. Former director of AI at Tesla and one of the most influential researchers in deep learning, he has a rare ability to choose examples that isolate a conceptual problem. The fact that he chose a visual puzzle suggests he considers it a test bed for a capability that current models haven’t yet mastered.

What happens in spatial reasoning.

Spatial reasoning requires the model to build and maintain a representation of space: where objects are, how they combine, which configuration achieves the goal. And to update this representation at each step. In language models this map is implicit and must be reconstructed from text, not from direct observation. It’s why a model can solve an equation but stumble on a problem a child solves by looking.

What it means for those using AI.

If you’re building agents or workflows that require reasoning about space, layout, or physical configurations, the point is practical: the model alone might not be enough. You need to give it an external representation of the problem or iterative feedback, as you would with a colleague who reasons better with a drawing in front of them. The principle is the same one three convergent papers demonstrated for textual reasoning: the tools and context you give the agent matter more than model power.

The limitations.

The video is short and not a formal study. There are no numbers, model comparisons, or repeatable methodologies. The Hacker News discussion remains a conversation, not a benchmark. The puzzle still raises a legitimate and verifiable question: measure how much spatial reasoning current models possess and where it’s worth integrating them with tools that help them operate on space rather than imagine it from text.