A research team led by Zhen Fang published on arXiv (August 4, 2026) Video-DeepResearch (Video-DR), a framework that extends vision-language models from static frames to continuous video, combined with open web search. The agent watches the video, identifies what it doesn’t know, then searches the web for it.

The paper documents two defects in current models. First: the agent ignores visual tools and resolves everything as text. Second: it answers from internal memory instead of using tools. Video-DR enforces a staged pipeline: watch frames first, then open the browser.

The numbers claimed in the paper: Video-DR-35B-A3B reaches 64.0% accuracy on Video-DR-Bench (200 multi-hop questions created by the team), versus 59.0% for Claude-4.5-Sonnet, 52.5% for GPT-5, and 57.5% for Gemini 2.5 Pro. The smaller variant, 30B-A3B, hits 59.3%, competitive with Claude.

Why it matters. If you work with video content, the difference between a model that describes what it sees and one that cross-references what it sees with external sources is the gap between a viewer and a researcher. As we reported on August 4, VLMs were learning to reason about space instead of just describing it. Video-DR shifts that frontier to video and the web.

Code is open on GitHub. It’s academic research with a small benchmark and self-reported results: the signal matters more than the numbers.

In detail

Vision-language models have made rapid progress on text and static images, but video remains difficult terrain. A video isn’t a sequence of independent frames: it’s a stream where information is distributed over time, and a useful question often requires linking a detail at minute 3 to one at minute 47. Current VLMs tend to sample few frames and treat them as separate photos.

The memory problem. Video-DR calls it “parametric knowledge leakage”: the phenomenon where a model answers a question about a video using what it already knows, without actually watching the content. If you ask what car brand appears in the video, the model can guess from memory instead of looking for the logo in the frames. This is a structural flaw: the agent seems to work, but for the wrong reasons.

The modal bias problem. The agent has access to visual tools (zoom, crop, frame extraction) but ignores them and searches everything as text. If you ask about an event in the video, it does a web search instead of watching the video. Video-DR solves this by enforcing an order: the perception phase comes first, and web search tools are unlocked only after the agent has examined the frames. Following a staged-architecture pattern, the pipeline forces the model to “know what it watched” before “searching for what it doesn’t know.”

The architecture. The framework has three components. A decoupled pipeline that separates perception and exploration. A data engine that generates multi-hop questions designed not to be bypassed with parametric memory. Two-phase training: first supervised fine-tuning to teach basic patterns, then Group Relative Policy Optimization (GRPO, a form of reinforcement learning) to let the agent discover tool-use patterns instead of copying them from the training set.

The numbers in context. Video-DR-Bench has 200 instances, created via human-AI collaboration. Video-DR-35B-A3B (35 billion total parameters, 3 billion active in MoE architecture) reaches 64.0%, beating Claude-4.5-Sonnet by 5 points. The 30B-A3B variant hits 59.3%, showing the method works even at more compact scale. Numbers are reported by the team in the August 4, 2026 paper.

The limits. The benchmark is small and built by the same team proposing the method: classic conflict of interest in evaluation. Comparisons with Claude, GPT-5, and Gemini are done in a specific configuration, and test conditions for proprietary models aren’t transparent. The framework is open source on GitHub, but training requires significant resources. Inference on a 35B MoE is feasible on consumer hardware only with aggressive quantization.

What remains open. Video-DR shows that reasoning over video streams is tractable with the right architecture, but doesn’t solve the cost question. Processing continuous video requires far more compute than a single image, and the paper doesn’t measure cost per query. For anyone building video products, alongside agent correctness comes the question of cost per answer.