State of AI — 16 August 2026

Evidence class: reported benchmark results plus editorial synthesis.

Scale Labs describes Terminal-Bench 3.0 as a collection of executable, programmatically verified terminal tasks and reports that the strongest frontier systems solve fewer than 40 percent of them. HORIZON analyzes more than 3,100 long-horizon agent trajectories to diagnose where execution breaks. Gaia2 and the Agents Research Environments program similarly emphasize interactive environments over static question answering.

Why it matters: a convincing explanation is not evidence that an agent changed the environment correctly. Teams need event traces, tool-call records, state transitions, recovery metrics, and independent end-state verifiers.

What would weaken this synthesis: if static-answer benchmarks reliably predicted success, recovery, and side effects on held-out executable work across organizations, the added cost of trajectory evaluation would be harder to justify.

Sources: Terminal-Bench 3.0, Scale Labs (opens in a new tab); HORIZON, arXiv:2604.11978 (opens in a new tab); Gaia2 and Agents Research Environments, Hugging Face (opens in a new tab).

Related topic: Reliability, uncertainty, and evaluation; Agent planning and cognitive architectures.

Evidence class: first-party benchmark analysis and technical reports; editorial synthesis.

OpenAI reports that reasoning-state retention and context compaction materially changed one system's ARC-AGI-3 score. Scale Labs has separately examined whether MCP and command-line tool interfaces alter agent performance. Andrew Ng's account of building “AI Andrew” emphasizes harness error analysis, and NVIDIA's autoresearch workflow packages skills and reinforcement-learning infrastructure around a model.

Why it matters: model comparisons that do not version prompts, context policy, tool schemas, permissions, memory, retry logic, and sandbox state may attribute system-level gains to the wrong component.

What would weaken this synthesis: repeated controlled studies showing negligible performance variance when harness components are changed while the base model and task distribution remain fixed.

Sources: OpenAI research index (opens in a new tab); Scale Labs (opens in a new tab); The Batch issue 353, DeepLearning.AI (opens in a new tab); NVIDIA developer research workflow (opens in a new tab).

Related topic: Agent planning and cognitive architectures; Infrastructure, efficiency, and open ecosystems.

Evidence class: first-party observational study, benchmark design, and practitioner synthesis.

Anthropic's analysis of roughly 400,000 Claude Code sessions reports that users commonly plan while the model executes and that domain expertise is associated with better outcomes. HiL-Bench asks whether an agent knows when to request help, while MIT CSAIL's Battleship-style work uses an internal world model to improve question selection.

Why it matters: “human in the loop” is too vague. A usable design specifies what the person knows, when the system should ask, the expected value of that information, the cost of interruption, and how escalation changes the action policy.

What would weaken this synthesis: strong evidence that expertise adds little after task difficulty and model choice are controlled, or that autonomous policies consistently outperform calibrated escalation on high-consequence tasks.

Sources: Claude Code and expertise, Anthropic (opens in a new tab); HiL-Bench, Scale (opens in a new tab); Teaching agents to ask better questions, MIT CSAIL (opens in a new tab).

Related topic: Human–AI interaction and adoption; Reliability, uncertainty, and evaluation.

Evidence class: vendor technical reports, university paper summary, and ecosystem review.

Google DeepMind describes a three-model Gemini Robotics 2 stack separating whole-body action, embodied reasoning, and on-device adaptation. MIT's VLASH work predicts the future state in which a next action will begin so planning can overlap execution. NVIDIA describes a progression from pretrained world representations to action-conditioned models, while a Hugging Face review maps the simulation substrate across MuJoCo, Isaac Sim, Isaac Lab, and related tooling.

Why it matters: slow deliberation and fast physical control have different latency and safety constraints. A layered design may make those constraints explicit, but the integrated closed loop—not the modules in isolation—must be evaluated.

What would weaken this synthesis: evidence that a single end-to-end policy consistently matches layered systems on long-horizon success, intervention rate, energy, latency, novel-body transfer, and safety under disturbance.

Sources: Gemini Robotics 2, Google DeepMind (opens in a new tab); VLASH, MIT News (opens in a new tab); World action models, NVIDIA (opens in a new tab); Simulation for physical AI, Hugging Face (opens in a new tab).

Related topic: Embodied AI and world models.

Evidence class: field reports, product notes, and benchmarks; editorial synthesis.

OpenAI reports researchers using coding agents to modernize scientific software. Anthropic presents Claude Science as an integrated workbench. Scale's DrugDiscoveryBench evaluates multi-step work in drug discovery, and Isomorphic Labs describes an engine intended to move beyond structure prediction toward design.

Why it matters: code generation and prediction are intermediate steps. Scientific value depends on provenance, executable methods, uncertainty, negative results, domain review, and experimental confirmation.

What would weaken this synthesis: prospective studies showing that isolated model predictions deliver equivalent validated discoveries, time-to-result, and reproducibility without workflow integration or expert review.

Sources: Scientific computing in the age of agentic AI, OpenAI (opens in a new tab); Claude Science, Anthropic (opens in a new tab); DrugDiscoveryBench, Scale Labs (opens in a new tab); IsoDDE, Isomorphic Labs (opens in a new tab).

Related topic: AI for science.

Evidence class: university-reported benchmark result and historical analogy.

Stanford HAI reports that CooperBench found paired coding agents could lose substantial capability relative to their solo performance. The result echoes older distributed-AI findings: communication, commitments, shared state, and task decomposition are engineering variables rather than free benefits of parallelism.

Why it matters: a multi-agent architecture should earn its complexity against a single-agent baseline. Parallelism helps only when work can be partitioned, interfaces are explicit, conflicts are resolved, and integration is independently checked.

What would weaken this synthesis: broad replications in which adding agents reliably improves quality-adjusted latency and cost without specialized coordination design.

Sources: AI coding agents fail at teamwork, Stanford HAI (opens in a new tab); Contract Net Protocol, IEEE (opens in a new tab).

Related topic: Multi-agent coordination.

Evidence class: benchmark and university research summary; editorial synthesis.

HiL-Bench makes help-seeking behavior an explicit evaluation target. MIT CSAIL's question-asking work treats questions as actions chosen using a model of the hidden environment. Both connect modern agent UX to decision theory: request information when its expected improvement to a decision exceeds its cost.

Why it matters: always asking creates interruption and latency; never asking creates silent failure. The policy should depend on consequence, uncertainty, reversibility, and the expected value of the person's information.

What would weaken this synthesis: deployments in which a fixed always-act or always-ask rule dominates adaptive policies across task types and consequence levels.

Sources: HiL-Bench, Scale (opens in a new tab); MIT CSAIL question-asking research (opens in a new tab).

Related topic: Human–AI interaction and adoption; Reliability, uncertainty, and evaluation.

Evidence class: first-party technical report and trustworthy-agent research.

OpenAI describes GPT-Red as a self-play system for generating attacks and training data and reports fewer failures on an internal direct prompt-injection benchmark. Anthropic's trustworthy-agent work emphasizes system-level controls for tool-using agents.

Why it matters: a connected agent changes the attack surface whenever its model, prompt, memory, connector, permission, or external content changes. A periodic checklist cannot cover that rate of change. Security evaluation must be repeatable, adversarial, and tied to release gates and incident learning.

What would weaken this synthesis: evidence that automated red teaming fails to find net-new vulnerabilities or that gains disappear on independently designed, held-out attacks and production incidents.

Sources: GPT-Red, OpenAI (opens in a new tab); Trustworthy agents, Anthropic (opens in a new tab).

Related topic: Safety, security, and alignment.

Evidence class: open artifacts, technical ecosystem reporting, and editorial synthesis.

Karpathy's microgpt continues the tradition of reducing a modern model to inspectable components. Hugging Face, EleutherAI, AI2, LAION, and open-model laboratories make weights, datasets, evaluation code, or educational implementations more accessible. At the same time, physical-AI simulation and agent benchmarks show how much results depend on hidden environment choices, compute, memory, latency, and data construction.

Why it matters: open weights are not the same as an auditable experiment. Reproducibility needs data provenance, preprocessing, configuration, seeds, evaluation code, hardware description, and known limitations.

What would weaken this synthesis: if nominally open releases repeatedly proved reproducible from documentation alone and independent teams found little sensitivity to data, harness, or infrastructure choices.

Sources: Karpathy's site and microgpt materials (opens in a new tab); Hugging Face blog (opens in a new tab); EleutherAI (opens in a new tab); Allen Institute for AI (opens in a new tab); LAION (opens in a new tab).

Related topic: Infrastructure, efficiency, and open ecosystems; Language models and representation.