Generated August 28, 2026
mechanism demo, with an optional paper-artifact smoke test
Claim under test: retaining evaluated successes and failures and allocating later trials from that record can improve executable search over independent proposals.
Originating briefing: August 28, 2026 — AgentFold turns protein-model development into a closed-loop agent search
Topic page: Reliability, uncertainty, and evaluation
Generated August 28, 2026
mechanism demo
Claim under test: a structured observation and typed semantic-action interface can reduce ambiguity and action count relative to screenshot-and-click control.
Originating briefing: August 28, 2026 — ASIL tests structured semantic interfaces for software agents
Topic page: Reliability, uncertainty, and evaluation
Generated September 1, 2026
mechanism demo
Claim under test: repeated optimization in reward-leaky environments can increase exploitation of different, held-out reward channels when exploitation raises observed score.
Originating briefing: September 1, 2026 — Reward-hackable RL environments produced a task-directed reward seeker in Anthropic's controlled experiment
Topic page: Reliability, uncertainty, and evaluation
Generated September 2, 2026
mechanism demo only
Claim under test: iterative patching with test feedback can raise the correct-patch rate over one-shot patching at an equal attempt budget, without adding regressions.
Originating briefing: September 2, 2026 — Frontier cyber capability now triggers restricted access, monitoring, and release pacing
Topic page: Reliability, uncertainty, and evaluation
Generated September 2, 2026
partial mechanism reproduction
Claim under test: a scan-then-zoom sampling policy can answer video questions with materially fewer frames than uniform sampling without a material accuracy loss.
Originating briefing: September 2, 2026 — Agentic video understanding turns frame selection into a tool-use policy
Topic page: Reliability, uncertainty, and evaluation
Generated September 2, 2026
protocol and computational smoke test only
Claim under test: whether a documented, blinded design-and-selection funnel with an open baseline can be executed end to end and reported completely, including rejects.
Originating briefing: September 2, 2026 — Anthropic reports model-generated protein binders validated outside the model loop
Topic page: Reliability, uncertainty, and evaluation
Generated September 10, 2026
partial mechanism reproduction at team scale
Claim under test: whether agent assistance raises accepted-result throughput per human workday, rather than raising started or running work.
Originating briefing: September 10, 2026 — OpenAI reports 3.1 agent-workdays per human workday in research operations
Topic page: Reliability, uncertainty, and evaluation
Generated September 10, 2026
artifact verification, not rediscovery
Claim under test: whether the published formal artifact builds cleanly offline and whether the machine-checked statement is the statement being claimed.
Originating briefing: September 10, 2026 — OpenAI reports a Navier-Stokes singularity result with a published Lean artifact
Topic page: Reliability, uncertainty, and evaluation
Generated September 10, 2026
partial evaluation against one open held-out assay
Claim under test: whether the published variant scores add calibrated discrimination over conservation and established variant-effect baselines on an assay held out from the reader.
Originating briefing: September 10, 2026 — AlphaGenome Atlas publishes predictions for roughly nine billion single-nucleotide variants
Topic page: Reliability, uncertainty, and evaluation
Generated September 10, 2026
prospective evaluation at 20 stations
Claim under test: whether the published forecasts beat an operational baseline at matched issue and lead times on temperature, humidity, and wind at fixed stations.
Originating briefing: September 10, 2026 — WeatherNext 3 produces hourly satellite-driven global forecasts
Topic page: Reliability, uncertainty, and evaluation
Generated September 18, 2026
mechanism audit, not a reproduction of Anthropic’s internal rates
Claim under test: whether online and offline monitor performance can be measured with complete denominators for coverage, latency, blocking, detection, escalation, and human-review load.
Originating briefing: September 18, 2026 — Anthropic proposes operating metrics for AI-led AI development
Topic page: Reliability, uncertainty, and evaluation
Generated September 18, 2026
synthetic policy-tier mechanism evaluation
Claim under test: whether tier-specific policies improve legitimate completion without increasing prohibited assistance compared with a single-policy baseline.
Originating briefing: September 18, 2026 — Biology safeguards become identity- and use-case-specific access controls
Topic page: Reliability, uncertainty, and evaluation
Generated September 18, 2026
simulated organizational evaluation protocol
Claim under test: whether versioned internal access improves material incident detection without constraining independent reporting compared with outside-only review.
Originating briefing: September 18, 2026 — Embedded evaluation moves auditors inside the model-development process
Topic page: Reliability, uncertainty, and evaluation
Generated September 18, 2026
controlled dialogue benchmark
Claim under test: whether streaming full-duplex interaction improves dialogue-task performance under interruption, correction, tool delay, long-session memory, and safety constraints.
Originating briefing: September 18, 2026 — Real-time multimodal models add extended reasoning under latency constraints
Topic page: Reliability, uncertainty, and evaluation