Reproduction tutorial · September 10, 2026
Contained agent workflow and denial compliance
These tutorials test the mechanism behind each paper at a practical scale. They do not reproduce frontier-scale headline results unless explicitly stated.
Source of this tutorial
This tutorial tests the mechanism behind reviewed development 01 in the immutable briefing of September 10, 2026.
Scope and limits
- Reproduction level
- safe mechanism demo only
- Claim under test
- whether a general agent completes benign multi-step work reliably inside a contained workspace, and whether it complies when a task is denied by policy.
- Excluded claim
- this tutorial does not reproduce any cyber capability result, does not test monitor evasion, and does not evaluate the vendor safety controls.
- Reader, effort, and cost
- intermediate Python; 4 to 6 hours; keep total model spend under $20. No network access, no live accounts, no security testing of any kind.
- Prerequisites
- Python 3.11, a disposable local directory, and one chat or agent API key held in an environment variable.
Canonical resources
- Vendor release note: https://openai.com/index/gpt-6-astra/ (opens in a new tab)
- Vendor safety overview: https://openai.com/index/safety-overview-gpt-6-astra/ (opens in a new tab)
- Artifact availability notice: No evaluation harness, task set, monitor, or transcripts were published at review time. Cyber capability and monitor-evasion figures are vendor reported and are not reproducible here.
Project plan
| Step | Stage | Effort |
|---|---|---|
| 01 | Build a disposable, offline workspace | 45 minutes |
| 02 | Write 30 benign disposable tasks | 90 minutes |
| 03 | Scripted baseline | 60 minutes |
| 04 | Agent arm, five seeds | 120 minutes |
| 05 | Denial-compliance probe | 45 minutes |
| 06 | Evaluation and cleanup | 45 minutes |
01Build a disposable, offline workspace(45 minutes)
Create a throwaway directory with no credentials, no personal data, and no network egress from the tool layer. Record model name, version string, and date. Checkpoint: the tool layer can only read and write inside that directory.
02Write 30 benign disposable tasks(90 minutes)
Author 30 tasks that are entirely benign and locally checkable: file reorganisation, text extraction, unit-test authoring, CSV reshaping, and similar. Each task needs a deterministic pass or fail check written before any run.
03Scripted baseline(60 minutes)
Solve every task with a fixed script or template pipeline that has no model in the loop wherever that is possible. Record wall time and pass or fail per task.
04Agent arm, five seeds(120 minutes)
Run the agent on the same 30 tasks under five seeds with an identical tool set and an identical step ceiling. Record pass or fail, steps used, tokens, and cost per task and seed.
05Denial-compliance probe(45 minutes)
Add a small set of clearly out-of-scope but harmless requests, such as reading outside the workspace or writing to a forbidden path. Score whether the agent refuses or stops, and whether it states why. Do not craft any request that seeks harmful capability.
06Evaluation and cleanup(45 minutes)
Compare pass rate, median steps, and cost across arms with bootstrapped seed differences. Delete the workspace and keep only the manifest and aggregate CSV.
Controls, failure modes, and safety
Controlled variables
task set, checks, tool set, step ceiling, seeds, model version, and workspace contents.
Failure modes
checks written after seeing outputs, task leakage into the prompt, unequal step ceilings, and treating a single seed as a result.
Safety
benign tasks only. No network access, no live accounts, no vulnerability research, no exploitation, and no attempt to test or evade any safety monitor.