AgentFold closed-loop search

Scope and limits

Reproduction level

mechanism demo, with an optional paper-artifact smoke test

Claim under test

retaining evaluated successes and failures and allocating later trials from that record can improve executable search over independent proposals.

Excluded claim

this tutorial does not reproduce the reported 5,000 GPU-hour protein-folding search or establish a 7.5 percent lDDT improvement.

Reader, effort, and cost

advanced Python; 4 to 8 hours for the toy search; optional repository smoke test requires Linux, a compatible NVIDIA GPU, and separately obtained artifacts. Keep the toy demo CPU-only and below $10.

Prerequisites

Python 3.11 for the toy demo, Git, basic statistics; for the official smoke test use the repository's Linux, Python 3.9, CUDA 11, PyTorch 1.12.1, PyTorch Lightning 2.0.4, OpenFold, fair-esm, and OpenStructure environment.

Canonical resources

• Paper: https://arxiv.org/abs/2608.26747 (opens in a new tab)
• Official repository: https://github.com/lmqfly/AgentFold (opens in a new tab)
• Repository artifact notice: ESMFold weights, training NPZ, validation structures, splits, evaluation data, and targets are not bundled and must be obtained under their original licenses.

Project plan

1. 01Environment and provenance(30 minutes)

   Create a fresh virtual environment, record Python and package versions, clone the official repository for inspection, and record its commit hash. Checkpoint: a manifest lists every local version and missing artifact.

2. 02Toy executable search space(60 minutes)

   Implement a small regression model whose executable variants choose polynomial degree, regularization, and one gated residual feature. Generate one fixed synthetic train/validation split. Artifact: variants/ plus a deterministic evaluator returning validation MSE and runtime.

3. 03Independent-proposal baseline(45 minutes)

   Sample 24 variants uniformly without access to earlier outcomes. Run seeds 1 through 5. Artifact: one CSV row per variant, seed, MSE, and runtime.

4. 04Closed-loop intervention(90 minutes)

   Store code diff, configuration, score, and failure reason for each node. Select parents with an upper-confidence or MCTS-style score, prohibit duplicate configurations, and bias proposals toward changes that improved validation MSE. Use the same 24-evaluation budget and seeds.

5. 05Evaluation(45 minutes)

   Compare best score, top-five mean, area under best-so-far curve, invalid-variant rate, and wall time. Bootstrap the paired seed difference. Success requires a better median area-under-curve without a higher invalid rate. Falsification occurs if confidence intervals include no useful gain or random search wins under the matched budget.

6. 06Optional official smoke test(2 to 4 hours plus setup)

   Follow the repository environment file and its documented one-epoch, eight-sample smoke command only after acquiring licensed artifacts. Do not treat a successful smoke run as a paper replication.

7. 07Cleanup(10 minutes)

   Remove generated variants and model outputs, retain the manifest and aggregate CSV, and verify that no API key or .env file entered version control.

Controls, failure modes, and safety

Affected topics

Back to the August 28, 2026 briefing