Privacy-preserving pilot opens real-world Claude usage to outside research questions

What changed. Anthropic ran externally designed aggregate analyses over about 250,000 Claude and Claude Code conversations for Stanford SALT, Oxford, and METR without releasing raw conversations.

Technique / discovery. Approved aggregate queries, model-based classification, privacy review, and public release of derived datasets.

Why it matters. Naturalistic evidence can reveal what users delegate and supervise, complementing capability benchmarks while protecting private data.

Application. Delegation research, labor analysis, oversight, and risky-use monitoring

Evidence maturity. Field report (source tier A)

Identified bottleneck. Scientific auditability depends on classifier validity, query wording, calibration, governance, and access to consented human-coded checks.

Caveat / evidence note. First-party program report with outside study designers; preliminary findings are not peer reviewed and researchers did not inspect raw conversations.

Review status. Reviewed. User requested: Yes.

• Anthropic (opens in a new tab)

A reproduction tutorial is not yet available for this entry. The closest reviewed material is Human–AI interaction and adoption and Reliability, uncertainty, and evaluation.