Human–AI interaction and adoption

Anthropic's analysis of Claude Code sessions reports a common division in which users plan and the model executes, with expertise associated with better outcomes. Claire Vo's hands-on model comparison argues from a small practitioner sample that interaction style and willingness to act affect product choice. Redpoint's Unsupervised Learning episode with Benedict Evans frames adoption through earlier platform transitions. These sources have different evidence strength: observational research, practitioner evaluation, and strategic commentary should not be collapsed.

The durable lesson from HCI and organizational research is that value belongs to the joint human–machine–workflow system. Capability, trust, skill, incentives, task redesign, and control interact.

A person should delegate when expected joint utility exceeds manual utility. Let q be probability the AI succeeds without correction, V the value of success, C_f failure cost, C_r review cost, and C_m manual cost. A simplified expected AI utility is

U_AI = qV - (1-q)C_f - C_r.

Manual utility is U_manual = V - C_m if manual success is assumed. Delegate when U_AI > U_manual, but the assumptions must be measured.

Worked example: q=.9, V=100, C_f=200, C_r=10, and C_m=45. Then U_AI=90-20-10=60; U_manual=55; delegation narrowly wins. If failure cost rises to 400, U_AI=40 and manual work wins. Consequence changes the policy even when model accuracy is unchanged.

Trust should be calibrated, not maximized. Over-trust creates misuse; under-trust creates disuse. An interface should help the person estimate limits, inspect evidence, and intervene at the right point.

1. 1962 — Augmenting Human Intellect. Douglas Engelbart framed computers as tools for improving collective problem solving and built NLS around new work practices. SRI/Engelbart archive (opens in a new tab).
2. 1983 — Model Human Processor and GOMS. Card, Moran, and Newell modeled human cognitive and motor operations to predict interface performance. Book record (opens in a new tab).
3. 1983 — Direct manipulation. Ben Shneiderman emphasized visible objects, rapid reversible actions, and immediate feedback. IEEE Computer (opens in a new tab).
4. 1989 — Technology Acceptance Model. Fred Davis linked perceived usefulness and ease of use to adoption intention. It is influential but does not capture every organizational or ethical factor. MIS Quarterly (opens in a new tab).
5. 1999 — Mixed initiative. Eric Horvitz studied systems in which people and software share initiative under uncertainty and interruption cost. CHI (opens in a new tab).
6. 2002 — Organizational complements. Bresnahan, Brynjolfsson, and Hitt linked IT value to workplace organization and skilled labor. Quarterly Journal of Economics (opens in a new tab).
7. 2003 — Task model of computerization. Autor, Levy, and Murnane distinguished routine and non-routine task content, shifting analysis from occupations to tasks. Quarterly Journal of Economics (opens in a new tab).

1. 2015 | Algorithm aversion. Berkeley Dietvorst, Joseph Simmons, and Cade Massey ran forecasting experiments showing that people can reject an algorithm after observing it make mistakes even when it outperforms them. The finding sharpened older automation-trust questions, but depended on task framing and controlled laboratory settings. Journal of Experimental Psychology (opens in a new tab).
2. 2019 | Guidelines for human-AI interaction. Saleema Amershi and collaborators synthesized and tested a practical set of interaction guidelines across the user journey. The work connected human-factors principles to AI products, but heuristics do not provide causal guarantees across every domain or population. CHI (opens in a new tab).
3. 2023 | Experimental evidence on generative AI and writing. Shakked Noy and Whitney Zhang randomly assigned professionals to complete writing tasks with or without a generative model and reported faster completion and higher evaluated quality. Short controlled tasks, a specific model, and a selected participant pool limit generalization to whole jobs. Science (opens in a new tab).
4. 2025 | Generative AI at Work. Erik Brynjolfsson, Danielle Li, and Lindsey Raymond studied 5,172 customer-support agents and reported an average 15 percent increase in resolutions per hour, with larger gains for less experienced workers. The setting offers field evidence, but it is one firm, occupation, workflow, and tool rather than a general labor-market estimate. Quarterly Journal of Economics (opens in a new tab).

Objective: evaluate delegation policies rather than raw model accuracy.

Environment: 200 synthetic support or document-review cases with difficulty, consequence, and reversibility labels. Provide a model score and a simulated expert whose accuracy and response time are measured rather than assumed.

Conditions: always act; always ask; fixed confidence threshold; cost-sensitive policy using estimated failure consequence and review cost.

Metrics: total utility under a prespecified cost matrix, harmful-error rate, human interruptions, latency, review workload, and calibration by consequence band.

Expected result: always ask should be safe but expensive; always act should fail on high-consequence ambiguity; a calibrated policy should improve expected utility if confidence and costs are informative.

Success criterion: lower total expected cost than both fixed policies on a locked test set.

Falsification: adaptive policy adds no value once human error, interruption, and latency are included.

Pitfalls: constructing costs after seeing outcomes, assuming the expert is perfect, and measuring satisfaction without task success.

Claim: domain expertise improves agent outcomes because experts plan, recognize failure, and review judgment-heavy steps.

• Baseline: novice users with a standard interface.
• Interventions: expert users; novices given structured planning and review support.
• Primary metric: verified task outcome, not self-reported productivity.
• Secondary metrics: time, interventions, error detection, and retained learning.
• Controls: task difficulty, prior tool familiarity, model version, and training time.
• Uncertainty: preregistered sample size and mixed-effects analysis by participant and task.
• Disconfirming result: expertise effect disappears after controls or structured support closes no part of the gap.

Stanford HAI (opens in a new tab), MIT (opens in a new tab), Microsoft Research (opens in a new tab), Anthropic Research (opens in a new tab), and Ethan Mollick (opens in a new tab) for adoption and joint-work research. Use How I AI with Claire Vo (opens in a new tab) for small-sample practitioner workflows and Unsupervised Learning with Jacob Effron (opens in a new tab) for strategy; label both as interpretation rather than primary proof. The immediate empirical source is Anthropic's Claude Code expertise study (opens in a new tab).

• High confidence, 6–24 months: enterprise evaluations will report human time, intervention, and verified outcome alongside model metrics. Disconfirming signal: these measures fail to predict adoption or realized value.
• Medium confidence, 12–36 months: interfaces will expose consequence-sensitive ask–act–escalate policies instead of one global autonomy setting. Disconfirming signal: users prefer and perform better with fixed autonomy across diverse task risks.

---