Infrastructure, efficiency, and open ecosystems

AI capability is constrained by memory bandwidth, accelerator availability, energy, interconnect, latency, data movement, software maturity, and evaluation cost. MIT CSAIL and Microsoft report work on agent speed and energy; NVIDIA remains central to accelerators and physical-AI simulation; Hugging Face and open research communities make models and artifacts easier to inspect and reuse.

Open weights, open code, open data, and reproducible experiments are different properties. A credible ecosystem record states which are available, under what license, with what provenance and hardware assumptions.

Amdahl's law bounds speedup when fraction s of a workload remains serial:

Speedup(n)=1/[s + (1-s)/n].

Worked example: if 10 percent is serial, 8 workers give 1/[.1+.9/8] ≈ 4.71, not 8. Infinite workers approach a maximum speedup of 10. Optimizing the parallel portion alone cannot cross the serial bound.

For inference, throughput and latency differ. Batching may increase tokens per second while worsening time-to-first-token. Quantization reduces memory per parameter but can alter accuracy and hardware efficiency. An honest comparison reports model, precision, sequence lengths, batch, cache policy, hardware, power boundary, warm-up, and output-quality check.

Energy per successful task is more decision-relevant than energy per token when systems use retries or different success rates:

E_success = total energy / verified successful tasks.

1. 1965 — Moore's observation. Gordon Moore extrapolated economical component-density growth. It became an engineering target, not a physical law. Original article (opens in a new tab).
2. 1967 — Amdahl's law. Gene Amdahl formalized the serial bottleneck in parallel speedup. AFIPS (opens in a new tab).
3. 1982 — Systolic arrays. H. T. Kung described regular arrays that move data rhythmically through processing elements, anticipating accelerator dataflow ideas. IEEE Computer (opens in a new tab).
4. 1987 — UCI repository. A shared collection of datasets supported comparable machine-learning experiments. UCI (opens in a new tab).
5. 1999 — WEKA. The Waikato team provided an accessible machine-learning workbench for reproducible education and experiments. Official project (opens in a new tab).
6. 2004 — MapReduce. Jeffrey Dean and Sanjay Ghemawat described a programming model for large-scale data processing on clusters. OSDI (opens in a new tab).
7. 2007 — CUDA. NVIDIA's general-purpose GPU platform made accelerator programming broadly accessible. CUDA archive (opens in a new tab).
8. 2009 — ImageNet. Fei-Fei Li and collaborators built a large labeled visual dataset and benchmark, showing the power and governance burden of shared data infrastructure. CVPR (opens in a new tab).
9. 2009 — Deep learning on GPUs. Raina, Madhavan, and Andrew Ng demonstrated large-scale deep learning using GPU computation. ICML paper (opens in a new tab).
10. 2010 — Theano. The Montréal team described a symbolic tensor compiler with automatic differentiation and GPU execution. SciPy paper (opens in a new tab).

1. 2012 | DistBelief. Jeffrey Dean and collaborators described distributed training with Downpour SGD and Sandblaster L-BFGS, including networks far larger than feasible on one machine. The work extended data-parallel optimization, while depending on specialized clusters and accepting asynchronous consistency tradeoffs. NeurIPS (opens in a new tab).
2. 2016 | TensorFlow. Martín Abadi and collaborators described a dataflow system spanning research experiments and production deployment, then released it as an open ecosystem. Its abstraction improved portability and tooling but introduced framework and graph-runtime complexity. OSDI (opens in a new tab).
3. 2020 | ZeRO. Samyam Rajbhandari and collaborators partitioned optimizer state, gradients, and parameters across devices to reduce redundant memory. The method enabled much larger models, but introduced distributed-communication, orchestration, and hardware assumptions that complicate reproduction. SC20 (opens in a new tab).
4. 2022 | FlashAttention. Tri Dao and collaborators tiled exact attention to reduce reads and writes between GPU memory levels. The input-output-aware design improved speed and memory use, while leaving quadratic arithmetic and hardware-specific kernel constraints. NeurIPS (opens in a new tab).
5. 2023 | PagedAttention and vLLM. Woosuk Kwon and collaborators adapted virtual-memory paging to key-value caches and reported two-to-four-times serving throughput at comparable latency in tested workloads. The gains depend on request mix, hardware, software version, and do not change model quality. Primary paper (opens in a new tab).
6. 2025 | DeepSeek-R1. DeepSeek-AI reported reinforcement-learning-based reasoning models and released model weights and distilled variants. The release broadened access to reasoning research, but vendor-reported comparisons, limited training-data and compute disclosure, and incomplete reproducibility constrain the conclusion. Primary paper (opens in a new tab).

Objective: compare full precision, reduced precision, and batching without hiding quality or system assumptions.

Environment: one small open model that fits locally, one fixed runtime, and a public prompt set with deterministic scoring. Record model hash, license, runtime version, device, memory, and power-measurement boundary. If power cannot be measured, label a proxy instead of calling it energy.

Procedure: run a warm-up; benchmark batch sizes 1, 4, and 8 at two sequence lengths; compare full and reduced precision; collect median and tail latency, throughput, peak memory, and verified task score. Use the same decoding and seeds. Repeat each condition enough times for stable intervals.

Expected result: quantization should reduce memory and may improve throughput; batching should improve throughput but can worsen individual latency. Some conditions may alter output quality.

Success criterion: publish a configuration-complete table and identify a Pareto frontier rather than one “fastest” system.

Falsification: apparent efficiency gains disappear when quality and successful-task rate are matched.

Pitfalls: mixing warm and cold runs, comparing different sequence lengths, ignoring failed outputs, and reporting wall-plug energy without defining the measurement boundary.

Claim: open, reproducible stacks accelerate useful experimentation.

• Baseline: a release with weights or API only.
• Intervention: weights, data documentation, preprocessing, configuration, seeds, evaluator, and hardware record.
• Primary metric: independent teams reproducing the headline result within a prespecified tolerance.
• Secondary metrics: time to reproduce, unresolved ambiguities, cost, and license/provenance blockers.
• Uncertainty: multiple independent teams; report variation rather than one successful replication.
• Stopping rule: freeze the release package before external teams begin.
• Disconfirming result: richer artifacts do not improve replication rate or time after project complexity is controlled.

NVIDIA Research (opens in a new tab), Hugging Face (opens in a new tab), Apple Machine Learning Research (opens in a new tab), Google Research (opens in a new tab), Microsoft Research (opens in a new tab), MIT CSAIL (opens in a new tab), Mistral AI (opens in a new tab), Cohere Research (opens in a new tab), EleutherAI (opens in a new tab), AI2 (opens in a new tab), LAION (opens in a new tab), and Epoch AI (opens in a new tab). Also monitor SemiAnalysis (opens in a new tab) as Tier B industry analysis, not as primary experimental proof.

• High confidence, 6–24 months: infrastructure comparisons will increasingly use quality-adjusted cost and energy per successful task. Disconfirming signal: these metrics fail to explain purchase or deployment outcomes better than raw throughput.
• Medium confidence, 12–36 months: reproducibility disclosures will become a differentiator among open-model releases. Disconfirming signal: independent reproduction remains unrelated to data, configuration, and evaluator availability.