1Five hardware worlds

Hooker [1] argues that ideas in AI win when they suit the available hardware. Backpropagation was published in 1986 [12], but deep networks took over only after 2012, when GPUs made dense matrix multiplication cheap [2, 14]. Table 1 lists the four alternatives we test. Each was built, and each lost commercially partly because GPUs improved too fast to compete with.

2Five models

0. Dense network.784 → 256 → 128 → 10 with ReLU: $f(x) = W_3\,\mathrm{ReLU}(W_2\,\mathrm{ReLU}(W_1 x))$.

A. Neighbor grid.Every pixel is a cell with a 16-number state; channel 0 holds the ink. For 10 steps each cell reads its 3 × 3 neighborhood with a learned filter $P$ and updates itself,$$s^{t+1}_{ij} = s^{t}_{ij} + U\,\mathrm{ReLU}\big((P * s^{t})_{ij}\big),$$(1)then all 784 cells vote. No cell ever sees beyond its neighbors.

B. Binary network.784 → 512 → 512 → 10 with every weight and activation in $\{-1,+1\}$, so each dot product is bit logic,$$a \cdot w = 2\,\mathrm{popcount}(\mathrm{XNOR}(a, w)) - n,$$(2)trained with the straight-through estimator [13].

C. Prototype lookup.k-means picks 30 prototypes per digit (300 total); a query takes the label of the nearest one,$$\hat{y}(x) = \arg\min_c \min_j \lVert x - p_{c,j} \rVert^2 .$$(3)

D. Spiking network.784 → 256 → 10 leaky integrate-and-fire neurons over 12 time steps,$$v_t = 0.85\, v_{t-1} + I, \qquad s_t = H(v_t - 1), \qquad v_t \leftarrow v_t - s_t ,$$(4)trained with a sigmoid surrogate gradient [11]. The output neuron that fires most is the answer.

Protocol. MNIST [9], 60,000 training and 10,000 test digits; Adam, cosine schedule, batch 128, 4 epochs, seed 0, no augmentation. Cost is counted in each world’s own unit (Table 2). Stress tests add Gaussian noise $\sigma \in [0, 0.5]$ or shift digits diagonally by 0–4 pixels. All models are exported to ONNX and checked against PyTorch on 100 test images (all agree).

3Results

3.1All five worlds, on your handwriting

Figure 1 runs all five models live on a digit you draw. The sliders apply the two stress tests to your drawing before it reaches the models. Increasing the noise usually leaves the lookup model’s answer unchanged long after the others fail; shifting the digit does the same for the neighbor grid.

3.2Accuracy and cost

Table 2 reports clean test accuracy alongside each model’s size and its work per digit in its native unit. Every world reads handwritten digits; the best alternative world (spiking) comes close to the dense control, and the weakest (the neighbor grid) still reaches about 90% with under 8,000 parameters, roughly 30 times fewer than the dense network. The binary network stores each weight in one bit, so its weights occupy only about 80 KB on a logic-gate chip.

Figure 2 shows test accuracy after each training epoch. The dense, binary and spiking networks are near their final accuracy after one epoch. The neighbor grid learns slowly, rising from 78% to 92% over four epochs, and is clearly not converged. A world built on local processors would have spent its research effort on exactly this problem: training dynamics for iterated local rules, much as our world spent the 2010s on training dynamics for deep stacks of dense layers.

3.3Each world is good at something different

Under pixel noise (Figure 3, top), every network trained by gradient descent degrades sharply: at σ = 0.3 the dense network falls to 59% and the neighbor grid has collapsed to near chance. The lookup model barely moves, keeping 88% even at σ = 0.5. Its prototypes are averages of many training digits, and independent noise averages out of a squared distance to such a smooth template. The binary network is the second most robust at moderate noise, because thresholding the input at 0.5 discards small perturbations.

Under translation (bottom) the ranking reverses. A four-pixel shift reduces every global model to 5–9%, close to the 10% expected by guessing, because each of their weights is tied to an absolute pixel position. The neighbor grid keeps 52%. Its update rule is the same at every cell, so a shifted digit produces a shifted, but otherwise similar, computation. This translation invariance comes from the hardware constraint, not from any design choice we made for robustness.

3.4Inside each world’s model

The figures below open up the three most unfamiliar models on whatever digit is currently in Figure 1. They are computed live from separately exported versions of the same trained networks, which return their internal state as well as their answer.

3.5Where each world makes mistakes

Figure 7 shows the full confusion matrix for any model on the 10,000 test digits. The models do not just differ in how often they err but in which digits they confuse. The lookup model’s errors concentrate on pairs whose average shapes overlap, such as 4 and 9; the neighbor grid confuses digits that share local stroke patterns.

4Discussion

4.1The lottery also picks the benchmark

The usual reading of the hardware lottery is that hardware decides which models get built. Our results suggest a second effect. Clean accuracy on centered, noise-free digits is the benchmark of a world where dense networks are cheap, and on that benchmark dense-style models do well. A world built on associative memory would naturally care about robustness to corruption and about explaining a decision by pointing at a stored example, and it would rank its lookup models first. A world of local processors would treat translation invariance as a basic requirement, the way our world treats it as something to be added through convolution or data augmentation. Each world would have built benchmarks that reward what its hardware does cheaply, and would have found the other worlds’ models disappointing.

4.2What AI would look like in each world

Many tiny processors.AI would be built from local rules iterated over time: models that settle into an answer, grow structure, and repair themselves, in the spirit of cellular automata and reaction–diffusion systems. Research effort would go into stable training of long iterative processes, rather than deep feed-forward stacks. Language models would likely resemble recurrent, spatially laid-out systems rather than transformers, because global attention, which lets every token read every other token, is precisely the communication pattern this hardware cannot afford.

Logic gates.AI would be discrete: binary and ternary networks, decision trees and ensembles, and learned rule systems. Because a trained model is literally a circuit, it could be verified and compiled to custom silicon, and interpretability would be closer to reading a program. The research frontier would be optimization over discrete structures, where gradient methods are approximations at best.

Instant lookup.AI would be memory-centric: retrieval over enormous stores of examples, with learning focused on what to store and how to compare. Many of the ideas that our world rediscovered late, such as retrieval-augmented generation, memory layers and nearest-neighbor language models, would have been the mainstream from the start. Models would be robust and easy to audit, but would need large memories and would generalize less far from what they had seen.

Brain-like.AI would be judged by energy per answer as much as by accuracy. Models would be sparse and temporal, with information carried by the timing and rate of spikes, and they would excel at always-on sensing at milliwatt power. Our spiking network was the most accurate model in this study, which suggests that the obstacle in our world was not capability but the absence of cheap hardware on which sparsity actually saves energy.

4.3Answer to Q1

4.4Limitations

• MNIST is a small, nearly solved task. Gaps between model families widen on harder data, and we cannot claim that our rankings would hold on natural images or language.
• We simulate every world on conventional hardware. Operation counts in Table 1 are proxies; real energy and latency depend on circuit design that does not exist for most of these worlds.
• Each model received the same small training budget but not the same amount of tuning. Each family has decades of published refinements we did not use, so all four alternative models are lower bounds on what their world would achieve.
• The stress tests are two of many possible. We chose them in advance because they test the strengths Table 1 predicts for each hardware.

5Implications for companies

Hardware decides which AI ideas win. Companies betting on new chip types, such as neuromorphic processors, analog in-memory computing, or wafer-scale arrays of small cores, are also betting on different kinds of AI models, whether or not they say so. Our results make that bet concrete: a chip that makes lookup cheap favors robust, auditable, memory-heavy models; a chip that makes local communication cheap favors translation-invariant, iterative ones. Whoever controls the next cheap hardware primitive will shape the next generation of models, and the benchmarks used to judge them. For firms buying AI rather than building chips, the practical lesson is to evaluate models on the failure modes that matter to their own use, not only on the benchmarks inherited from the current hardware winner.

6Data and code availability

Every number, figure and model above comes from the files below. They are static files on this site; the figures read them directly and the models run in your browser.

References

1. S. Hooker. The hardware lottery. Communications of the ACM, 64(12), 2021. arXiv:2009.06489.
2. A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. NeurIPS, 2012.
3. W. D. Hillis. The Connection Machine. MIT Press, 1985.
4. M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio. Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or −1. arXiv:1602.02830, 2016.
5. M. Rastegari, V. Ordonez, J. Redmon, A. Farhadi. XNOR-Net: ImageNet classification using binary convolutional neural networks. ECCV, 2016.
6. K. Pagiamtzis, A. Sheikholeslami. Content-addressable memory (CAM) circuits and architectures: a tutorial and survey. IEEE Journal of Solid-State Circuits, 41(3), 2006.
7. M. Davies et al. Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1), 2018.
8. P. A. Merolla et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197), 2014.
9. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 1998.
10. E. Randazzo, A. Mordvintsev, E. Niklasson, M. Levin, S. Greydanus. Self-classifying MNIST digits. Distill, 2020.
11. E. O. Neftci, H. Mostafa, F. Zenke. Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine, 36(6), 2019.
12. D. E. Rumelhart, G. E. Hinton, R. J. Williams. Learning representations by back-propagating errors. Nature, 323, 1986.
13. Y. Bengio, N. Léonard, A. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv:1308.3432, 2013.
14. N. P. Jouppi et al. In-datacenter performance analysis of a tensor processing unit. ISCA, 2017.
15. A. Sebastian, M. Le Gallo, R. Khaddam-Aljameh, E. Eleftheriou. Memory devices and applications for in-memory computing. Nature Nanotechnology, 15, 2020.
16. S. P. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2), 1982.
17. T. Cover, P. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 1967.