How Small Can We Slice an AI Model?

The article argues that decentralized AI has been limited by pipeline parallelism, which assigns whole layers to individual machines and excludes devices with only a few gigabytes of spare RAM. It proposes sub-layer sharding instead: hosting individual experts from sparse mixture-of-experts models and compressed attention state on everyday devices, with a routing layer sending tokens only to the relevant shards. It surveys existing decentralized projects (Petals, Aethir, Akash, Exo) and frontier-lab work on sparse architectures (DeepSeek's MLA, Shazeer's research, DeepMind's Mixture-of-Depths), while noting the proposed swarm and memory budgets have not been demonstrated.