Artificial Analysis v5 and the benchmarker's dilemma

Interconnects AI analyzes Artificial Analysis's Intelligence Index v5, which weights Knowledge Work, Coding, Science and General at 25% each across benchmarks including AA-Briefcase, FrontierCode 2.0, Terminal-Bench 4.0 and HLE-Diamond, and questions the new ranking that places eight closed models ahead of the best open-weight model, Kimi K3.1 Pro. The author argues that aggregating scores into a single number always involves trade-offs, that the shift to private benchmarks trades community bug-finding for trust in the benchmark builder, and that Artificial Analysis's business model of paid verification by labs creates a trust dilemma. The piece calls for more independent intelligence indices with different biases, noting that closed labs like Anthropic and OpenAI have largely moved past caring about these indices while they remain existential for smaller labs.