Weighted State Souping for Efficient Long-Context Retrieval

The paper proposes weighted state souping, a method that learns query-dependent weights for composing cached state encodings from an SSM, addressing the degradation of state souping and vanilla SSMs on long, crowded contexts. The authors report substantial long-context improvements across single- and multi-hop document QA, needle-in-a-haystack retrieval, variable tracking, and tool use, plus a sparse variant with near-constant encoding cost relative to context length. They also present theoretical arguments on why souping works and when it may fail.