WEKA pitches shared flash KV cache to reduce LLM inference recomputation
The AMW Read
An incremental storage-infrastructure update with potentially segment-level implications for inference capacity, although the reported performance gains lack independent validation in the supplied account.
WEKA pitches shared flash KV cache to reduce LLM inference recomputation
WEKA is pitching its Augmented Memory Grid, running on NeuralMesh storage software, as a way to reuse LLM context across GPU servers. The system captures KV cache and makes it available across a data center, allowing sessions to resume without recomputing context. WEKA reports supporting seven times as many users on the same GPUs and increasing output from 200,000 to 2 million tokens in the same timeframe compared with DRAM offloading. These are company-reported results; the supplied account does not establish independent validation or specify the test conditions.
The market significance lies in inference infrastructure: WEKA is positioning storage as a way to increase useful output from existing GPUs. Its proposition shifts attention from adding hardware to retaining and sharing computation already performed. That puts the product in the execution layer of the AI stack, where storage performance can influence serving capacity. The reported gains make context reuse a potentially consequential purchasing consideration, but they do not establish equivalent improvements across models, workloads, or deployment environments.
For builders, the concrete next step is to benchmark the approach against their own DRAM-offloading setup, measuring concurrent users, token throughput, response latency, and total serving cost under matched conditions. The sevenfold user-capacity claim and tenfold token-output claim describe different measures and should be assessed separately. A purchasing decision should turn on whether those gains persist for the buyer's session patterns and infrastructure costs.