Thus November 2024 paper and new analysis in September 2025 provide a comprehensive overview of a novel **Analog In-Memory Computing (AIMC)** architecture designed to accelerate the attention mechanism in Large Language Models (LLMs). The core technology involves using **capacitor-based gain cells** (made from emerging OSFETs like IGZO) to store the Key (K) and Value (V) projections of the KV cache directly within the memory arrays, enabling parallel, analog dot-product computation that drastically reduces the latency and energy consumed by data movement in traditional GPUs. Simulations indicate performance improvements of up to **7,000× speedup and 90,000× energy reduction** compared to NVIDIA A100 GPUs for the attention step alone, and the research introduces a **hardware-aware training methodology** to maintain accuracy despite analog non-idealities and the use of a simplified ReLU-based activation function instead of softmax. The text also notes that while major chipmakers are engaged in tangential AIMC research, this specific attention mechanism design is currently a prototype from academic institutions and faces a multi-year timeline for commercial readiness and scaling to trillion-parameter models.
Sources:
https://arxiv.org/pdf/2409.19315
https://www.nextbigfuture.com/2025/09/analog-in-memory-computing-attention-mechanism-for-fast-and-energy-efficient-large-language-models.html
