HBM is the critical bottleneck in AI infrastructure, and HBM4 is about to change its architecture fundamentally. Every major AI accelerator, from Nvidia to OpenAI's custom silicon, runs on HBM. No alternative memory type, including LPDDR or on-chip SRAM, matches its bandwidth-per-package ratio. SK Hynix HBM3 delivers 0.16 Gb/mm2 bit density, 85% lower than DDR4 at 0.296 Gb/mm2, because TSVs eat die area. That tradeoff is the price of stacking over 1,000 signal wires between a processor and a single memory stack on a 2.5D CoWoS interposer.
The report from SemiAnalysis covers the full stack: manufacturing constraints, vendor dynamics between SK Hynix, Micron, and Samsung, and systems-level techniques like KVCache offload, disaggregated prefill-decode, and wide high-rank expert parallelism. The shoreline problem, where HBM can only occupy 2 edges of an SoC die, is treated as a first-order architectural constraint, not a footnote. Custom base dies for HBM4 are the headline structural change, enabling accelerator makers to push memory controllers off the compute die entirely.
The subscriber section addresses Samsung's viability as an HBM supplier directly, a question the rest of the industry is dancing around. It also covers one specific technology shift that could reverse the trend of ever-increasing HBM stack capacity. LPDDR plus HBM hybrid configurations, SRAM tags, compute-under-memory, and repeater PHYs all get detailed treatment. Read this if you are making decisions about AI hardware roadmaps, memory supply chains, or accelerator architecture.
[READ ORIGINAL →]