Frictionless with Logan Jastremski

Why Memory Is AI's Biggest Bottleneck with Vikram Sekar | EP 170

Frictionless with Logan Jastremski

Show Notes

Vikram Sekar, writer of Vik's Newsletter and co-host of Semi Doped, joins me to discuss AI agents, memory, and the hardware behind the data center buildout.

We break down why training and inference need different hardware, why copper becomes harder to use as GPUs move farther apart, and why the industry is turning to optical interconnects.

Vik explains the tradeoffs across SRAM, DRAM, high bandwidth memory, and storage. We discuss why useful work per token matters more than speed alone, how much context an agent needs, and why he sees memory as AI's biggest bottleneck.

Enjoy.

00:00 Highlights 01:18 Vikram Sekar and Semi Doped 02:55 AI agents and hardware demand 18:57 AI spending and the risk of overbuilding 25:10 Training versus inference 36:25 Why distance limits copper bandwidth 41:18 Co-packaged optics and the push beyond copper 46:04 New approaches to optical interconnects 52:24 The AI memory hierarchy 56:39 Useful work per token 57:51 How much context does an agent need 1:03:35 DRAM, HBM, and memory on logic 1:05:54 The next generation of inference chips 1:08:25 AI's biggest engineering bottleneck

Follow Vik https://x.com/vikramskr

Vik's Newsletter https://www.viksnewsletter.com/

Semi Doped https://www.semidoped.fm/

Vik's institutional research https://www.semiexponent.com/

More from Frictionless with Logan Jastremski

Oct 8, 2026

Dropping a podcast with @benfielding, Co-Founder and CEO at @gensynai

Oct 1, 2026

Armani Ferrante, Founder and CEO of Backpack, joins me to talk about tokenized stocks and the next phase of global capital markets. We discuss the different tokenization models, why redeemability and liquidity matter, and Backpack's ambition to connect traditional markets with Solana. We also get into the licensed path, Hyperliquid and Kraken, and competing with the largest exchanges. Armani explains the tokenomics of Backpack's own BP token, how growth milestones shape its allocations, and the equity-conversion option he describes for token holders who contribute to the product. We also discuss the corporate treasury and a potential future IPO. Enjoy. 00:00 Highlights 01:26 Armani Ferrante and Backpack 03:18 Why tokenized equities matter 13:05 The different models for tokenized stocks 19:33 Backpack's approach to tokenized stocks 28:58 Backpack's place in the market 31:47 Why Backpack chose the licensed path 35:25 One global liquidity layer 40:00 Hyperliquid, Kraken and derivatives 49:29 Competing with the incumbents 53:47 Why a token launch is a starting point 1:00:04 Backpack's three allocation buckets 1:02:31 The treasury lockup and a potential IPO 1:07:07 The option to convert BP tokens into equity Follow Armani https://x.com/armaniferrante Backpack https://backpack.exchange/

Sep 28, 2026

Dropping a podcast with @mdudas, Managing Partner at @6thManVentures

Sep 25, 2026

Dropping a podcast with @pequityresearch Mr. P has been doing some great work breaking down the AI buildout and following where hyperscaler capex actually goes. In this episode I wanted to walk through the full stack: logic, memory, power, and networking. We get into why memory could become the largest line item in the AI bill, what old GPU rental prices tell us about compute demand, and where value is going to accrue as the physical constraints get harder to solve. At the center of the conversation is his view that AI spending cannot grow forever. Long-term contracts might change the shape of the next memory downturn, but they don’t eliminate the cycle. And signing a 10-year contract does not mean anyone can actually see 10 years of demand. We also get into why he’s excited about optics, where NAND and HBF fit as agents use more memory, and his views on Chinese open source and the future of US model development. We discuss: Why he thinks 10-year demand visibility is bullshit

Sep 21, 2026

Mr. Bubble is world class at explaining very technical concepts in layman terms. In this episode I wanted to chat through the 0 to 1 of the chips, racks, packaging, the memory hierarchy, scaling up and scaling out. We spend a lot of time mapping computer-architecture primitives directly onto the AI factory and examining where value is actually going to accrue as packaging, HBM, and the next memory tier mature. At the center of the conversation is the belief that the bottleneck is no longer designing a faster chip, it is the physical stack from silicon to rack. While the frontier labs settle into a more mature “closed demand” phase focused on their own silicon, their own scale-up domains, and capturing the value themselves. We also get into his view on AI alignment and the future of Labs. We discuss: The 0 to 1 of chips: what actually has to be true before a die is useful Why packaging is the new Moore’s law Racks as the real unit of compute The memory hierarchy after HBM Scaling up vs scaling out, and why they get confused What the labs actually want from the supply chain AI alignment, and whether the labs are aligning the world or just themselves The future of Labs: closed frontier, custom silicon, and who gets the model Timestamps: 0:00 – Introduction & Why Hardware Is the Real AI Conversation 8:00 – 0 to 1 of the Chip 18:00 – Packaging as the New Moore’s Law 28:00 – Racks, Trays, and the Unit of Compute 38:00 – Memory Hierarchy: HBM, DRAM, and the Next Tier 50:00 – Scaling Up vs Scaling Out 1:02:00 – AI Alignment 1:10:00 – The Future of Labs 1:17:00 – Closing Thoughts and Where Value Accrues Next

See all episodes →