Skip to content
Sonic
AI
Sonic
AI
Home
Discover
Ask Sonic
Projects
Use with Claude or ChatGPT
Show me around
Request source or feature
Batch Inference Explained... with Popcorn! (feat. Linda Haviv), Sonic AI
Home
/
Super Data Science: ML & AI Podcast with Jon Krohn
/
Batch Inference Explained... with Popcorn! (feat. Linda Haviv)
Super Data Science: ML & AI Podcast with Jon Krohn
Notify me
•
May 3, 2026
•
7:51
Interview
Batch Inference Explained... with Popcorn! (feat. Linda Haviv)
Linda Haviv
(guest)
Get the full transcript next time Super Data Science: ML & AI Podcast with Jon Krohn releases an episode
Summary, key quotes, top claims, and the searchable transcript — emailed automatically. No card needed.
Sign up
Executive Summary
AI infrastructure is defined by its unique challenges compared to traditional infrastructure, primarily driven by compute-heavy, GPU-dependent workloads.
A new ecosystem of specialized providers, or "neoclouds" (e.g., CoreWeave, Lightning AI), is emerging to offer optimized, bare-metal solutions for AI, competing with generalist cloud providers.
Specialized open-source frameworks like Ray (for distributing Python-native workloads) and VLLM (for inference) are critical for managing the complexity and cost of modern AI systems.
A core economic and technical challenge is maximizing GPU utilization to avoid wasted resources, a problem often described as "starving your GPUs," with inference being a key area for optimization.
Continue your research
Keep pulling the thread on Linda Haviv.
The Emergence of Neoclouds
Defining AI Infrastructure
Specialized Tooling for AI Workloads
Or ask anything across 400+ expert conversations
10
quotes
Transcript
Key Arguments
Analysis
Quotes & Entities
10
Related
Loading transcript...
Processed May 4, 2026
Daily intelligence brief →
yt-dlp + mlx-whisper + Gemini