<p>Large Language Model (LLM) inference efficiency directly shapes both user experience and deployment costs.</p>
<p>In this lightning talk, we’ll show how vLLM boosts inference throughput through optimized KV cache management and parallelization, and compare it against Ollama, a popular local inference tool.</p>
<p>We’ll walk through a practical, reproducible benchmark that delivers actionable performance insights.</p>
<p>By the end, attendees will have a ready-to-use testing workflow and a clear framework for choosing the right serving stack for their needs.</p>