notes

writing about language models, CUDA, and systems engineering

Sat Aug 15 2026 00:00:00 GMT+0000 (Coordinated Universal Time)·systems

vllmini: Building a Minimal LLM Inference Engine in PyTorch

A from-scratch PyTorch inference engine for Llama, Qwen, and Mistral — covering meta-device weight loading, FlashAttention, KV caching, streaming generation, and 4-bit NF4 quantization.

~ ~ ~ ~ ~ ~ ~