- inference
- llm
- training
- systems
- engineering
•
•
•
•
-
Lessons I Learnt from Building vLLM Internals
What building a small inference engine revealed about paged KV cache, scheduling, and state.
-
Context in LLMs: What Determines It, What It Costs, and What Actually Works
What determines context in LLMs, what it costs, and which approaches work.
-
Serving LLMs with vLLM on RunPod: A Complete Guide
A practical guide to serving LLMs with vLLM on RunPod.
-
GPU Fundamentals & LLM Inference Mental Models
GPU architecture, roofline analysis, memory estimation, and LLM inference mental models.
-
ML Training Optimization: FLOPs, Profiling, and Learning Strategies
FLOPs, profiling, and learning strategies for ML training optimization.