vLLM quickstart: serve a model with an OpenAI-style API
Install vLLM, start an OpenAI-compatible server with one command, call it from Python, and run it in Docker. With the flags that control GPU memory.
Install vLLM, start an OpenAI-compatible server with one command, call it from Python, and run it in Docker. With the flags that control GPU memory.
Install the NVIDIA Container Toolkit, prove the GPU works inside a container, start vLLM with docker run, then make it a Compose service that survives reboots.
Pick a Qwen model that fits your GPU, start it with the right vLLM flags, then use thinking mode, tool calling and structured output from the OpenAI client.
Wrap a model that ships without a chat template: read its files, write and test the template, pick an A100 layout, and launch a coding assistant for a team.
How one vLLM server answers many people at once, which limits decide who waits, why --api-key is not enough, and a tested nginx proxy with per-user keys.
Work out GPU memory from a model's config file, turn a number of users into concurrent requests, then load-test with vllm bench serve to find the real limit.
One Compose file for vLLM, node exporter, the NVIDIA DCGM exporter, Prometheus and Grafana, with the queries and alerts for CPU, RAM, GPU and the model server.
Send your vLLM server's alerts to your phone with Alertmanager and Telegram: routing by urgency, timing, silences, and a full test without a GPU.
Point Claude Code at a model you host with vLLM: the six settings, what Claude Code really sends, what stops working, and what Anthropic does not support.
Three ways to run a language model yourself, compared on platforms, model files, concurrency, defaults and security, with a short guide to choosing.
vLLM runs out of memory in three different ways: at startup, under load, and in system RAM. Six checks that tell them apart, with the fix for each.