A100 vs L40S vs H100 for LLM inference: how to choose
Choose a GPU for serving language models by memory first, then interconnect, then price per token: NVIDIA's specifications, cloud prices and worked sums.
Topic · 83 articles
Understand AI tools, build useful workflows, and check what a new release actually changes.
Page 2 of 7
Choose a GPU for serving language models by memory first, then interconnect, then price per token: NVIDIA's specifications, cloud prices and worked sums.
Three ways to run a language model yourself, compared on platforms, model files, concurrency, defaults and security, with a short guide to choosing.
vLLM runs out of memory in three different ways: at startup, under load, and in system RAM. Six checks that tell them apart, with the fix for each.
Turn a Mac mini into an always-on private AI assistant: choose the memory, install Ollama, keep it awake, reach it from your phone and add a chat interface.
Nine working habits for coding agents such as Claude Code: what to write down, how to phrase a task, how to make the agent check its work, and when to restart.
One Compose file for vLLM, node exporter, the NVIDIA DCGM exporter, Prometheus and Grafana, with the queries and alerts for CPU, RAM, GPU and the model server.
Work out GPU memory from a model's config file, turn a number of users into concurrent requests, then load-test with vllm bench serve to find the real limit.
How one vLLM server answers many people at once, which limits decide who waits, why --api-key is not enough, and a tested nginx proxy with per-user keys.
Wrap a model that ships without a chat template: read its files, write and test the template, pick an A100 layout, and launch a coding assistant for a team.
Pick a Qwen model that fits your GPU, start it with the right vLLM flags, then use thinking mode, tool calling and structured output from the OpenAI client.
Install the NVIDIA Container Toolkit, prove the GPU works inside a container, start vLLM with docker run, then make it a Compose service that survives reboots.
Z.ai sells GLM models two ways: a pay-per-token API and a coding subscription from $18 a month. Prices, credit limits, key types and the Claude Code settings.