A100 vs L40S vs H100 for LLM inference: how to choose
Choose a GPU for serving language models by memory first, then interconnect, then price per token: NVIDIA's specifications, cloud prices and worked sums.
The GPUs most AI runs on. Choosing a card and getting a model to fit.
Number 2 this week: 2,396 points on Hacker News in the past 7 days and about 296 news articles a day.
Choose a GPU for serving language models by memory first, then interconnect, then price per token: NVIDIA's specifications, cloud prices and worked sums.
Install the NVIDIA Container Toolkit, prove the GPU works inside a container, start vLLM with docker run, then make it a Compose service that survives reboots.
One Compose file for vLLM, node exporter, the NVIDIA DCGM exporter, Prometheus and Grafana, with the queries and alerts for CPU, RAM, GPU and the model server.
AI & technologyNVIDIA’s GTC Berlin 2026 local AI announcements include items already available, starting supply on October 23, and releases later this month or in December.
AI & technologyNVIDIA unveiled two open models, Nemotron 3.5 Super VL and Nemotron 3.5 Bolt, on Oct 15th 2026 (U.S. time), spanning large data-center agents to compact models for PCs and edge devices.
AI & technologyMicrosoft and NVIDIA announced Windows AI PCs launching in October 2026; check regional availability, memory capacity, local-versus-cloud costs, and Microsoft Execution Containers support before buying.
At Nvidia GTC 2026, a simulated Olaf robot from Disney appeared first in Omniverse. Then a physical Olaf walked out on stage. The clip covers the handoff from simulation to live robot. Source: NVIDIA GTC 2026 keynote; Wikipedia (Nvidia GTC
Check out the videos and card news published by AI2Now.