vLLM vs Ollama vs llama.cpp: which one to run
Three ways to run a language model yourself, compared on platforms, model files, concurrency, defaults and security, with a short guide to choosing.
At a glanceThe saved x86, Arm and RISC-V experiment gives the same answer, different machine code, and no evidence of which processor is faster.
Not testedThe supplied run printed 120.0 on x86 and QEMU-emulated Arm and RISC-V; its saved output was reviewed, not rerun, and contains no hardware timings.
The saved run ends with the same answer on x86-64, Arm and RISC-V: 120.0. The interesting difference appears below that answer, in the instructions GCC emitted for dot(). The function-size labels are 64, 52 and 38 bytes respectively. There is no stopwatch in this experiment.
This is part one of a processor series built around a small piece of code rather than a shopping comparison. All numbers below come from the supplied experiment files. Expand an architecture to inspect its output, or open the complete original record near the end.
The C program supplies the arrays {1, 2, 3, 4, 5, 6, 7, 8} and {8, 7, 6, 5, 4, 3, 2, 1}. Its loop multiplies each pair and accumulates a floating-point sum. Here is the source exactly as supplied:
/* dot.c: the dot product, the small loop at the centre of most AI arithmetic. */
#include <stdio.h>
float dot(const float *a, const float *b, int n) {
float sum = 0.0f;
for (int i = 0; i < n; i++)
sum += a[i] * b[i];
return sum;
}
int main(void) {
float a[8] = {1, 2, 3, 4, 5, 6, 7, 8};
float b[8] = {8, 7, 6, 5, 4, 3, 2, 1};
printf("%.1f\n", dot(a, b, 8));
return 0;
}
The build script compiles this source for each target. The baseline uses -O2 -static. It runs the x86 binary directly and the other binaries through qemu-aarch64 and qemu-riscv64. The saved tool headers identify GCC 13.3.0 and QEMU AArch64 8.2.2.
These are excerpts from the saved disassembly, not simulated CPU execution. Each panel works with a keyboard and without JavaScript.
The displayed instructions range from 1 to 5 bytes. In the loop, mulss and addss are separate instructions. The baseline command does not request FMA; the vector build later adds -mfma.
## dot() on x86-64: 64 bytes
4 bytes endbr64
2 bytes test %edx,%edx
2 bytes jle 4018d8
3 bytes movslq %edx,%rdx
2 bytes xor %eax,%eax
4 bytes pxor %xmm1,%xmm1
4 bytes shl $0x2,%rdx
3 bytes nopl (%rax)
5 bytes movss (%rdi,%rax,1),%xmm0
5 bytes mulss (%rsi,%rax,1),%xmm0
4 bytes add $0x4,%rax
4 bytes addss %xmm0,%xmm1
3 bytes cmp %rax,%rdx
2 bytes jne 4018b8
3 bytes movaps %xmm1,%xmm0
1 bytes ret
5 bytes nopl 0x0(%rax,%rax,1)
4 bytes pxor %xmm1,%xmm1
3 bytes movaps %xmm1,%xmm0
1 bytes retEvery displayed instruction occupies 4 bytes. The loop has two ldr instructions and an fmadd combining multiplication and addition.
## dot() on Arm (AArch64): 52 bytes
4 bytes movi v0.2s, #0x0
4 bytes cmp w2, #0x0
4 bytes b.le 400820
4 bytes sbfiz x3, x2, #2, #32
4 bytes mov x2, #0x0
4 bytes nop
4 bytes ldr s2, [x0, x2]
4 bytes ldr s1, [x1, x2]
4 bytes add x2, x2, #0x4
4 bytes fmadd s0, s2, s1, s0
4 bytes cmp x3, x2
4 bytes b.ne 400808
4 bytes ret
4 bytes nop
4 bytes nop
4 bytes nopThe displayed instructions occupy either 2 or 4 bytes. The loop loads each input with flw, advances both pointers and uses fmadd.s.
## dot() on RISC-V (RV64GC): 38 bytes
4 bytes fmv.w.x fa0,zero
4 bytes blez a2,10650
2 bytes slli a2,a2,0x2
4 bytes add a5,a0,a2
4 bytes flw fa4,0(a0)
4 bytes flw fa5,0(a1)
2 bytes addi a0,a0,4
2 bytes addi a1,a1,4
4 bytes fmadd.s fa0,fa4,fa5,fa0
4 bytes bne a0,a5,1063a
2 bytes ret
2 bytes retThe size in each panel is the symbol size reported by nm -S. The script separately prints objdump lines until a blank line, so the listing can include trailing padding beyond that symbol. Do not add every displayed row and call the result the function size.
Counting the loop bodies shown gives six instructions for each target. That count is a description of this compilation, not a measure of elapsed work or a ranking of processors. Nor does the smallest function here establish which architecture produces the smallest applications.
The second group switches to -O3 -ffast-math and adds target-specific options. The saved x86 AVX2 excerpt uses ymm registers, with eight numbers at a time; Arm NEON uses v0.4s, with four. The Arm SVE excerpt instead contains whilelo predicates and predicated loads. The supplied notes leave SVE's width to the processor rather than assigning it a fixed lane count.
These are different compilation settings, not a controlled timing comparison. In particular, do not assume this one input proves identical floating-point behaviour for every possible input under the changed flags.
With GCC 13.3 and -march=rv64gcv, the supplied record reports no vector instructions for this target. It still prints 120.0. Nothing here establishes what another compiler version would emit.
## Tools
gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
aarch64-linux-gnu-gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
riscv64-linux-gnu-gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
qemu-aarch64 version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18)
## The same answer from three processors
x86-64: 120.0
Arm: 120.0
RISC-V: 120.0
## dot() on x86-64: 64 bytes
4 bytes endbr64
2 bytes test %edx,%edx
2 bytes jle 4018d8
3 bytes movslq %edx,%rdx
2 bytes xor %eax,%eax
4 bytes pxor %xmm1,%xmm1
4 bytes shl $0x2,%rdx
3 bytes nopl (%rax)
5 bytes movss (%rdi,%rax,1),%xmm0
5 bytes mulss (%rsi,%rax,1),%xmm0
4 bytes add $0x4,%rax
4 bytes addss %xmm0,%xmm1
3 bytes cmp %rax,%rdx
2 bytes jne 4018b8
3 bytes movaps %xmm1,%xmm0
1 bytes ret
5 bytes nopl 0x0(%rax,%rax,1)
4 bytes pxor %xmm1,%xmm1
3 bytes movaps %xmm1,%xmm0
1 bytes ret
## dot() on Arm (AArch64): 52 bytes
4 bytes movi v0.2s, #0x0
4 bytes cmp w2, #0x0
4 bytes b.le 400820
4 bytes sbfiz x3, x2, #2, #32
4 bytes mov x2, #0x0
4 bytes nop
4 bytes ldr s2, [x0, x2]
4 bytes ldr s1, [x1, x2]
4 bytes add x2, x2, #0x4
4 bytes fmadd s0, s2, s1, s0
4 bytes cmp x3, x2
4 bytes b.ne 400808
4 bytes ret
4 bytes nop
4 bytes nop
4 bytes nop
## dot() on RISC-V (RV64GC): 38 bytes
4 bytes fmv.w.x fa0,zero
4 bytes blez a2,10650
2 bytes slli a2,a2,0x2
4 bytes add a5,a0,a2
4 bytes flw fa4,0(a0)
4 bytes flw fa5,0(a1)
2 bytes addi a0,a0,4
2 bytes addi a1,a1,4
4 bytes fmadd.s fa0,fa4,fa5,fa0
4 bytes bne a0,a5,1063a
2 bytes ret
2 bytes ret
## With vector instructions switched on (-O3 -ffast-math)
x86-64 AVX2: 120.0
Arm NEON: 120.0
Arm SVE: 120.0
RISC-V V: 120.0
### x86-64 with AVX2 (256-bit, 8 numbers at a time): the instructions that do the arithmetic
vmovups (%rdi,%rax,1),%ymm4
vmulps (%rcx,%rax,1),%ymm4,%ymm0
vaddps %ymm0,%ymm1,%ymm1
vextractf128 $0x1,%ymm1,%xmm3
### Arm with NEON (128-bit, 4 numbers at a time): the instructions that do the arithmetic
movi v0.4s, #0x0
fmla v0.4s, v2.4s, v1.4s
faddp v0.4s, v0.4s, v0.4s
faddp v0.4s, v0.4s, v0.4s
### Arm with SVE (the processor decides the width): the instructions that do the arithmetic
whilelo p0.s, wzr, w2
ld1w {z2.s}, p0/z, [x0, x3, lsl #2]
ld1w {z1.s}, p0/z, [x1, x3, lsl #2]
fmla z0.s, p0/m, z1.s, z2.s
whilelo p0.s, w3, w2
### RISC-V with the V extension: the instructions that do the arithmetic
(none: this compiler produced no vector instructions for this target)
Arm and RISC-V ran under QEMU user-mode emulation on an x86 machine, according to the supplied experiment notes. Their printed answers demonstrate this sample's result under that setup. They provide no native hardware speed, energy or thermal measurements. The experiment also does not cover arbitrary inputs or prove that every instruction shown in the standalone function was executed on the sample path.
To investigate a real purchase, the missing evidence would be results from the intended workload on the actual machines. This notebook stops at the compiled code and the recorded answers; it has no fastest-processor conclusion.
This article reflects public information available when it was written. Versions, defaults and prices can change; check the linked official documentation.
Based on the supplied, unchanged experiment/output.txt and the accompanying C program and build script. The owner-provided experiment used GCC 13.3 and QEMU user-mode emulation for Arm and RISC-V. No compiler or processor experiment was rerun for this article; no timing, power or cross-machine benchmark was performed. Successful sample outputs do not establish correctness for arbitrary floating-point inputs. Article rendering checks are separate from hardware testing. Written for this site with AI assistance, from the sources above. For errors or additional information, please request a correction. Editorial policy
Found this useful? Share it, or follow for new guides.