gpu-unlock on Two RTX 3090s: Faster Prefill, No Decode Gain

· 905 words · 5 minute read

I have two RTX 3090s in the six-GPU box, the same cards from my PCIe splitter post. gpu-unlock is a Linux kernel module that removes a cap on the RTX 3090. The 3090 runs FP16, BF16 and TF32 matrix multiplies that accumulate in FP32 at half the tensor core rate, while the same chip sold as a workstation card runs them at full rate.

On my cards, vLLM with a W4A16 model (4-bit weights, 16-bit activations) got 13 to 20% faster prefill and llama.cpp got 1.3 to 1.5% faster on 2,048-token prompts. Decode showed no clear gain in either.

Cartoon: two blue graphics cards labelled 3090 lift barbells in a gym. The one at the rack labelled FP16 accumulate lifts easily. The one at the rack labelled FP32 accumulate strains, its plates chained and padlocked, with an orange tag reading half speed hanging from the lock. A robot official with a clipboard stands to one side while a person in an orange hoodie crouches beside the rack, working the chain loose with a screwdriver labelled gpu-unlock.

cuBLAS matrix multiplies 🔗

My two cards are different boards: GPU0 is an ASUS ROG Strix at a 450 W power limit, and GPU1 is a Founders Edition (FE) at 350 W. I ran cuBLAS matrix multiplies in each format on 8192-square matrices, with 30 iterations and one run per state.

Bar chart titled cuBLAS GEMM per card, TFLOPS, with grey bars for locked and blue bars for unlocked, on a scale from 0 to 160. Each of four groups has a locked and an unlocked bar for the Strix, 450 W, and for the FE, 350 W. FP16 in, FP16 accumulate, marked never capped, the control: Strix 147.4 locked and 147.4 unlocked, FE 133.7 and 134.5. FP16 in, FP32 accumulate, marked Strix +88%, FE +57%: Strix 83.0 and 155.8, FE 79.5 and 124.9. BF16 in, FP32 accumulate, marked Strix +91%, FE +61%: Strix 82.6 and 157.5, FE 80.7 and 130.0. TF32, marked Strix +99%, FE +84%: Strix 41.1 and 81.7, FE 40.1 and 73.9. Footer: 8192-square matrices, 30 iterations, one run per state, same night. Power limits: GPU0, an ASUS ROG Strix, 450 W; GPU1, a Founders Edition, 350 W.

FP16 with FP16 accumulation was never capped on the 3090. I used it as the control, and it didn’t move. The other three roughly doubled on the Strix. The FE falls well behind unlocked and is also slower on the control (133.7 vs 147.4). My guess is its lower power limit, but I didn’t log power or clocks.

vLLM at tensor parallel 2 🔗

This is the vLLM 0.28 image I used in September, at tensor parallel 2, with multi-token prediction (MTP) at k=3 and the KVarN cache. It serves Swift-1.5, a W4A16 build of Qwen3.8-27B. P2P is off (NCCL_P2P_DISABLE=1, --disable-custom-all-reduce), since the stock driver doesn’t support P2P between these cards. The launcher raises the FE to its 400 W max for vLLM, so these runs are at 450 W and 400 W.

Two bar charts side by side under the heading vLLM, tensor parallel 2, with grey bars for locked and blue for unlocked, each bar the mean of two runs per state. Left, Prefill, tok/s, faster, more so on longer prompts, on a scale from 0 to 2500: at a 10.8k prompt 2114 locked and 2394 unlocked, +13%; at 38.7k 1959 and 2278, +16%; at 76k 1743 and 2098, +20%. Right, Decode, tok/s, no gain, on its own scale from 0 to 450: at 1 concurrent stream 66 and 67, at 4 streams 245 and 235, at 8 streams 393 and 399. Footer: vLLM 0.28, Swift-1.5 (W4A16 Qwen3.8-27B), P2P off. Probe at fixed sizes, bars are the mean of two runs per state, rounded. Separate scales.

Prefill gained 280 to 355 tok/s, with bigger gains on longer prompts. Two runs of the same state never differed by more than 16. I suspect attention takes a bigger share of each step on longer prompts. Decode showed no gain, and 4 streams were about 4% slower unlocked.

Swift replay, cold time to first token locked (2 runs) unlocked (2 runs) change
p50 20.7 / 20.6 s 18.2 / 18.2 s -12%
p90 41.5 / 41.4 s 34.7 / 34.7 s -16%

The replay uses a stretch of recorded traffic from my coding agents. Cold means the prompt wasn’t cached yet. The model writes as much as it wants in the replay, so I also ran the base Qwen3.8-27B W4A16 AutoRound once per state, forcing 82,640 output tokens to keep the work equal.

base Qwen3.8-27B W4A16, forced length locked unlocked change
prefill tok/s, 10.8k prompt 2136 2402 +12.5%
prefill tok/s, 38.7k prompt 1969 2295 +16.6%
prefill tok/s, 76k prompt 1755 2109 +20.2%
cold time to first token, p50 20.4 s 18.3 s -10%
total time for the replay 682 s 673 s -1.3%

Median decode went from 45.2 to 48.3 tok/s, within the 4.0 gap between two identical September runs. Decode after 124.5k tokens went from 58.8 to 67.1 tok/s, on one run each, so I’d want to repeat it.

llama.cpp with Q4 GGUFs 🔗

I ran llama bench from llama.cpp’s official CUDA image, build b10902, across both cards with layer split, flash attention on and 5 repetitions per test. These runs used 450 W on the Strix and 350 W on the FE.

model test locked, tok/s unlocked, tok/s
Gemma-4 31B QAT, UD-Q4_K_XL pp512 1513.97 ±19.04 1514.99 ±25.74
pp2048 2320.02 ±1.10 2355.22 ±1.97
tg128 43.27 ±0.04 43.28 ±0.02
Gemma-4 26B-A4B, UD-Q4_K_XL pp512 4976.96 ±151.91 5015.35 ±149.97
pp2048 7489.03 ±50.70 7583.93 ±53.58
tg128 142.84 ±0.25 142.60 ±0.26

That’s +1.5% and +1.3% on 2,048-token prompts (pp2048) and nothing on 512-token prompts (pp512) or on generating 128 tokens (tg128).

How gpu-unlock works 🔗

According to its source, gpu-unlock runs NVIDIA’s signed Booter firmware on the card’s SEC2 security processor and reuses pieces of its code as a chain of privileged register writes. Those writes open the privilege masks that guard the speed selector registers. Then the module sets the full-rate values and resets the card before the driver sees it.

Installing gpu-unlock 🔗

Run make install from a clone to build an Arch, Debian or RPM package, then add this kernel parameter:

gpu_unlock.targets=all

It’s built for the stock NVIDIA driver I run on the 3090s. The package puts the module in the initramfs so it loads before nvidia binds a card. My full kernel command line is in the build post.

The module does nothing without the parameter. After a warm reboot without it, a BF16 test measured 82.5 and 79.3 TFLOPS on the two cards, back at the locked rate.

Before you install gpu-unlock 🔗

  1. Check that the project marks your card’s generation as stable. Only the RTX 30 series is.
  2. Measure a BF16 matrix multiply with FP32 accumulation on each card, before and after, and log power and clocks while you do.
  3. Benchmark your engine and model, because a quantized model may not use the capped math.
  4. Decide whether you’re comfortable running a firmware exploit on a card you depend on.

I had help with this one. Claude ran the benchmarks on the box over SSH and helped me draft this post. I did the physical work, read all the words, checked the numbers, and rewrote anything that sounded like a chatbot, so the mistakes are mine.

Sources 🔗

linux gpu nvidia kernel llm vllm qwen self-hosted

More in the Multi-GPU builds series

Related posts