# gpu-unlock on Two RTX 3090s: Faster Prefill, No Decode Gain

> gpu-unlock lifts the half-rate cap on RTX 3090 BF16 tensor cores: matrix math 61 to 91% faster, vLLM prefill up 13 to 20%, no decode gain.

- URL: https://doug.sh/posts/gpu-unlock-rtx-3090/
- Author: Doug Calobrisi
- Date: 2026-11-11
- Tags: linux, gpu, nvidia, kernel, llm, vllm, qwen, self-hosted


I have two RTX 3090s in [the six-GPU box](/posts/six-gpus-one-x16-slot/), the same cards from
[my PCIe splitter post](/posts/pcie-splitter-rtx-3090/). [gpu-unlock](https://github.com/Zagitta/gpu-unlock)
is a Linux kernel module that removes a cap on the RTX 3090. The 3090 runs FP16, BF16 and TF32 matrix
multiplies that accumulate in FP32 at half the tensor core rate, while the same chip sold as a
workstation card runs them at full rate.

On my cards, vLLM with a W4A16 model (4-bit weights, 16-bit activations) got 13 to 20% faster prefill and
llama.cpp got 1.3 to 1.5% faster on 2,048-token prompts. Decode showed no clear gain in either.

![Cartoon: two blue graphics cards labelled 3090 lift barbells in a gym. The one at the rack labelled FP16 accumulate lifts easily. The one at the rack labelled FP32 accumulate strains, its plates chained and padlocked, with an orange tag reading half speed hanging from the lock. A robot official with a clipboard stands to one side while a person in an orange hoodie crouches beside the rack, working the chain loose with a screwdriver labelled gpu-unlock.](/images/gpu-unlock/half-speed-sign.png)

## cuBLAS matrix multiplies

My two cards are different boards: GPU0 is an ASUS ROG Strix at a 450 W power limit, and GPU1 is a
Founders Edition (FE) at 350 W. I ran cuBLAS matrix multiplies in each format on 8192-square matrices,
with 30 iterations and one run per state.

![Bar chart titled cuBLAS GEMM per card, TFLOPS, with grey bars for locked and blue bars for unlocked, on a scale from 0 to 160. Each of four groups has a locked and an unlocked bar for the Strix, 450 W, and for the FE, 350 W. FP16 in, FP16 accumulate, marked never capped, the control: Strix 147.4 locked and 147.4 unlocked, FE 133.7 and 134.5. FP16 in, FP32 accumulate, marked Strix +88%, FE +57%: Strix 83.0 and 155.8, FE 79.5 and 124.9. BF16 in, FP32 accumulate, marked Strix +91%, FE +61%: Strix 82.6 and 157.5, FE 80.7 and 130.0. TF32, marked Strix +99%, FE +84%: Strix 41.1 and 81.7, FE 40.1 and 73.9. Footer: 8192-square matrices, 30 iterations, one run per state, same night. Power limits: GPU0, an ASUS ROG Strix, 450 W; GPU1, a Founders Edition, 350 W.](/images/gpu-unlock/gemm-locked-unlocked.svg)

FP16 with FP16 accumulation was never capped on the 3090. I used it as the control, and it didn't move.
The other three roughly doubled on the Strix. The FE falls well behind unlocked and is also slower on
the control (133.7 vs 147.4). My guess is its lower power limit, but I didn't log power or clocks.

## vLLM at tensor parallel 2

This is the vLLM 0.28 image I used in September, at tensor parallel 2, with multi-token
prediction (MTP) at k=3 and the KVarN cache. It serves Swift-1.5, a W4A16 build of Qwen3.8-27B. P2P is off
(`NCCL_P2P_DISABLE=1`, `--disable-custom-all-reduce`), since [the stock driver doesn't support
P2P](/posts/pci-address-space-six-gpus/#keeping-128-gb-of-ram-and-giving-up-p2p-on-the-3090s) between these cards. The launcher
raises the FE to its 400 W max for vLLM, so these runs are at 450 W and 400 W.

![Two bar charts side by side under the heading vLLM, tensor parallel 2, with grey bars for locked and blue for unlocked, each bar the mean of two runs per state. Left, Prefill, tok/s, faster, more so on longer prompts, on a scale from 0 to 2500: at a 10.8k prompt 2114 locked and 2394 unlocked, +13%; at 38.7k 1959 and 2278, +16%; at 76k 1743 and 2098, +20%. Right, Decode, tok/s, no gain, on its own scale from 0 to 450: at 1 concurrent stream 66 and 67, at 4 streams 245 and 235, at 8 streams 393 and 399. Footer: vLLM 0.28, Swift-1.5 (W4A16 Qwen3.8-27B), P2P off. Probe at fixed sizes, bars are the mean of two runs per state, rounded. Separate scales.](/images/gpu-unlock/vllm-prefill-decode.svg)

Prefill gained 280 to 355 tok/s, with bigger gains on longer prompts. Two runs of the same state never
differed by more than 16. I suspect attention takes a bigger share of each step on longer prompts.
Decode showed no gain, and 4 streams were about 4% slower unlocked.

| Swift replay, cold time to first token | locked (2 runs) | unlocked (2 runs) | change |
|---|---|---|---|
| p50 | 20.7 / 20.6 s | 18.2 / 18.2 s | -12% |
| p90 | 41.5 / 41.4 s | 34.7 / 34.7 s | -16% |

The replay uses a stretch of recorded traffic from my coding agents. Cold means the prompt wasn't cached
yet. The model writes as much as it wants in the replay, so I also ran the base Qwen3.8-27B W4A16
AutoRound once per state, forcing 82,640 output tokens to keep the work equal.

| base Qwen3.8-27B W4A16, forced length | locked | unlocked | change |
|---|---|---|---|
| prefill tok/s, 10.8k prompt | 2136 | 2402 | +12.5% |
| prefill tok/s, 38.7k prompt | 1969 | 2295 | +16.6% |
| prefill tok/s, 76k prompt | 1755 | 2109 | +20.2% |
| cold time to first token, p50 | 20.4 s | 18.3 s | -10% |
| total time for the replay | 682 s | 673 s | -1.3% |

Median decode went from 45.2 to 48.3 tok/s, within the 4.0 gap between two identical September runs.
Decode after 124.5k tokens went from 58.8 to 67.1 tok/s, on one run each, so I'd want to repeat it.

## llama.cpp with Q4 GGUFs

I ran `llama bench` from llama.cpp's official CUDA image, build b10902, across both cards with layer
split, flash attention on and 5 repetitions per test. These runs used 450 W on the Strix and 350 W on the
FE.

| model | test | locked, tok/s | unlocked, tok/s |
|---|---|---|---|
| Gemma-4 31B QAT, UD-Q4_K_XL | pp512 | 1513.97 ±19.04 | 1514.99 ±25.74 |
| | pp2048 | 2320.02 ±1.10 | 2355.22 ±1.97 |
| | tg128 | 43.27 ±0.04 | 43.28 ±0.02 |
| Gemma-4 26B-A4B, UD-Q4_K_XL | pp512 | 4976.96 ±151.91 | 5015.35 ±149.97 |
| | pp2048 | 7489.03 ±50.70 | 7583.93 ±53.58 |
| | tg128 | 142.84 ±0.25 | 142.60 ±0.26 |

That's +1.5% and +1.3% on 2,048-token prompts (pp2048) and nothing on 512-token prompts (pp512) or on
generating 128 tokens (tg128).

## How gpu-unlock works

According to [its source](https://github.com/Zagitta/gpu-unlock/blob/main/kernel/gpu_unlock/recipes.c),
gpu-unlock runs NVIDIA's
signed Booter firmware on the card's SEC2 security processor and reuses pieces of its code as a chain of
privileged register writes. Those writes open the privilege masks that guard the speed selector
registers. Then the module sets the full-rate values and resets the card before the driver sees it.

## Installing gpu-unlock

Run `make install` from a clone to build an Arch, Debian or RPM package, then add this kernel
parameter:

```
gpu_unlock.targets=all
```

It's built for the stock NVIDIA driver I run on the 3090s. The package puts the module in the
initramfs so it loads before `nvidia` binds a card. My full kernel command line is in
[the build post](/posts/six-gpus-one-x16-slot/#kernel-command-line-bars-gpu-unlock-baco-and-acs).

The module does nothing without the parameter. After a warm reboot without it, a BF16 test measured 82.5
and 79.3 TFLOPS on the two cards, back at the locked rate.

## Before you install gpu-unlock

1. Check that the project marks your card's generation as stable. Only the RTX 30 series is.
2. Measure a BF16 matrix multiply with FP32 accumulation on each card, before and after, and log power
   and clocks while you do.
3. Benchmark your engine and model, because a quantized model may not use the capped math.
4. Decide whether you're comfortable running a firmware exploit on a card you depend on.

*I had help with this one. Claude ran the benchmarks on the box over SSH and helped me draft this post. I
did the physical work, read all the words, checked the numbers, and rewrote anything that sounded like a
chatbot, so the mistakes are mine.*

## Sources

- [gpu-unlock README](https://github.com/Zagitta/gpu-unlock), what the module does, how it installs and loads, and supported GPUs
- [gpu-unlock `recipes.c`](https://github.com/Zagitta/gpu-unlock/blob/main/kernel/gpu_unlock/recipes.c), the per-chip SEC2 Booter recipes and write chains behind the unlock