llama.cpp skips draft probability floor for faster MTP and DFlash2
Optional bypass of the speculative decoding p_min check lifts tokens per second by roughly 20 to 40 percent on tested hardware.
llama.cpp can optionally skip the minimum draft probability check during speculative decoding when using multi-token prediction (MTP) or DFlash2 drafts, a change that measurably raises inference throughput without altering greedy output.
praneshgo contributed the work. Default behavior is unchanged when the threshold remains at zero; the gains appear when probabilistic draft sampling is paired with a non-zero floor such as 0.75. Across eleven temperatures, twelve prompts, and three seeds, mean tokens-per-second rose 20 to 43 percent versus current master, depending on model, draft method, n_max, and temperature band.
On an RTX 5090, Qwen3.8-27B with DFlash2 (n_max 3) gained 32.5 percent for temperatures 0.1-1.0 and 43.3 percent for 0.5-1.0. Muse-Glimmer-30B with the same draft path posted about 20 to 27 percent. MTP drafts on the same 27B model delivered 28 to 37 percent at n_max 3 and still nearly 20 percent at n_max 8. A DGX Spark system saw smaller but consistent lifts of roughly 10 to 17 percent. Temperature 0 produced identical text in every configuration; higher temperatures accepted more draft tokens per target pass and therefore showed larger speedups.