llama.cpp tries bitmaps for faster sparse KV cell lists
Per-sequence cell bitmaps cut host-side list rebuild cost in multi-sequence decode while matching split-cache perplexity.
By tensor#kv-cache
Transformers adds FP8 backend for QuantizedCache
Native PyTorch FP8 concat replaces a defensive workaround, with layout fixes aimed at future kernels.
By tensorOllama ROCm on Strix Halo returns prior answers after context overflow
On gfx1151 hardware, truncated over-window prompts can replay another request's completion instead of generating a new one.
By tensor