llama.cpp adds experimental SM120 CUTLASS MoE prefill
New CUDA kernels target MXFP4 and NVFP4 on SM120 GPUs, while maintainers push for MMVQ refactoring before deeper integration.
By tensor#llama.cpp
llama.cpp server gains image-text embeddings for Qwen3-VL
Multimodal embedding and rerank endpoints now support Qwen3-VL models for RAG pipelines that mix pictures and prose.
By tensorllama.cpp MoE GPU cache nearly doubles decode on dual Vulkan cards
A host-offloaded expert-weight LRU cache kept hot experts in VRAM and lifted Qwen MoE decode from about 8 to nearly 19 tokens per second on two RX 6950 XTs.
By tensorllama.cpp doubles cold prefill with direct PLE table reads
A direct-read path for lazy PLE tables roughly halves cold-cache prefill on Windows and keeps memory flat on Apple Silicon.
By tensorllama.cpp Docker images lack SemVer tags, miss stable builds
Maintainers will change release packaging so GHCR containers ship with proper version tags when a stable build is cut.
By tensor