freenode
Kernel & Low-Level

KVM patch series stops permission upgrades from flushing guest mappings

Carlos López proposes skipping secondary-MMU zaps when host permissions only widen, cutting snapshot-time refaults on x86 and s390.

A patch series from Carlos López would stop KVM on x86 and s390 from tearing down guest page-table entries when the host only widens access to memory the guest already maps. Today those upgrades look the same to KVM as a permission narrowing, so the hypervisor drops its shadow mappings and forces the guest to fault the pages back in.

The cost shows up in live snapshots. QEMU's background snapshot write-protects guest RAM with userfaultfd, then restores access to each page once it has been saved. Because that unprotect is reported as a protection change, KVM zaps mappings that still point at the same physical pages and revoke no access. On a 64GB guest under a background snapshot, López measured a read-mostly workload dropping secondary-MMU refaults from about a million a second to about 2,500, and cross-vCPU TLB shootdowns from about 1,400 a second to near zero.

The series adds an MMU-notifier flag that marks an invalidation as a strict permission upgrade, set for mprotect calls that only add read, write, or execute rights and for userfaultfd unprotects. KVM then leaves existing entries in place and picks up the wider rights on the next fault. For the flag to be accurate, change_protection() must stop needlessly leaving writable entries read-only; several of those cases, which only affect performance, are fixed in the same series. Huge-page paths are excluded, because they do not reliably restore the write bit and can drop shared page tables.

Secondary MMUs that share the CPU page tables, such as the ARM SMMU, already hear about permission upgrades on the TLB-flush path, because a fault there does not flush stale read-only entries. KVM keeps its own tables and faults permissions in on demand, so it does not need that notice.

The change is limited to x86 and s390. The s390 half is only compile-tested; López says skipping the zap should be safe there, and other architectures could likely do the same once someone measures them.