BPF landing pads let unwinds run cleanup for Rust panics

Yonghong Song has sent work for bpf-next that teaches the BPF verifier and runtime to honor exception cleanup landing pads when bpf_unwind() walks the call stack. It is the kernel half of support LLVM 23 already added in the compiler, and it removes the main reason a Rust BPF program still cannot treat bpf_throw()-style failure as a panic path.

Today bpf_throw() climbs to the exception boundary and discards every frame in between. A frame that holds an RCU read lock, a preemption-disabled region, or a referenced kernel pointer never runs the code that gives that resource back, so the verifier simply forbids such frames from throwing. Rust Drop glue is exactly that give-back, and there has been nowhere to run it.

The design leaves bpf_throw() alone. A new bpf_unwind() kfunc raises an unwind that consults a flat cleanup table the compiler emits: ranges of call sites paired with landing pads. As the walk rewrites each suspended frame's saved return address, a covered call resumes at its pad instead of at the next instruction. The pad runs in the owning frame, does the unlock or Drop work, and continues via bpf_unwind_resume(). Only cleanup pads that resume are supported; catch pads that stop the unwind are not.

Because the pad is an ordinary second successor of the call in the same frame, the verifier can build its entry state from the call site without re-verifying the callee, and existing resource checks at exits cover the pad like any other path. libbpf grows parsing, linking, and load paths for the cleanup section (including light skeletons), and the x86-64 and arm64 JITs build the native table and redirect return-address slots, with pointer-authentication fixups on arm64 where needed. Programs that carry a cleanup table are forced off private stacks so a pad cannot see a stale frame pointer.

C selftests hand-assemble the labels, pads, and records a frontend would emit, covering successful unwinds through locked and non-preemptible frames and the shapes the kernel refuses. The practical payoff is straightforward: BPF can unwind without leaking kernel state, which is what Rust panics inside the sandbox have been waiting on.