bpf: Avoid soft lockup in __htab_map_lookup_and_delete_batch()

Overview

In the Linux kernel, the following vulnerability has been resolved:

bpf: Avoid soft lockup in __htab_map_lookup_and_delete_batch()

__htab_map_lookup_and_delete_batch() has no rescheduling point. The batch count bounds how many entries are copied out, not how many buckets are visited, so one BPF_MAP_LOOKUP_BATCH call can walk the map end to end. The empty-bucket fast path is worse: it stays inside a single rcu_read_lock() / bpf_disable_instrumentation() section for any run of consecutive empty buckets.

That holds up on small maps, but it falls apart at scale. On a 144-CPU arm64 host running a CONFIG_PREEMPT_NONE kernel, periodic BPF_MAP_LOOKUP_BATCH calls against an LRU hash map with 16,777,216 buckets held a CPU inside the batch op for 77+ seconds and triggered the soft lockup watchdog.

Commit 75134f16e7dd ("bpf: Add schedule points in batch ops") fixed this same problem in the generic batch ops, but not in this htab-native path, which every htab-based hash map variant uses for its lookup[_and_delete] batch ops.

Complete that fix here. Leave the critical section after 64 consecutive empty buckets, call cond_resched_tasks_rcu_qs(), and resume at the saved bucket cursor. No locks are held at that point, and resuming from the cursor is already the function's behavior for non-empty buckets. Add the same call to the per-bucket loop after copy_to_user(), where every lock has been dropped. cond_resched_rcu() is not enough here: sleeping with bpf_prog_active elevated makes tracing programs on that CPU silently skip their invocations.

Plain cond_resched() is not enough either. It is a no-op under PREEMPT and PREEMPT_LAZY, the only models arm64 and x86 have offered since commit 7dadeaa6e851 ("sched: Further restrict the preemption modes"). It is also never a Tasks RCU quiescent state, in any model: the reschedule counts as a preemption. The walking task stays a holdout and stalls every synchronize_rcu_tasks() caller, ftrace and BPF trampoline teardown included, until the syscall returns [1]. cond_resched_tasks_rcu_qs() is the usual tool for that [2]. It reports the quiescent state at each yield and still reschedules as cond_resched() does on PREEMPT_NONE and PREEMPT_VOLUNTARY kernels.

Affected

• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < 5c8b1869cb99a32a962388848cd19604fb41263e
• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < ca5bd55fc508fbb5c08c62206d02237a5b6fc7e8
• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < 77a3b52b4455e6377e549539e757bc55eedefb51
• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < f0c09de08ec077098b7e1563f2285f1c136cd816
• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < 44f9bcb5ab2b268bc763ed3ebec83308b4972936
• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < 4e48fe9b1c77966bede0a73feb3059d7595f642d
• Linux >= 057996380a42bb64ccc04383cfa9c0ace4ea11f0 < 85136bf22404474a815fc0ed26ec0d1cbc1bc3f9
• Linux 5.6

Remediation

Refer to the linked advisories for vendor-supplied fixes and affected version ranges.