openvswitch: fix soft lockup in the netlink flow dump

Overview

In the Linux kernel, the following vulnerability has been resolved:

openvswitch: fix soft lockup in the netlink flow dump

A production compute node carrying a few thousand datapath flows hit a soft lockup inside a single netlink flow dump and panicked.

ovs_flow_cmd_dump() calls ovs_flow_stats_get() for every flow it emits, and that releases stats->lock with spin_unlock_bh() once per CPU that has touched the flow. Every release is a local_bh_enable(), and each one runs the pending softirq backlog in the dumping thread's own context.

The skb bounds the flows one callback emits, but not the softirq work it absorbs. On a CPU that carries the box's packet load the backlog refills as fast as it drains, so the dumping thread becomes that CPU's softirq engine. It never sleeps and it has no reschedule point, so under voluntary preemption nothing can take the CPU away from it: neither the ksoftirqd the kernel woke to take the work over, nor the stopper thread the softlockup detector dispatches to refresh its timestamp.

Hold BH off across the whole callback instead, the way ctnetlink_dump_table() does, so the nested spin_unlock_bh() stop draining softirqs. The loop already runs under rcu_read_lock() and cannot sleep. What it gives up is preemption under CONFIG_PREEMPT, since a BH-off region is not preemptible outside PREEMPT_RT. The region stays short: the skb caps the flows one callback emits, and empty buckets cost no skb space but are each visited once per dump, as the cursor only moves forward. The table grows on insert and shrinks only on flush, so the walk is bounded by the largest flow count the datapath has held. The softirq backlog the callback used to absorb has no bound at all.

Affected

• Linux >= 63e7959c4b9bd6f791061c460a22d9ee32ae2240 < c995c52eba1926a5e82823fa5573a6f1b1010e5d
• Linux >= 63e7959c4b9bd6f791061c460a22d9ee32ae2240 < b872de585edff534056c65ba30970b68499fca14
• Linux >= 63e7959c4b9bd6f791061c460a22d9ee32ae2240 < 55f12552b00a34ca7ab5cf2d2b5ed02f8f06a965
• Linux >= 63e7959c4b9bd6f791061c460a22d9ee32ae2240 < 80db24f3c16fdeef85460d9d66a4aae7a9cb92b4
• Linux >= 63e7959c4b9bd6f791061c460a22d9ee32ae2240 < aaaaf87ea99b8766c9a8aa0e71aa42e6bc8a5320
• Linux 3.16

Remediation

Refer to the linked advisories for vendor-supplied fixes and affected version ranges.