Robots::Validate versions from 0.4.3 before 0.4.10 for Perl report clients as imposter robots when User-Agent matcher state carries over from a previous request

Overview

Robots::Validate versions from 0.4.3 before 0.4.10 for Perl report clients as imposter robots when User-Agent matcher state carries over from a previous request.

Each Robots::Validate object holds one pure-Perl Aho-Corasick search machine and feeds every User-Agent into it without resetting its state. A User-Agent that ends part-way through a robot name leaves the machine mid-match, so the next User-Agent checked by the same object can complete that name. That client's address is then checked against the robot's DNS rule, which it fails.

A remote client can use this to make a site that blocks bad robots block other visitors where the prefix User-Agent string is a suffix of a robot being validated.

Note that deployments with Algorithm::AhoCorasick::XS are also affected, since it was unused due to a different bug.

Affected

• Robots-Validate >= 0.4.3 < 0.4.10

Remediation

Upgrade to Robots-Validate v0.4.10 or later.

Deployments with Algorithm::AhoCorasick::XS should upgrade to v0.4.11, which fixed a bug with using that version.