scrapy — protego
Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego._urlpattern._URLPattern._prepare_pattern_for_regex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks creates exponential backtracking. After protego.Protego.parse processes a crafted robots.txt file, protego.Protego.can_fetch can spend an attacker-controlled period matching a near-miss URL and deny service to the crawler. The vulnerable path is src/protego/_urlpattern.py in the _URLPattern match logic. This issue is fixed in version 0.6.2.
Available for technologies you monitor inside Seraph CTI.
Surfaced automatically when a documented actor is linked to this CVE.
Shown when this vendor/product is relevant to your industry.
Campaign and indicator context, where Seraph analysts have published it.
Declare your technology stack inside Seraph CTI and get notified the moment a vulnerability matches it.
