]> granicus.if.org Git - python/commit
projects / python / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: a8e7ebe) | patch
bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request...
authorMiss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Sun, 16 Jun 2019 07:07:54 +0000 (00:07 -0700)
committerGitHub <noreply@github.com>
Sun, 16 Jun 2019 07:07:54 +0000 (00:07 -0700)
commit58a1a76baefc92d9e2392a5dbf65e39e44fb8f55
tree04b3001a1bc09863fcd62a0442d0da2445ad0fbatree | snapshot
parenta8e7ebe2880f4c1d3b91d40b9730bb4032d514d0commit | diff
bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791)

Co-Authored-By: Tal Einat <taleinat+github@gmail.com>
(cherry picked from commit 8047e0e1c620f69cc21f9ca48b24bf2cdd5c3668)

Co-authored-by: Rémi Lapeyre <remi.lapeyre@henki.fr>
Lib/test/test_robotparser.py diff | blob | history
Lib/urllib/robotparser.py diff | blob | history
Misc/NEWS.d/next/Library/2019-06-11-19-34-29.bpo-35922.rxpzWr.rst [new file with mode: 0644] blob
Unnamed repository; edit this file 'description' to name the repository.