bpo-30759: Copy test_robotparser from master (#2546)

bpo-30759: Copy test_robotparser from master (#2546)

Copy Lib/test/test_robotparser.py from master to 2.7 and adapt it for
Python 2.7:

* Replace urllib.robotparser with robotparser
* Adjust HTTPServer import
* Replace io.StringIO with StringIO.StringIO
* Remove tests on crawl_delay() and request_rate() since these
  methods were added to Python 3
* Remove subTest()
* Add test_main() which explicitly lists all test cases

Patch based on the commit 4da0fd06ce49132d5c2e0525c4a31f807690a9ed
written by Berker Peksag.

Co-Authored-By: Berker Peksag <berker.peksag@gmail.com>