[Bug #792570] Under Windows, socket.read() seems to run into trouble when

asked to read tens of megabytes of data.  On my Mac, it hits MemoryErrors
when reading around 15Mb in one chunk.  The fix is to read the body in several
parts, not as one big piece.

It would be nice to fix the underlying socket.read() problem, too.

2.4 bugfix candidate.

diff --git a/Lib/SimpleXMLRPCServer.py b/Lib/SimpleXMLRPCServer.py
index ae06bda354b99a31bba7c33582f6b361d0e1f0a0..f9999f6d8c78429e020b65c095adb3eee316c0b0 100644 (file)
--- a/Lib/SimpleXMLRPCServer.py
+++ b/Lib/SimpleXMLRPCServer.py
@@ -422,8 +422,19 @@ class SimpleXMLRPCRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
         """
 
         try:
-            # get arguments
-            data = self.rfile.read(int(self.headers["content-length"]))
+            # Get arguments by reading body of request. 
+            # We read this in chunks to avoid straining 
+            # socket.read(); around the 10 or 15Mb mark, some platforms
+            # begin to have problems (bug #792570).
+            max_chunk_size = 10*1024*1024
+            size_remaining = int(self.headers["content-length"])
+            L = []
+            while size_remaining:
+                chunk_size = min(size_remaining, max_chunk_size)
+                L.append(self.rfile.read(chunk_size))
+                size_remaining -= len(L[-1])
+            data = ''.join(L)
+
             # In previous versions of SimpleXMLRPCServer, _dispatch
             # could be overridden in this class, instead of in
             # SimpleXMLRPCDispatcher. To maintain backwards compatibility,

diff --git a/Misc/NEWS b/Misc/NEWS
index c140d7c1fe6ca3fee268f143e4556bec07656b8d..bf682a22646bf1ade3612acf0754324985ccbfbe 100644 (file)
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -451,6 +451,9 @@ Library
 - Bug #1222790: in SimpleXMLRPCServer, set the reuse-address and close-on-exec 
   flags on the HTTP listening socket.
 
+- Bug #792570: SimpleXMLRPCServer had problems if the request grew too large.
+  Fixed by reading the HTTP body in chunks instead of one big socket.read().
+
 - Bug #1110478: Revert os.environ.update to do putenv again.
 
 - Bug #1103844: fix distutils.install.dump_dirs() with negated options.