Subject: Buglet in PyLong_AsLong

From: "Tim Peters" <tim_one@email.msn.com>
To: "Guido van Rossum" <guido@CNRI.Reston.VA.US>
Date: Sat, 23 May 1998 21:45:53 -0400

Guido, the overflow checking in PyLong_AsLong is off a little:

1) If the C in use sign-extends right shifts on signed longs, there's a
spurious overflow error when converting the most-negative int:

Python 1.5.1 (#0, Apr 13 1998, 20:22:04) [MSC 32 bit (Intel)] on win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> x = -1L << 31
>>> x
-2147483648L
>>> int(x)
Traceback (innermost last):
  File "<stdin>", line 1, in ?
OverflowError: long int too long to convert
>>>

2) If C does not sign-extend, some genuine overflows won't be caught.

The attached should repair both, and, because I installed a new disk and a C
compiler today, it's even been compiled this time <wink>.

Python 1.5.1 (#0, May 23 1998, 20:24:58) [MSC 32 bit (Intel)] on win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> x = -1L << 31
>>> x
-2147483648L
>>> int(x)
-2147483648
>>> int(-x)
Traceback (innermost last):
  File "<stdin>", line 1, in ?
OverflowError: long int too long to convert
>>> int(-x-1)
2147483647
>>> int(x-1)
Traceback (innermost last):
  File "<stdin>", line 1, in ?
OverflowError: long int too long to convert
>>>

end-casing-ly y'rs  - tim

diff --git a/Objects/longobject.c b/Objects/longobject.c
index becf1bc46548fa603152c52eacd55960d548b999..9657c08f74f2ac56c6c567964592257f647d4b94 100644 (file)
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -175,10 +175,11 @@ long
 PyLong_AsLong(vv)
        PyObject *vv;
 {
+       /* This version by Tim Peters */
        register PyLongObject *v;
-       long x, prev;
+       unsigned long x, prev;
        int i, sign;
-       
+
        if (vv == NULL || !PyLong_Check(vv)) {
                PyErr_BadInternalCall();
                return -1;
@@ -194,13 +195,22 @@ PyLong_AsLong(vv)
        while (--i >= 0) {
                prev = x;
                x = (x << SHIFT) + v->ob_digit[i];
-               if ((x >> SHIFT) != prev) {
-                       PyErr_SetString(PyExc_OverflowError,
-                               "long int too long to convert");
-                       return -1;
-               }
-       }
-       return x * sign;
+               if ((x >> SHIFT) != prev)
+                       goto overflow;
+       }
+       /* Haven't lost any bits, but if the sign bit is set we're in
+        * trouble *unless* this is the min negative number.  So,
+        * trouble iff sign bit set && (positive || some bit set other
+        * than the sign bit).
+        */
+       if ((long)x < 0 && (sign > 0 || (x << 1) != 0))
+               goto overflow;
+       return (long)x * sign;
+
+ overflow:
+       PyErr_SetString(PyExc_OverflowError,
+                       "long int too long to convert");
+       return -1;
 }
 
 /* Get a C long int from a long int object.