Critical fix: if cPickle on a sizeof(long)==8 box is used to read a

tim-one · tim-one · commit bfa18f711f01 · 2001-04-10T01:54:42.000Z
binary pickle, and the latter contains a pickle of a negative Python
int i written on a sizeof(long)==4 box (and whether by cPickle or
pickle.py), it's read incorrectly as i + 2**32.  The patch repairs that,
and allows test_cpickle.py (to which I added a relevant test case earlier
today) to work again on sizeof(long)==8 boxes.
There's another (at least one) sizeof(long)==8 binary pickle bug, but in
pickle.py instead.  That bug is still there, and test_pickle.py doesn't
catch it yet (try pickling and unpickling, e.g., 1 &lt;&lt; 46).
diff --git a/Modules/cPickle.c b/Modules/cPickle.c
@@ -2528,7 +2528,14 @@ calc_binint(char *s, int  x) {
         c = (unsigned char)s[i];
         l |= (long)c << (i * 8);
     }
-
+#if SIZEOF_LONG > 4
+    /* Unlike BININT1 and BININT2, BININT (more accurately BININT4)
+     * is signed, so on a box with longs bigger than 4 bytes we need
+     * to extend a BININT's sign bit to the full width.
+     */
+    if (x == 4 && l & (1L << 31))
+        l |= (~0L) << 32;
+#endif
     return l;
 }
 
