Issue #4751: hashlib now releases the GIL when hashing large buffers

pitrou · pitrou · commit bcd5cbe01ef4 · 2009-01-08T21:17:16.000Z
(with a hardwired threshold of 2048 bytes), allowing better parallelization
on multi-CPU systems. Contributed by Lukas Lueg (ebfe) and Victor Stinner.
diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst
@@ -35,6 +35,11 @@ to the buffer interface (normally :class:`bytes` objects) using the
 concatenation of the data fed to it so far using the :meth:`digest` or
 :meth:`hexdigest` methods.
 
+.. note::
+
+   For better multithreading performance, the Python GIL is released for
+   strings of more than 2047 bytes at object creation or on update.
+
 .. note::
 
    Feeding string objects is to :meth:`update` is not supported, as hashes work
diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py
@@ -198,6 +198,19 @@ def test_case_sha512_3(self):
           "e718483d0ce769644e2e42c7bc15b4638e1f98b13b2044285632a803afa973eb"+
           "de0ff244877ea60a4cb0432ce577c31beb009c5c2c49aa2e4eadb217ad8cc09b")
 
+    def test_gil(self):
+        # Check things work fine with an input larger than the size required
+        # for multithreaded operation (which is hardwired to 2048).
+        gil_minsize = 2048
+
+        m = hashlib.md5()
+        m.update(b'1')
+        m.update(b'#' * gil_minsize)
+        m.update(b'1')
+        self.assertEquals(m.hexdigest(), 'cb1e1a2cbc80be75e19935d621fb9b21')
+
+        m = hashlib.md5(b'x' * gil_minsize)
+        self.assertEquals(m.hexdigest(), 'cfb767f225d58469c5de3632a8803958')
 
 def test_main():
     support.run_unittest(HashLibTestCase)
diff --git a/Misc/NEWS b/Misc/NEWS
@@ -256,6 +256,10 @@ C-API
 Extension Modules
 -----------------
 
+- Issue #4751: hashlib now releases the GIL when hashing large buffers
+  (with a hardwired threshold of 2048 bytes), allowing better parallelization
+  on multi-CPU systems. Contributed by Lukas Lueg (ebfe) and Victor Stinner.
+
 - Issue #4051: Prevent conflict of UNICODE macros in cPickle.
 
 - Issue #4738: Each zlib object now has a separate lock, allowing to compress
diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c
