Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 3267a30

Browse files
committed
Close #13585: add contextlib.ExitStack to replace the ill-fated contextlib.nested API
1 parent 6e49ac2 commit 3267a30

5 files changed

Lines changed: 539 additions & 6 deletions

File tree

Doc/library/contextlib.rst

Lines changed: 278 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,11 @@ This module provides utilities for common tasks involving the :keyword:`with`
1212
statement. For more information see also :ref:`typecontextmanager` and
1313
:ref:`context-managers`.
1414

15-
Functions provided:
1615

16+
Utilities
17+
---------
18+
19+
Functions and classes provided:
1720

1821
.. decorator:: contextmanager
1922

@@ -168,6 +171,280 @@ Functions provided:
168171
.. versionadded:: 3.2
169172

170173

174+
.. class:: ExitStack()
175+
176+
A context manager that is designed to make it easy to programmatically
177+
combine other context managers and cleanup functions, especially those
178+
that are optional or otherwise driven by input data.
179+
180+
For example, a set of files may easily be handled in a single with
181+
statement as follows::
182+
183+
with ExitStack() as stack:
184+
files = [stack.enter_context(open(fname)) for fname in filenames]
185+
# All opened files will automatically be closed at the end of
186+
# the with statement, even if attempts to open files later
187+
# in the list throw an exception
188+
189+
Each instance maintains a stack of registered callbacks that are called in
190+
reverse order when the instance is closed (either explicitly or implicitly
191+
at the end of a ``with`` statement). Note that callbacks are *not* invoked
192+
implicitly when the context stack instance is garbage collected.
193+
194+
This stack model is used so that context managers that acquire their
195+
resources in their ``__init__`` method (such as file objects) can be
196+
handled correctly.
197+
198+
Since registered callbacks are invoked in the reverse order of
199+
registration, this ends up behaving as if multiple nested ``with``
200+
statements had been used with the registered set of callbacks. This even
201+
extends to exception handling - if an inner callback suppresses or replaces
202+
an exception, then outer callbacks will be passed arguments based on that
203+
updated state.
204+
205+
This is a relatively low level API that takes care of the details of
206+
correctly unwinding the stack of exit callbacks. It provides a suitable
207+
foundation for higher level context managers that manipulate the exit
208+
stack in application specific ways.
209+
210+
.. method:: enter_context(cm)
211+
212+
Enters a new context manager and adds its :meth:`__exit__` method to
213+
the callback stack. The return value is the result of the context
214+
manager's own :meth:`__enter__` method.
215+
216+
These context managers may suppress exceptions just as they normally
217+
would if used directly as part of a ``with`` statement.
218+
219+
.. method:: push(exit)
220+
221+
Adds a context manager's :meth:`__exit__` method to the callback stack.
222+
223+
As ``__enter__`` is *not* invoked, this method can be used to cover
224+
part of an :meth:`__enter__` implementation with a context manager's own
225+
:meth:`__exit__` method.
226+
227+
If passed an object that is not a context manager, this method assumes
228+
it is a callback with the same signature as a context manager's
229+
:meth:`__exit__` method and adds it directly to the callback stack.
230+
231+
By returning true values, these callbacks can suppress exceptions the
232+
same way context manager :meth:`__exit__` methods can.
233+
234+
The passed in object is returned from the function, allowing this
235+
method to be used is a function decorator.
236+
237+
.. method:: callback(callback, *args, **kwds)
238+
239+
Accepts an arbitrary callback function and arguments and adds it to
240+
the callback stack.
241+
242+
Unlike the other methods, callbacks added this way cannot suppress
243+
exceptions (as they are never passed the exception details).
244+
245+
The passed in callback is returned from the function, allowing this
246+
method to be used is a function decorator.
247+
248+
.. method:: pop_all()
249+
250+
Transfers the callback stack to a fresh :class:`ExitStack` instance
251+
and returns it. No callbacks are invoked by this operation - instead,
252+
they will now be invoked when the new stack is closed (either
253+
explicitly or implicitly).
254+
255+
For example, a group of files can be opened as an "all or nothing"
256+
operation as follows::
257+
258+
with ExitStack() as stack:
259+
files = [stack.enter_context(open(fname)) for fname in filenames]
260+
close_files = stack.pop_all().close
261+
# If opening any file fails, all previously opened files will be
262+
# closed automatically. If all files are opened successfully,
263+
# they will remain open even after the with statement ends.
264+
# close_files() can then be invoked explicitly to close them all
265+
266+
.. method:: close()
267+
268+
Immediately unwinds the callback stack, invoking callbacks in the
269+
reverse order of registration. For any context managers and exit
270+
callbacks registered, the arguments passed in will indicate that no
271+
exception occurred.
272+
273+
.. versionadded:: 3.3
274+
275+
276+
Examples and Recipes
277+
--------------------
278+
279+
This section describes some examples and recipes for making effective use of
280+
the tools provided by :mod:`contextlib`.
281+
282+
283+
Cleaning up in an ``__enter__`` implementation
284+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
285+
286+
As noted in the documentation of :meth:`ExitStack.push`, this
287+
method can be useful in cleaning up an already allocated resource if later
288+
steps in the :meth:`__enter__` implementation fail.
289+
290+
Here's an example of doing this for a context manager that accepts resource
291+
acquisition and release functions, along with an optional validation function,
292+
and maps them to the context management protocol::
293+
294+
from contextlib import contextmanager, ExitStack
295+
296+
class ResourceManager(object):
297+
298+
def __init__(self, acquire_resource, release_resource, check_resource_ok=None):
299+
self.acquire_resource = acquire_resource
300+
self.release_resource = release_resource
301+
if check_resource_ok is None:
302+
def check_resource_ok(resource):
303+
return True
304+
self.check_resource_ok = check_resource_ok
305+
306+
@contextmanager
307+
def _cleanup_on_error(self):
308+
with ExitStack() as stack:
309+
stack.push(self)
310+
yield
311+
# The validation check passed and didn't raise an exception
312+
# Accordingly, we want to keep the resource, and pass it
313+
# back to our caller
314+
stack.pop_all()
315+
316+
def __enter__(self):
317+
resource = self.acquire_resource()
318+
with self._cleanup_on_error():
319+
if not self.check_resource_ok(resource):
320+
msg = "Failed validation for {!r}"
321+
raise RuntimeError(msg.format(resource))
322+
return resource
323+
324+
def __exit__(self, *exc_details):
325+
# We don't need to duplicate any of our resource release logic
326+
self.release_resource()
327+
328+
329+
Replacing any use of ``try-finally`` and flag variables
330+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
331+
332+
A pattern you will sometimes see is a ``try-finally`` statement with a flag
333+
variable to indicate whether or not the body of the ``finally`` clause should
334+
be executed. In its simplest form (that can't already be handled just by
335+
using an ``except`` clause instead), it looks something like this::
336+
337+
cleanup_needed = True
338+
try:
339+
result = perform_operation()
340+
if result:
341+
cleanup_needed = False
342+
finally:
343+
if cleanup_needed:
344+
cleanup_resources()
345+
346+
As with any ``try`` statement based code, this can cause problems for
347+
development and review, because the setup code and the cleanup code can end
348+
up being separated by arbitrarily long sections of code.
349+
350+
:class:`ExitStack` makes it possible to instead register a callback for
351+
execution at the end of a ``with`` statement, and then later decide to skip
352+
executing that callback::
353+
354+
from contextlib import ExitStack
355+
356+
with ExitStack() as stack:
357+
stack.callback(cleanup_resources)
358+
result = perform_operation()
359+
if result:
360+
stack.pop_all()
361+
362+
This allows the intended cleanup up behaviour to be made explicit up front,
363+
rather than requiring a separate flag variable.
364+
365+
If a particular application uses this pattern a lot, it can be simplified
366+
even further by means of a small helper class::
367+
368+
from contextlib import ExitStack
369+
370+
class Callback(ExitStack):
371+
def __init__(self, callback, *args, **kwds):
372+
super(Callback, self).__init__()
373+
self.callback(callback, *args, **kwds)
374+
375+
def cancel(self):
376+
self.pop_all()
377+
378+
with Callback(cleanup_resources) as cb:
379+
result = perform_operation()
380+
if result:
381+
cb.cancel()
382+
383+
If the resource cleanup isn't already neatly bundled into a standalone
384+
function, then it is still possible to use the decorator form of
385+
:meth:`ExitStack.callback` to declare the resource cleanup in
386+
advance::
387+
388+
from contextlib import ExitStack
389+
390+
with ExitStack() as stack:
391+
@stack.callback
392+
def cleanup_resources():
393+
...
394+
result = perform_operation()
395+
if result:
396+
stack.pop_all()
397+
398+
Due to the way the decorator protocol works, a callback function
399+
declared this way cannot take any parameters. Instead, any resources to
400+
be released must be accessed as closure variables
401+
402+
403+
Using a context manager as a function decorator
404+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
405+
406+
:class:`ContextDecorator` makes it possible to use a context manager in
407+
both an ordinary ``with`` statement and also as a function decorator.
408+
409+
For example, it is sometimes useful to wrap functions or groups of statements
410+
with a logger that can track the time of entry and time of exit. Rather than
411+
writing both a function decorator and a context manager for the task,
412+
inheriting from :class:`ContextDecorator` provides both capabilities in a
413+
single definition::
414+
415+
from contextlib import ContextDecorator
416+
import logging
417+
418+
logging.basicConfig(level=logging.INFO)
419+
420+
class track_entry_and_exit(ContextDecorator):
421+
def __init__(self, name):
422+
self.name = name
423+
424+
def __enter__(self):
425+
logging.info('Entering: {}'.format(name))
426+
427+
def __exit__(self, exc_type, exc, exc_tb):
428+
logging.info('Exiting: {}'.format(name))
429+
430+
Instances of this class can be used as both a context manager::
431+
432+
with track_entry_and_exit('widget loader'):
433+
print('Some time consuming activity goes here')
434+
load_widget()
435+
436+
And also as a function decorator::
437+
438+
@track_entry_and_exit('widget loader')
439+
def activity():
440+
print('Some time consuming activity goes here')
441+
load_widget()
442+
443+
Note that there is one additional limitation when using context managers
444+
as function decorators: there's no way to access the return value of
445+
:meth:`__enter__`. If that value is needed, then it is still necessary to use
446+
an explicit ``with`` statement.
447+
171448
.. seealso::
172449

173450
:pep:`0343` - The "with" statement

Doc/whatsnew/3.3.rst

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -696,6 +696,21 @@ collections classes. Aliases for ABCs are still present in the
696696
.. XXX addition of __slots__ to ABCs not recorded here: internal detail
697697
698698
699+
contextlib
700+
----------
701+
702+
:class:`~collections.ExitStack` now provides a solid foundation for
703+
programmatic manipulation of context managers and similar cleanup
704+
functionality. Unlike the previous ``contextlib.nested`` API (which was
705+
deprecated and removed), the new API is designed to work correctly
706+
regardless of whether context managers acquire their resources in
707+
their ``__init`` method (for example, file objects) or in their
708+
``__enter__`` method (for example, synchronisation objects from the
709+
:mod:`threading` module).
710+
711+
(:issue:`13585`)
712+
713+
699714
crypt
700715
-----
701716

0 commit comments

Comments
 (0)