Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 743453a

Browse files
Ev2genyEv2geny
authored
gh-58451: Add optional delete_on_close parameter to NamedTemporaryFile (GH-97015)
1 parent bbc7cd6 commit 743453a

5 files changed

Lines changed: 216 additions & 51 deletions

File tree

‎Doc/library/tempfile.rst‎

Lines changed: 71 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -75,20 +75,61 @@ The module defines the following user-callable items:
7575
Added *errors* parameter.
7676

7777

78-
.. function:: NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None)
79-
80-
This function operates exactly as :func:`TemporaryFile` does, except that
81-
the file is guaranteed to have a visible name in the file system (on
82-
Unix, the directory entry is not unlinked). That name can be retrieved
83-
from the :attr:`name` attribute of the returned
84-
file-like object. Whether the name can be
85-
used to open the file a second time, while the named temporary file is
86-
still open, varies across platforms (it can be so used on Unix; it cannot
87-
on Windows). If *delete* is true (the default), the file is
88-
deleted as soon as it is closed.
89-
The returned object is always a file-like object whose :attr:`!file`
90-
attribute is the underlying true file object. This file-like object can
91-
be used in a :keyword:`with` statement, just like a normal file.
78+
.. function:: NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None, delete_on_close=True)
79+
80+
This function operates exactly as :func:`TemporaryFile` does, except the
81+
following differences:
82+
83+
* This function returns a file that is guaranteed to have a visible name in
84+
the file system.
85+
* To manage the named file, it extends the parameters of
86+
:func:`TemporaryFile` with *delete* and *delete_on_close* parameters that
87+
determine whether and how the named file should be automatically deleted.
88+
89+
The returned object is always a :term:`file-like object` whose :attr:`!file`
90+
attribute is the underlying true file object. This :term:`file-like object`
91+
can be used in a :keyword:`with` statement, just like a normal file. The
92+
name of the temporary file can be retrieved from the :attr:`name` attribute
93+
of the returned file-like object. On Unix, unlike with the
94+
:func:`TemporaryFile`, the directory entry does not get unlinked immediately
95+
after the file creation.
96+
97+
If *delete* is true (the default) and *delete_on_close* is true (the
98+
default), the file is deleted as soon as it is closed. If *delete* is true
99+
and *delete_on_close* is false, the file is deleted on context manager exit
100+
only, or else when the :term:`file-like object` is finalized. Deletion is not
101+
always guaranteed in this case (see :meth:`object.__del__`). If *delete* is
102+
false, the value of *delete_on_close* is ignored.
103+
104+
Therefore to use the name of the temporary file to reopen the file after
105+
closing it, either make sure not to delete the file upon closure (set the
106+
*delete* parameter to be false) or, in case the temporary file is created in
107+
a :keyword:`with` statement, set the *delete_on_close* parameter to be false.
108+
The latter approach is recommended as it provides assistance in automatic
109+
cleaning of the temporary file upon the context manager exit.
110+
111+
Opening the temporary file again by its name while it is still open works as
112+
follows:
113+
114+
* On POSIX the file can always be opened again.
115+
* On Windows, make sure that at least one of the following conditions are
116+
fulfilled:
117+
118+
* *delete* is false
119+
* additional open shares delete access (e.g. by calling :func:`os.open`
120+
with the flag ``O_TEMPORARY``)
121+
* *delete* is true but *delete_on_close* is false. Note, that in this
122+
case the additional opens that do not share delete access (e.g.
123+
created via builtin :func:`open`) must be closed before exiting the
124+
context manager, else the :func:`os.unlink` call on context manager
125+
exit will fail with a :exc:`PermissionError`.
126+
127+
On Windows, if *delete_on_close* is false, and the file is created in a
128+
directory for which the user lacks delete access, then the :func:`os.unlink`
129+
call on exit of the context manager will fail with a :exc:`PermissionError`.
130+
This cannot happen when *delete_on_close* is true because delete access is
131+
requested by the open, which fails immediately if the requested access is not
132+
granted.
92133

93134
On POSIX (only), a process that is terminated abruptly with SIGKILL
94135
cannot automatically delete any NamedTemporaryFiles it created.
@@ -98,6 +139,9 @@ The module defines the following user-callable items:
98139
.. versionchanged:: 3.8
99140
Added *errors* parameter.
100141

142+
.. versionchanged:: 3.12
143+
Added *delete_on_close* parameter.
144+
101145

102146
.. class:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None)
103147

@@ -346,6 +390,19 @@ Here are some examples of typical usage of the :mod:`tempfile` module::
346390
>>>
347391
# file is now closed and removed
348392

393+
# create a temporary file using a context manager
394+
# close the file, use the name to open the file again
395+
>>> with tempfile.TemporaryFile(delete_on_close=False) as fp:
396+
... fp.write(b'Hello world!')
397+
... fp.close()
398+
# the file is closed, but not removed
399+
# open the file again by using its name
400+
... with open(fp.name) as f
401+
... f.read()
402+
b'Hello world!'
403+
>>>
404+
# file is now removed
405+
349406
# create a temporary directory using the context manager
350407
>>> with tempfile.TemporaryDirectory() as tmpdirname:
351408
... print('created temporary directory', tmpdirname)

‎Doc/whatsnew/3.12.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,11 @@ unicodedata
148148
* The Unicode database has been updated to version 15.0.0. (Contributed by
149149
Benjamin Peterson in :gh:`96734`).
150150

151+
tempfile
152+
--------
153+
154+
The :class:`tempfile.NamedTemporaryFile` function has a new optional parameter
155+
*delete_on_close* (Contributed by Evgeny Zorin in :gh:`58451`.)
151156

152157
Optimizations
153158
=============

‎Lib/tempfile.py‎

Lines changed: 40 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -418,42 +418,42 @@ class _TemporaryFileCloser:
418418
underlying file object, without adding a __del__ method to the
419419
temporary file."""
420420

421-
file = None # Set here since __del__ checks it
421+
cleanup_called = False
422422
close_called = False
423423

424-
def __init__(self, file, name, delete=True):
424+
def __init__(self, file, name, delete=True, delete_on_close=True):
425425
self.file = file
426426
self.name = name
427427
self.delete = delete
428+
self.delete_on_close = delete_on_close
428429

429-
# NT provides delete-on-close as a primitive, so we don't need
430-
# the wrapper to do anything special. We still use it so that
431-
# file.name is useful (i.e. not "(fdopen)") with NamedTemporaryFile.
432-
if _os.name != 'nt':
433-
# Cache the unlinker so we don't get spurious errors at
434-
# shutdown when the module-level "os" is None'd out. Note
435-
# that this must be referenced as self.unlink, because the
436-
# name TemporaryFileWrapper may also get None'd out before
437-
# __del__ is called.
438-
439-
def close(self, unlink=_os.unlink):
440-
if not self.close_called and self.file is not None:
441-
self.close_called = True
442-
try:
430+
def cleanup(self, windows=(_os.name == 'nt'), unlink=_os.unlink):
431+
if not self.cleanup_called:
432+
self.cleanup_called = True
433+
try:
434+
if not self.close_called:
435+
self.close_called = True
443436
self.file.close()
444-
finally:
445-
if self.delete:
437+
finally:
438+
# Windows provides delete-on-close as a primitive, in which
439+
# case the file was deleted by self.file.close().
440+
if self.delete and not (windows and self.delete_on_close):
441+
try:
446442
unlink(self.name)
443+
except FileNotFoundError:
444+
pass
447445

448-
# Need to ensure the file is deleted on __del__
449-
def __del__(self):
450-
self.close()
451-
452-
else:
453-
def close(self):
454-
if not self.close_called:
455-
self.close_called = True
446+
def close(self):
447+
if not self.close_called:
448+
self.close_called = True
449+
try:
456450
self.file.close()
451+
finally:
452+
if self.delete and self.delete_on_close:
453+
self.cleanup()
454+
455+
def __del__(self):
456+
self.cleanup()
457457

458458

459459
class _TemporaryFileWrapper:
@@ -464,11 +464,11 @@ class _TemporaryFileWrapper:
464464
remove the file when it is no longer needed.
465465
"""
466466

467-
def __init__(self, file, name, delete=True):
467+
def __init__(self, file, name, delete=True, delete_on_close=True):
468468
self.file = file
469469
self.name = name
470-
self.delete = delete
471-
self._closer = _TemporaryFileCloser(file, name, delete)
470+
self._closer = _TemporaryFileCloser(file, name, delete,
471+
delete_on_close)
472472

473473
def __getattr__(self, name):
474474
# Attribute lookups are delegated to the underlying file
@@ -499,7 +499,7 @@ def __enter__(self):
499499
# deleted when used in a with statement
500500
def __exit__(self, exc, value, tb):
501501
result = self.file.__exit__(exc, value, tb)
502-
self.close()
502+
self._closer.cleanup()
503503
return result
504504

505505
def close(self):
@@ -518,18 +518,21 @@ def __iter__(self):
518518
for line in self.file:
519519
yield line
520520

521-
522521
def NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None,
523522
newline=None, suffix=None, prefix=None,
524-
dir=None, delete=True, *, errors=None):
523+
dir=None, delete=True, *, errors=None,
524+
delete_on_close=True):
525525
"""Create and return a temporary file.
526526
Arguments:
527527
'prefix', 'suffix', 'dir' -- as for mkstemp.
528528
'mode' -- the mode argument to io.open (default "w+b").
529529
'buffering' -- the buffer size argument to io.open (default -1).
530530
'encoding' -- the encoding argument to io.open (default None)
531531
'newline' -- the newline argument to io.open (default None)
532-
'delete' -- whether the file is deleted on close (default True).
532+
'delete' -- whether the file is automatically deleted (default True).
533+
'delete_on_close' -- if 'delete', whether the file is deleted on close
534+
(default True) or otherwise either on context manager exit
535+
(if context manager was used) or on object finalization. .
533536
'errors' -- the errors argument to io.open (default None)
534537
The file is created as mkstemp() would do it.
535538
@@ -548,7 +551,7 @@ def NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None,
548551

549552
# Setting O_TEMPORARY in the flags causes the OS to delete
550553
# the file when it is closed. This is only supported by Windows.
551-
if _os.name == 'nt' and delete:
554+
if _os.name == 'nt' and delete and delete_on_close:
552555
flags |= _os.O_TEMPORARY
553556

554557
if "b" not in mode:
@@ -567,12 +570,13 @@ def opener(*args):
567570
raw = getattr(file, 'buffer', file)
568571
raw = getattr(raw, 'raw', raw)
569572
raw.name = name
570-
return _TemporaryFileWrapper(file, name, delete)
573+
return _TemporaryFileWrapper(file, name, delete, delete_on_close)
571574
except:
572575
file.close()
573576
raise
574577
except:
575-
if name is not None and not (_os.name == 'nt' and delete):
578+
if name is not None and not (
579+
_os.name == 'nt' and delete and delete_on_close):
576580
_os.unlink(name)
577581
raise
578582

‎Lib/test/test_tempfile.py‎

Lines changed: 99 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
import stat
1212
import types
1313
import weakref
14+
import gc
1415
from unittest import mock
1516

1617
import unittest
@@ -1013,6 +1014,102 @@ def use_closed():
10131014
pass
10141015
self.assertRaises(ValueError, use_closed)
10151016

1017+
def test_context_man_not_del_on_close_if_delete_on_close_false(self):
1018+
# Issue gh-58451: tempfile.NamedTemporaryFile is not particulary useful
1019+
# on Windows
1020+
# A NamedTemporaryFile is NOT deleted when closed if
1021+
# delete_on_close=False, but is deleted on context manager exit
1022+
dir = tempfile.mkdtemp()
1023+
try:
1024+
with tempfile.NamedTemporaryFile(dir=dir,
1025+
delete=True,
1026+
delete_on_close=False) as f:
1027+
f.write(b'blat')
1028+
f_name = f.name
1029+
f.close()
1030+
with self.subTest():
1031+
# Testing that file is not deleted on close
1032+
self.assertTrue(os.path.exists(f.name),
1033+
f"NamedTemporaryFile {f.name!r} is incorrectly "
1034+
f"deleted on closure when delete_on_close=False")
1035+
1036+
with self.subTest():
1037+
# Testing that file is deleted on context manager exit
1038+
self.assertFalse(os.path.exists(f.name),
1039+
f"NamedTemporaryFile {f.name!r} exists "
1040+
f"after context manager exit")
1041+
1042+
finally:
1043+
os.rmdir(dir)
1044+
1045+
def test_context_man_ok_to_delete_manually(self):
1046+
# In the case of delete=True, a NamedTemporaryFile can be manually
1047+
# deleted in a with-statement context without causing an error.
1048+
dir = tempfile.mkdtemp()
1049+
try:
1050+
with tempfile.NamedTemporaryFile(dir=dir,
1051+
delete=True,
1052+
delete_on_close=False) as f:
1053+
f.write(b'blat')
1054+
f.close()
1055+
os.unlink(f.name)
1056+
1057+
finally:
1058+
os.rmdir(dir)
1059+
1060+
def test_context_man_not_del_if_delete_false(self):
1061+
# A NamedTemporaryFile is not deleted if delete = False
1062+
dir = tempfile.mkdtemp()
1063+
f_name = ""
1064+
try:
1065+
# Test that delete_on_close=True has no effect if delete=False.
1066+
with tempfile.NamedTemporaryFile(dir=dir, delete=False,
1067+
delete_on_close=True) as f:
1068+
f.write(b'blat')
1069+
f_name = f.name
1070+
self.assertTrue(os.path.exists(f.name),
1071+
f"NamedTemporaryFile {f.name!r} exists after close")
1072+
finally:
1073+
os.unlink(f_name)
1074+
os.rmdir(dir)
1075+
1076+
def test_del_by_finalizer(self):
1077+
# A NamedTemporaryFile is deleted when finalized in the case of
1078+
# delete=True, delete_on_close=False, and no with-statement is used.
1079+
def my_func(dir):
1080+
f = tempfile.NamedTemporaryFile(dir=dir, delete=True,
1081+
delete_on_close=False)
1082+
tmp_name = f.name
1083+
f.write(b'blat')
1084+
# Testing extreme case, where the file is not explicitly closed
1085+
# f.close()
1086+
return tmp_name
1087+
# Make sure that the garbage collector has finalized the file object.
1088+
gc.collect()
1089+
dir = tempfile.mkdtemp()
1090+
try:
1091+
tmp_name = my_func(dir)
1092+
self.assertFalse(os.path.exists(tmp_name),
1093+
f"NamedTemporaryFile {tmp_name!r} "
1094+
f"exists after finalizer ")
1095+
finally:
1096+
os.rmdir(dir)
1097+
1098+
def test_correct_finalizer_work_if_already_deleted(self):
1099+
# There should be no error in the case of delete=True,
1100+
# delete_on_close=False, no with-statement is used, and the file is
1101+
# deleted manually.
1102+
def my_func(dir)->str:
1103+
f = tempfile.NamedTemporaryFile(dir=dir, delete=True,
1104+
delete_on_close=False)
1105+
tmp_name = f.name
1106+
f.write(b'blat')
1107+
f.close()
1108+
os.unlink(tmp_name)
1109+
return tmp_name
1110+
# Make sure that the garbage collector has finalized the file object.
1111+
gc.collect()
1112+
10161113
def test_bad_mode(self):
10171114
dir = tempfile.mkdtemp()
10181115
self.addCleanup(os_helper.rmtree, dir)
@@ -1081,7 +1178,8 @@ def test_iobase_interface(self):
10811178
missing_attrs = iobase_attrs - spooledtempfile_attrs
10821179
self.assertFalse(
10831180
missing_attrs,
1084-
'SpooledTemporaryFile missing attributes from IOBase/BufferedIOBase/TextIOBase'
1181+
'SpooledTemporaryFile missing attributes from '
1182+
'IOBase/BufferedIOBase/TextIOBase'
10851183
)
10861184

10871185
def test_del_on_close(self):

‎Misc/NEWS.d/next/Library/2020-09-28-04-56-04.bpo-14243.YECnxv.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
The :class:`tempfile.NamedTemporaryFile` function has a new optional parameter *delete_on_close*

0 commit comments

Comments
 (0)