Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit b9783d2

Browse files
vstinnervstinner
authored
bpo-39429: Add a new "Python Development Mode" doc page (GH-18132)
1 parent e131c97 commit b9783d2

13 files changed

Lines changed: 268 additions & 49 deletions

File tree

‎Doc/c-api/init_config.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -466,7 +466,7 @@ PyConfig
466466
467467
.. c:member:: int dev_mode
468468
469-
Development mode: see :option:`-X dev <-X>`.
469+
If non-zero, enable the :ref:`Python Development Mode <devmode>`.
470470
471471
.. c:member:: int dump_refs
472472

‎Doc/library/asyncio-dev.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ There are several ways to enable asyncio debug mode:
2525

2626
* Setting the :envvar:`PYTHONASYNCIODEBUG` environment variable to ``1``.
2727

28-
* Using the :option:`-X` ``dev`` Python command line option.
28+
* Using the :ref:`Python Development Mode <devmode>`.
2929

3030
* Passing ``debug=True`` to :func:`asyncio.run`.
3131

‎Doc/library/asyncio-eventloop.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1200,7 +1200,7 @@ Enabling debug mode
12001200

12011201
.. versionchanged:: 3.7
12021202

1203-
The new ``-X dev`` command line option can now also be used
1203+
The new :ref:`Python Development Mode <devmode>` can now also be used
12041204
to enable the debug mode.
12051205

12061206
.. seealso::

‎Doc/library/development.rst‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,10 @@ The list of modules described in this chapter is:
1818

1919
typing.rst
2020
pydoc.rst
21+
devmode.rst
2122
doctest.rst
2223
unittest.rst
2324
unittest.mock.rst
2425
unittest.mock-examples.rst
2526
2to3.rst
2627
test.rst
27-
28-
See also the Python development mode: the :option:`-X` ``dev`` option and
29-
:envvar:`PYTHONDEVMODE` environment variable.

‎Doc/library/devmode.rst‎

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
.. _devmode:
2+
3+
Python Development Mode
4+
=======================
5+
6+
.. versionadded:: 3.7
7+
8+
The Python Development Mode introduces additional runtime checks that are too
9+
expensive to be enabled by default. It should not be more verbose than the
10+
default if the code is correct; new warnings are only emitted when an issue is
11+
detected.
12+
13+
It can be enabled using the :option:`-X dev <-X>` command line option or by
14+
setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``.
15+
16+
Effects of the Python Development Mode
17+
======================================
18+
19+
Enabling the Python Development Mode is similar to the following command, but
20+
with additional effects described below::
21+
22+
PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python3 -W default -X faulthandler
23+
24+
Effects of the Python Development Mode:
25+
26+
* Add ``default`` :ref:`warning filter <describing-warning-filters>`. The
27+
following warnings are shown:
28+
29+
* :exc:`DeprecationWarning`
30+
* :exc:`ImportWarning`
31+
* :exc:`PendingDeprecationWarning`
32+
* :exc:`ResourceWarning`
33+
34+
Normally, the above warnings are filtered by the default :ref:`warning
35+
filters <describing-warning-filters>`.
36+
37+
It behaves as if the :option:`-W default <-W>` command line option is used.
38+
39+
Use the :option:`-W error <-W>` command line option or set the
40+
:envvar:`PYTHONWARNINGS` environment variable to ``error`` to treat warnings
41+
as errors.
42+
43+
* Install debug hooks on memory allocators to check for:
44+
45+
* Buffer underflow
46+
* Buffer overflow
47+
* Memory allocator API violation
48+
* Unsafe usage of the GIL
49+
50+
See the :c:func:`PyMem_SetupDebugHooks` C function.
51+
52+
It behaves as if the :envvar:`PYTHONMALLOC` environment variable is set to
53+
``debug``.
54+
55+
To enable the Python Development Mode without installing debug hooks on
56+
memory allocators, set the :envvar:`PYTHONMALLOC` environment variable to
57+
``default``.
58+
59+
* Call :func:`faulthandler.enable` at Python startup to install handlers for
60+
the :const:`SIGSEGV`, :const:`SIGFPE`, :const:`SIGABRT`, :const:`SIGBUS` and
61+
:const:`SIGILL` signals to dump the Python traceback on a crash.
62+
63+
It behaves as if the :option:`-X faulthandler <-X>` command line option is
64+
used or if the :envvar:`PYTHONFAULTHANDLER` environment variable is set to
65+
``1``.
66+
67+
* Enable :ref:`asyncio debug mode <asyncio-debug-mode>`. For example,
68+
:mod:`asyncio` checks for coroutines that were not awaited and logs them.
69+
70+
It behaves as if the :envvar:`PYTHONASYNCIODEBUG` environment variable is set
71+
to ``1``.
72+
73+
* Check the *encoding* and *errors* arguments for string encoding and decoding
74+
operations. Examples: :func:`open`, :meth:`str.encode` and
75+
:meth:`bytes.decode`.
76+
77+
By default, for best performance, the *errors* argument is only checked at
78+
the first encoding/decoding error and the *encoding* argument is sometimes
79+
ignored for empty strings.
80+
81+
* The :class:`io.IOBase` destructor logs ``close()`` exceptions.
82+
* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to
83+
``True``.
84+
85+
The Python Development Mode does not enable the :mod:`tracemalloc` module by
86+
default, because the overhead cost (to performance and memory) would be too
87+
large. Enabling the :mod:`tracemalloc` module provides additional information
88+
on the origin of some errors. For example, :exc:`ResourceWarning` logs the
89+
traceback where the resource was allocated, and a buffer overflow error logs
90+
the traceback where the memory block was allocated.
91+
92+
The Python Development Mode does not prevent the :option:`-O` command line
93+
option from removing :keyword:`assert` statements nor from setting
94+
:const:`__debug__` to ``False``.
95+
96+
.. versionchanged:: 3.8
97+
The :class:`io.IOBase` destructor now logs ``close()`` exceptions.
98+
99+
.. versionchanged:: 3.9
100+
The *encoding* and *errors* arguments are now checked for string encoding
101+
and decoding operations.
102+
103+
104+
ResourceWarning Example
105+
=======================
106+
107+
Example of a script counting the number of lines of the text file specified in
108+
the command line::
109+
110+
import sys
111+
112+
def main():
113+
fp = open(sys.argv[1])
114+
nlines = len(fp.readlines())
115+
print(nlines)
116+
# The file is closed implicitly
117+
118+
if __name__ == "__main__":
119+
main()
120+
121+
The script does not close the file explicitly. By default, Python does not emit
122+
any warning. Example using README.txt, which has 269 lines:
123+
124+
.. code-block:: shell-session
125+
126+
$ python3 script.py README.txt
127+
269
128+
129+
Enabling the Python Development Mode displays a :exc:`ResourceWarning` warning:
130+
131+
.. code-block:: shell-session
132+
133+
$ python3 -X dev script.py README.txt
134+
269
135+
script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
136+
main()
137+
ResourceWarning: Enable tracemalloc to get the object allocation traceback
138+
139+
In addition, enabling :mod:`tracemalloc` shows the line where the file was
140+
opened:
141+
142+
.. code-block:: shell-session
143+
144+
$ python3 -X dev -X tracemalloc=5 script.py README.rst
145+
269
146+
script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>
147+
main()
148+
Object allocated at (most recent call last):
149+
File "script.py", lineno 10
150+
main()
151+
File "script.py", lineno 4
152+
fp = open(sys.argv[1])
153+
154+
The fix is to close explicitly the file. Example using a context manager::
155+
156+
def main():
157+
# Close the file explicitly when exiting the with block
158+
with open(sys.argv[1]) as fp:
159+
nlines = len(fp.readlines())
160+
print(nlines)
161+
162+
Not closing a resource explicitly can leave a resource open for way longer than
163+
expected; it can cause severe issues upon exiting Python. It is bad in
164+
CPython, but it is even worse in PyPy. Closing resources explicitly makes an
165+
application more deterministic and more reliable.
166+
167+
168+
Bad file descriptor error example
169+
=================================
170+
171+
Script displaying the first line of itself::
172+
173+
import os
174+
175+
def main():
176+
fp = open(__file__)
177+
firstline = fp.readline()
178+
print(firstline.rstrip())
179+
os.close(fp.fileno())
180+
# The file is closed implicitly
181+
182+
main()
183+
184+
By default, Python does not emit any warning:
185+
186+
.. code-block:: shell-session
187+
188+
$ python3 script.py
189+
import os
190+
191+
The Python Development Mode shows a :exc:`ResourceWarning` and logs a "Bad file
192+
descriptor" error when finalizing the file object:
193+
194+
.. code-block:: shell-session
195+
196+
$ python3 script.py
197+
import os
198+
script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
199+
main()
200+
ResourceWarning: Enable tracemalloc to get the object allocation traceback
201+
Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>
202+
Traceback (most recent call last):
203+
File "script.py", line 10, in <module>
204+
main()
205+
OSError: [Errno 9] Bad file descriptor
206+
207+
``os.close(fp.fileno())`` closes the file descriptor. When the file object
208+
finalizer tries to close the file descriptor again, it fails with the ``Bad
209+
file descriptor`` error. A file descriptor must be closed only once. In the
210+
worst case scenario, closing it twice can lead to a crash (see :issue:`18748`
211+
for an example).
212+
213+
The fix is to remove the ``os.close(fp.fileno())`` line, or open the file with
214+
``closefd=False``.

‎Doc/library/exceptions.rst‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -688,6 +688,10 @@ The following exceptions are used as warning categories; see the
688688
Base class for warnings about deprecated features when those warnings are
689689
intended for other Python developers.
690690

691+
Ignored by the default warning filters, except in the ``__main__`` module
692+
(:pep:`565`). Enabling the :ref:`Python Development Mode <devmode>` shows
693+
this warning.
694+
691695

692696
.. exception:: PendingDeprecationWarning
693697

@@ -699,6 +703,9 @@ The following exceptions are used as warning categories; see the
699703
upcoming deprecation is unusual, and :exc:`DeprecationWarning`
700704
is preferred for already active deprecations.
701705

706+
Ignored by the default warning filters. Enabling the :ref:`Python
707+
Development Mode <devmode>` shows this warning.
708+
702709

703710
.. exception:: SyntaxWarning
704711

@@ -720,6 +727,9 @@ The following exceptions are used as warning categories; see the
720727

721728
Base class for warnings about probable mistakes in module imports.
722729

730+
Ignored by the default warning filters. Enabling the :ref:`Python
731+
Development Mode <devmode>` shows this warning.
732+
723733

724734
.. exception:: UnicodeWarning
725735

@@ -733,8 +743,10 @@ The following exceptions are used as warning categories; see the
733743

734744
.. exception:: ResourceWarning
735745

736-
Base class for warnings related to resource usage. Ignored by the default
737-
warning filters.
746+
Base class for warnings related to resource usage.
747+
748+
Ignored by the default warning filters. Enabling the :ref:`Python
749+
Development Mode <devmode>` shows this warning.
738750

739751
.. versionadded:: 3.2
740752

‎Doc/library/faulthandler.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,9 @@ alternatively be passed to :func:`faulthandler.enable`.
4040
The module is implemented in C, so tracebacks can be dumped on a crash or when
4141
Python is deadlocked.
4242

43+
The :ref:`Python Development Mode <devmode>` calls :func:`faulthandler.enable`
44+
at Python startup.
45+
4346

4447
Dumping the traceback
4548
---------------------

‎Doc/library/stdtypes.rst‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1559,8 +1559,8 @@ expression support in the :mod:`re` module).
15591559
list of possible encodings, see section :ref:`standard-encodings`.
15601560

15611561
By default, the *errors* argument is not checked for best performances, but
1562-
only used at the first encoding error. Enable the development mode
1563-
(:option:`-X` ``dev`` option), or use a debug build, to check *errors*.
1562+
only used at the first encoding error. Enable the :ref:`Python Development
1563+
Mode <devmode>`, or use a debug build to check *errors*.
15641564

15651565
.. versionchanged:: 3.1
15661566
Support for keyword arguments added.
@@ -2596,8 +2596,8 @@ arbitrary binary data.
25962596
list of possible encodings, see section :ref:`standard-encodings`.
25972597

25982598
By default, the *errors* argument is not checked for best performances, but
2599-
only used at the first decoding error. Enable the development mode
2600-
(:option:`-X` ``dev`` option), or use a debug build, to check *errors*.
2599+
only used at the first decoding error. Enable the :ref:`Python Development
2600+
Mode <devmode>`, or use a debug build to check *errors*.
26012601

26022602
.. note::
26032603

‎Doc/library/sys.rst‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -428,9 +428,9 @@ always available.
428428
The :term:`named tuple` *flags* exposes the status of command line
429429
flags. The attributes are read only.
430430

431-
============================= =============================
431+
============================= ================================================================
432432
attribute flag
433-
============================= =============================
433+
============================= ================================================================
434434
:const:`debug` :option:`-d`
435435
:const:`inspect` :option:`-i`
436436
:const:`interactive` :option:`-i`
@@ -444,9 +444,9 @@ always available.
444444
:const:`bytes_warning` :option:`-b`
445445
:const:`quiet` :option:`-q`
446446
:const:`hash_randomization` :option:`-R`
447-
:const:`dev_mode` :option:`-X` ``dev``
448-
:const:`utf8_mode` :option:`-X` ``utf8``
449-
============================= =============================
447+
:const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode <devmode>`)
448+
:const:`utf8_mode` :option:`-X utf8 <-X>`
449+
============================= ================================================================
450450

451451
.. versionchanged:: 3.2
452452
Added ``quiet`` attribute for the new :option:`-q` flag.
@@ -461,8 +461,9 @@ always available.
461461
Added ``isolated`` attribute for :option:`-I` ``isolated`` flag.
462462

463463
.. versionchanged:: 3.7
464-
Added ``dev_mode`` attribute for the new :option:`-X` ``dev`` flag
465-
and ``utf8_mode`` attribute for the new :option:`-X` ``utf8`` flag.
464+
Added the ``dev_mode`` attribute for the new :ref:`Python Development
465+
Mode <devmode>` and the ``utf8_mode`` attribute for the new :option:`-X`
466+
``utf8`` flag.
466467

467468

468469
.. data:: float_info

0 commit comments

Comments
 (0)