+++++++++++

Python News

+++++++++++

What's New in Python 3.6.0 alpha 4

==================================

*Release date: XXXX-XX-XX*

Core and Builtins

-----------------

- Issue #22557: Now importing already imported modules is up to 2.5 times faster.

- Issue #17596: Include <wincrypt.h> to help with Min GW building.

- Issue #27507: Add integer overflow check in bytearray.extend(). Patch by

Xiang Zhang.

- Issue #27581: Don't rely on wrapping for overflow check in

PySequence_Tuple(). Patch by Xiang Zhang.

- Issue #1621: Avoid signed integer overflow in list and tuple operations.

Patch by Xiang Zhang.

- Issue #27419: Standard __import__() no longer look up "__import__" in globals

or builtins for importing submodules or "from import". Fixed a crash if

raise a warning about unabling to resolve package from __spec__ or

__package__.

- Issue #27083: Respect the PYTHONCASEOK environment variable under Windows.

- Issue #27514: Make having too many statically nested blocks a SyntaxError

instead of SystemError.

- Issue #27366: Implemented PEP 487 (Simpler customization of class creation).

Upon subclassing, the __init_subclass__ classmethod is called on the base

class. Descriptors are initialized with __set_name__ after class creation.

Library

-------

- Issue #27661: Added tzinfo keyword argument to datetime.combine.

- Issue #27568: Prevent HTTPoxy attack (CVE-2016-1000110). Ignore the

HTTP_PROXY variable when REQUEST_METHOD environment is set, which indicates

that the script is in CGI mode.

- Issue #7063: Remove dead code from the "array" module's slice handling.

Patch by Chuck.

- Issue #27656: Do not assume sched.h defines any SCHED_* constants.

- Issue #27130: In the "zlib" module, fix handling of large buffers

(typically 4 GiB) when compressing and decompressing. Previously, inputs

were limited to 4 GiB, and compression and decompression operations did not

properly handle results of 4 GiB.

- Issue #24773: Implemented PEP 495 (Local Time Disambiguation).

- Expose the EPOLLEXCLUSIVE constant (when it is defined) in the select module.

- Issue #27567: Expose the EPOLLRDHUP and POLLRDHUP constants in the select

module.

- Issue #1621: Avoid signed int negation overflow in the "audioop" module.

- Issue #27533: Release GIL in nt._isdir

- Issue #17711: Fixed unpickling by the persistent ID with protocol 0.

Original patch by Alexandre Vassalotti.

- Issue #27522: Avoid an unintentional reference cycle in email.feedparser.

- Issue 27512: Fix a segfault when os.fspath() called a an __fspath__() method

that raised an exception. Patch by Xiang Zhang.

Tests

-----

- Issue #27472: Add test.support.unix_shell as the path to the default shell.

- Issue #27369: In test_pyexpat, avoid testing an error message detail that

changed in Expat 2.2.0.

Windows

-------

- Issue #27647: Update bundled Tcl/Tk to 8.6.6.

- Issue #27610: Adds PEP 514 metadata to Windows installer

- Issue #27469: Adds a shell extension to the launcher so that drag and drop

works correctly.

- Issue #27309: Enables proper Windows styles in python[w].exe manifest.

Build

-----

- Issue #27453: CPP invocation in configure must use CPPFLAGS. Patch by

Chi Hsuan Yen.

- Issue #27490: Do not build pgen when cross-compiling. Patch by Thomas

Perl.

- Issue #26662: Set PYTHON_FOR_GEN in configure as the Python program to be

used for file generation during the build.

What's New in Python 3.6.0 alpha 3

==================================

*Release date: 2016-07-11*

Core and Builtins

-----------------

- Issue #27473: Fixed possible integer overflow in bytes and bytearray

concatenations. Patch by Xiang Zhang.

- Issue #23034: The output of a special Python build with defined COUNT_ALLOCS,

SHOW_ALLOC_COUNT or SHOW_TRACK_COUNT macros is now off by default. It can

be re-enabled using the "-X showalloccount" option. It now outputs to stderr

instead of stdout.

- Issue #27443: __length_hint__() of bytearray iterators no longer return a

negative integer for a resized bytearray.

- Issue #27007: The fromhex() class methods of bytes and bytearray subclasses

now return an instance of corresponding subclass.

Library

-------

- Issue #26844: Fix error message for imp.find_module() to refer to 'path'

instead of 'name'. Patch by Lev Maximov.

- Issue #23804: Fix SSL zero-length recv() calls to not block and not raise

an error about unclean EOF.

- Issue #27466: Change time format returned by http.cookie.time2netscape,

confirming the netscape cookie format and making it consistent with

documentation.

- Issue #21708: Deprecated dbm.dumb behavior that differs from common dbm

behavior: creating a database in 'r' and 'w' modes and modifying a database

in 'r' mode.

- Issue #26721: Change the socketserver.StreamRequestHandler.wfile attribute

to implement BufferedIOBase. In particular, the write() method no longer

does partial writes.

- Issue #22115: Added methods trace_add, trace_remove and trace_info in the

tkinter.Variable class. They replace old methods trace_variable, trace,

trace_vdelete and trace_vinfo that use obsolete Tcl commands and might

not work in future versions of Tcl. Fixed old tracing methods:

trace_vdelete() with wrong mode no longer break tracing, trace_vinfo() now

always returns a list of pairs of strings, tracing in the "u" mode now works.

- Issue #26243: Only the level argument to zlib.compress() is keyword argument

now. The first argument is positional-only.

- Issue #27038: Expose the DirEntry type as os.DirEntry. Code patch by

Jelle Zijlstra.

- Issue #27186: Update os.fspath()/PyOS_FSPath() to check the return value of

__fspath__() to be either str or bytes.

- Issue #18726: All optional parameters of the dump(), dumps(),

load() and loads() functions and JSONEncoder and JSONDecoder class

constructors in the json module are now keyword-only.

- Issue #27319: Methods selection_set(), selection_add(), selection_remove()

and selection_toggle() of ttk.TreeView now allow passing multiple items as

multiple arguments instead of passing them as a tuple. Deprecated

undocumented ability of calling the selection() method with arguments.

- Issue #27079: Fixed curses.ascii functions isblank(), iscntrl() and ispunct().

- Issue #27294: Numerical state in the repr for Tkinter event objects is now

represented as a compination of known flags.

- Issue #27177: Match objects in the re module now support index-like objects

as group indices. Based on patches by Jeroen Demeyer and Xiang Zhang.

- Issue #26754: Some functions (compile() etc) accepted a filename argument

encoded as an iterable of integers. Now only strings and byte-like objects

are accepted.

- Issue #26536: socket.ioctl now supports SIO_LOOPBACK_FAST_PATH. Patch by

Daniel Stokes.

- Issue #27048: Prevents distutils failing on Windows when environment

variables contain non-ASCII characters

- Issue #27330: Fixed possible leaks in the ctypes module.

- Issue #27238: Got rid of bare excepts in the turtle module. Original patch

by Jelle Zijlstra.

- Issue #27122: When an exception is raised within the context being managed

by a contextlib.ExitStack() and one of the exit stack generators

catches and raises it in a chain, do not re-raise the original exception

when exiting, let the new chained one through. This avoids the PEP 479

bug described in issue25782.

- [Security] Issue #27278: Fix os.urandom() implementation using getrandom() on Linux.

Truncate size to INT_MAX and loop until we collected enough random bytes,

instead of casting a directly Py_ssize_t to int.

- Issue #16864: sqlite3.Cursor.lastrowid now supports REPLACE statement.

Initial patch by Alex LordThorsen.

- Issue #26386: Fixed ttk.TreeView selection operations with item id's

containing spaces.

- Issue #8637: Honor a pager set by the env var MANPAGER (in preference to

one set by the env var PAGER).

- [Security] Issue #22636: Avoid shell injection problems with

ctypes.util.find_library().

- Issue #16182: Fix various functions in the "readline" module to use the

locale encoding, and fix get_begidx() and get_endidx() to return code point

indexes.

- Issue #27392: Add loop.connect_accepted_socket().

Patch by Jim Fulton.

IDLE

----

- Issue #27477: IDLE search dialogs now use ttk widgets.

- Issue #27173: Add 'IDLE Modern Unix' to the built-in key sets.

Make the default key set depend on the platform.

Add tests for the changes to the config module.

- Issue #27452: make command line "idle-test> python test_help.py" work.

__file__ is relative when python is started in the file's directory.

- Issue #27452: add line counter and crc to IDLE configHandler test dump.

- Issue #27380: IDLE: add query.py with base Query dialog and ttk widgets.

Module had subclasses SectionName, ModuleName, and HelpSource, which are

used to get information from users by configdialog and file =>Load Module.

Each subclass has itw own validity checks. Using ModuleName allows users

to edit bad module names instead of starting over.

Add tests and delete the two files combined into the new one.

- Issue #27372: Test_idle no longer changes the locale.

- Issue #27365: Allow non-ascii chars in IDLE NEWS.txt, for contributor names.

- Issue #27245: IDLE: Cleanly delete custom themes and key bindings.

Previously, when IDLE was started from a console or by import, a cascade

of warnings was emitted. Patch by Serhiy Storchaka.

- Issue #24137: Run IDLE, test_idle, and htest with tkinter default root disabled.

Fix code and tests that fail with this restriction.

Fix htests to not create a second and redundant root and mainloop.

- Issue #27310: Fix IDLE.app failure to launch on OS X due to vestigial import.

C API

-----

- Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as

an iterable of integers. Now only strings and byte-like objects are accepted.

Build

-----

- Issue #27442: Expose the Android API level that python was built against, in

sysconfig.get_config_vars() as 'ANDROID_API_LEVEL'.

- Issue #27434: The interpreter that runs the cross-build, found in PATH, must

now be of the same feature version (e.g. 3.6) as the source being built.

- Issue #26930: Update Windows builds to use OpenSSL 1.0.2h.

- Issue #23968: Rename the platform directory from plat-$(MACHDEP) to

plat-$(PLATFORM_TRIPLET).

Rename the config directory (LIBPL) from config-$(LDVERSION) to

config-$(LDVERSION)-$(PLATFORM_TRIPLET).

Install the platform specifc _sysconfigdata module into the platform

directory and rename it to include the ABIFLAGS.

- Don't use largefile support for GNU/Hurd.

Tools/Demos

-----------

- Issue #27332: Fixed the type of the first argument of module-level functions

generated by Argument Clinic. Patch by Petr Viktorin.

- Issue #27418: Fixed Tools/importbench/importbench.py.

Documentation

-------------

- Issue #27285: Update documentation to reflect the deprecation of ``pyvenv``

and normalize on the term "virtual environment". Patch by Steve Piercy.

Tests

-----

- Issue #27027: Added test.support.is_android that is True when this is an

Android build.

What's New in Python 3.6.0 alpha 2

==================================

*Release date: 2016-06-13*

Core and Builtins

-----------------

- Issue #27095: Simplified MAKE_FUNCTION and removed MAKE_CLOSURE opcodes.

Patch by Demur Rumed.

- Issue #27190: Raise NotSupportedError if sqlite3 is older than 3.3.1.

Patch by Dave Sawyer.

- Issue #27286: Fixed compiling BUILD_MAP_UNPACK_WITH_CALL opcode. Calling

function with generalized unpacking (PEP 448) and conflicting keyword names

could cause undefined behavior.

- Issue #27140: Added BUILD_CONST_KEY_MAP opcode.

- Issue #27186: Add support for os.PathLike objects to open() (part of PEP 519).

- Issue #27066: Fixed SystemError if a custom opener (for open()) returns a

negative number without setting an exception.

- Issue #26983: float() now always return an instance of exact float.

The deprecation warning is emitted if __float__ returns an instance of

a strict subclass of float. In a future versions of Python this can

be an error.

- Issue #27097: Python interpreter is now about 7% faster due to optimized

instruction decoding. Based on patch by Demur Rumed.

- Issue #26647: Python interpreter now uses 16-bit wordcode instead of bytecode.

Patch by Demur Rumed.

- Issue #23275: Allow assigning to an empty target list in round brackets:

() = iterable.

- Issue #27243: Update the __aiter__ protocol: instead of returning

an awaitable that resolves to an asynchronous iterator, the asynchronous

iterator should be returned directly. Doing the former will trigger a

PendingDeprecationWarning.

Library

-------

- Comment out socket (SO_REUSEPORT) and posix (O_SHLOCK, O_EXLOCK) constants

exposed on the API which are not implemented on GNU/Hurd. They would not

work at runtime anyway.

- Issue #25455: Fixed crashes in repr of recursive ElementTree.Element and

functools.partial objects.

- Issue #27294: Improved repr for Tkinter event objects.

- Issue #20508: Improve exception message of IPv{4,6}Network.__getitem__.

Patch by Gareth Rees.

- [Security] Issue #26556: Update expat to 2.1.1, fixes CVE-2015-1283.

- [Security] Fix TLS stripping vulnerability in smtplib, CVE-2016-0772. Reported by Team

Oststrom

- Issue #21386: Implement missing IPv4Address.is_global property. It was

documented since 07a5610bae9d. Initial patch by Roger Luethi.

- Issue #27029: Removed deprecated support of universal newlines mode from

ZipFile.open().

- Issue #27030: Unknown escapes consisting of ``'\'`` and an ASCII letter in

regular expressions now are errors. The re.LOCALE flag now can be used

only with bytes patterns.

- Issue #27186: Add os.PathLike support to DirEntry (part of PEP 519).

Initial patch by Jelle Zijlstra.

- Issue #20900: distutils register command now decodes HTTP responses

correctly. Initial patch by ingrid.

- Issue #27186: Add os.PathLike support to pathlib, removing its provisional

status (part of PEP 519). Initial patch by Dusty Phillips.

- Issue #27186: Add support for os.PathLike objects to os.fsencode() and

os.fsdecode() (part of PEP 519).

- Issue #27186: Introduce os.PathLike and os.fspath() (part of PEP 519).

- A new version of typing.py provides several new classes and

features: @overload outside stubs, Reversible, DefaultDict, Text,

ContextManager, Type[], NewType(), TYPE_CHECKING, and numerous bug

fixes (note that some of the new features are not yet implemented in

mypy or other static analyzers). Also classes for PEP 492

(Awaitable, AsyncIterable, AsyncIterator) have been added (in fact

they made it into 3.5.1 but were never mentioned).

- Issue #25738: Stop http.server.BaseHTTPRequestHandler.send_error() from

sending a message body for 205 Reset Content. Also, don't send Content

header fields in responses that don't have a body. Patch by Susumu

Koshiba.

- Issue #21313: Fix the "platform" module to tolerate when sys.version

contains truncated build information.

- [Security] Issue #26839: On Linux, :func:`os.urandom` now calls ``getrandom()`` with

``GRND_NONBLOCK`` to fall back on reading ``/dev/urandom`` if the urandom

entropy pool is not initialized yet. Patch written by Colm Buckley.

- Issue #23883: Added missing APIs to __all__ to match the documented APIs

for the following modules: cgi, mailbox, mimetypes, plistlib and smtpd.

Patches by Jacek Kołodziej.

- Issue #27164: In the zlib module, allow decompressing raw Deflate streams

with a predefined zdict. Based on patch by Xiang Zhang.

- Issue #24291: Fix wsgiref.simple_server.WSGIRequestHandler to completely

write data to the client. Previously it could do partial writes and

truncate data. Also, wsgiref.handler.ServerHandler can now handle stdout

doing partial writes, but this is deprecated.

- Issue #21272: Use _sysconfigdata.py to initialize distutils.sysconfig.

- Issue #19611: :mod:`inspect` now reports the implicit ``.0`` parameters

generated by the compiler for comprehension and generator expression scopes

as if they were positional-only parameters called ``implicit0``.

Patch by Jelle Zijlstra.

- Issue #26809: Add ``__all__`` to :mod:`string`. Patch by Emanuel Barry.

- Issue #26373: subprocess.Popen.communicate now correctly ignores

BrokenPipeError when the child process dies before .communicate()

is called in more/all circumstances.

- signal, socket, and ssl module IntEnum constant name lookups now return a

consistent name for values having multiple names. Ex: signal.Signals(6)

now refers to itself as signal.SIGALRM rather than flipping between that

and signal.SIGIOT based on the interpreter's hash randomization seed.

- Issue #27167: Clarify the subprocess.CalledProcessError error message text

when the child process died due to a signal.

- Issue #25931: Don't define socketserver.Forking* names on platforms such

as Windows that do not support os.fork().

- Issue #21776: distutils.upload now correctly handles HTTPError.

Initial patch by Claudiu Popa.

- Issue #26526: Replace custom parse tree validation in the parser

module with a simple DFA validator.

- Issue #27114: Fix SSLContext._load_windows_store_certs fails with

PermissionError

- Issue #18383: Avoid creating duplicate filters when using filterwarnings

and simplefilter. Based on patch by Alex Shkop.

- Issue #23026: winreg.QueryValueEx() now return an integer for REG_QWORD type.

- Issue #26741: subprocess.Popen destructor now emits a ResourceWarning warning

if the child process is still running.

- Issue #27056: Optimize pickle.load() and pickle.loads(), up to 10% faster

to deserialize a lot of small objects.

- Issue #21271: New keyword only parameters in reset_mock call.

IDLE

----

- Issue #5124: Paste with text selected now replaces the selection on X11.

This matches how paste works on Windows, Mac, most modern Linux apps,

and ttk widgets. Original patch by Serhiy Storchaka.

- Issue #24750: Switch all scrollbars in IDLE to ttk versions.

Where needed, minimal tests are added to cover changes.

- Issue #24759: IDLE requires tk 8.5 and availability ttk widgets.

Delete now unneeded tk version tests and code for older versions.

Add test for IDLE syntax colorizoer.

- Issue #27239: idlelib.macosx.isXyzTk functions initialize as needed.

- Issue #27262: move Aqua unbinding code, which enable context menus, to maxosx.

- Issue #24759: Make clear in idlelib.idle_test.__init__ that the directory

is a private implementation of test.test_idle and tool for maintainers.

- Issue #27196: Stop 'ThemeChanged' warnings when running IDLE tests.

These persisted after other warnings were suppressed in #20567.

Apply Serhiy Storchaka's update_idletasks solution to four test files.

Record this additional advice in idle_test/README.txt

- Issue #20567: Revise idle_test/README.txt with advice about avoiding

tk warning messages from tests. Apply advice to several IDLE tests.

- Issue #24225: Update idlelib/README.txt with new file names

and event handlers.

- Issue #27156: Remove obsolete code not used by IDLE. Replacements:

1. help.txt, replaced by help.html, is out-of-date and should not be used.

Its dedicated viewer has be replaced by the html viewer in help.py.

2. ‘`import idlever; I = idlever.IDLE_VERSION`’ is the same as

‘`import sys; I = version[:version.index(' ')]`’

3. After ‘`ob = stackviewer.VariablesTreeItem(*args)`’,

‘`ob.keys() == list(ob.object.keys)`’.

4. In macosc, runningAsOSXAPP == isAquaTk; idCarbonAquaTk == isCarbonTk

- Issue #27117: Make colorizer htest and turtledemo work with dark themes.

Move code for configuring text widget colors to a new function.

- Issue #24225: Rename many `idlelib/*.py` and `idle_test/test_*.py` files.

Edit files to replace old names with new names when the old name

referred to the module rather than the class it contained.

See the issue and IDLE section in What's New in 3.6 for more.

- Issue #26673: When tk reports font size as 0, change to size 10.

Such fonts on Linux prevented the configuration dialog from opening.

- Issue #21939: Add test for IDLE's percolator.

Original patch by Saimadhav Heblikar.

- Issue #21676: Add test for IDLE's replace dialog.

Original patch by Saimadhav Heblikar.

- Issue #18410: Add test for IDLE's search dialog.

Original patch by Westley Martínez.

- Issue #21703: Add test for undo delegator. Patch mostly by

Saimadhav Heblikar .

- Issue #27044: Add ConfigDialog.remove_var_callbacks to stop memory leaks.

- Issue #23977: Add more asserts to test_delegator.

Documentation

-------------

- Issue #16484: Change the default PYTHONDOCS URL to "https:", and fix the

resulting links to use lowercase. Patch by Sean Rodman, test by Kaushik

Nadikuditi.

- Issue #24136: Document the new PEP 448 unpacking syntax of 3.5.

- Issue #22558: Add remaining doc links to source code for Python-coded modules.

Patch by Yoni Lavi.

Tests

-----

- Issue #25285: regrtest now uses subprocesses when the -j1 command line option

is used: each test file runs in a fresh child process. Before, the -j1 option

was ignored.

- Issue #25285: Tools/buildbot/test.bat script now uses -j1 by default to run

each test file in fresh child process.

Windows

-------

- Issue #27064: The py.exe launcher now defaults to Python 3.

The Windows launcher ``py.exe`` no longer prefers an installed

Python 2 version over Python 3 by default when used interactively.

Build

-----

- Issue #27229: Fix the cross-compiling pgen rule for in-tree builds. Patch

by Xavier de Gaye.

- Issue #26930: Update OS X 10.5+ 32-bit-only installer to build

and link with OpenSSL 1.0.2h.

Misc

----

- Issue #17500, and https://github.com/python/pythondotorg/issues/945: Remove

unused and outdated icons.

C API

-----

- Issue #27186: Add the PyOS_FSPath() function (part of PEP 519).

- Issue #26282: PyArg_ParseTupleAndKeywords() now supports positional-only

parameters.

Tools/Demos

-----------

- Issue #26282: Argument Clinic now supports positional-only and keyword

parameters in the same function.

What's New in Python 3.6.0 alpha 1?

===================================

Release date: 2016-05-16

Core and Builtins

-----------------

- Issue #20041: Fixed TypeError when frame.f_trace is set to None.

Patch by Xavier de Gaye.

- Issue #26168: Fixed possible refleaks in failing Py_BuildValue() with the "N"

format unit.

- Issue #26991: Fix possible refleak when creating a function with annotations.

- Issue #27039: Fixed bytearray.remove() for values greater than 127. Based on

patch by Joe Jevnik.

- Issue #23640: int.from_bytes() no longer bypasses constructors for subclasses.

- Issue #27005: Optimized the float.fromhex() class method for exact float.

It is now 2 times faster.

- Issue #18531: Single var-keyword argument of dict subtype was passed

unscathed to the C-defined function. Now it is converted to exact dict.

- Issue #26811: gc.get_objects() no longer contains a broken tuple with NULL

pointer.

- Issue #20120: Use RawConfigParser for .pypirc parsing,

removing support for interpolation unintentionally added

with move to Python 3. Behavior no longer does any

interpolation in .pypirc files, matching behavior in Python

2.7 and Setuptools 19.0.

- Issue #26249: Memory functions of the :c:func:`PyMem_Malloc` domain

(:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc allocator <pymalloc>`

rather than system :c:func:`malloc`. Applications calling

:c:func:`PyMem_Malloc` without holding the GIL can now crash: use

``PYTHONMALLOC=debug`` environment variable to validate the usage of memory

allocators in your application.

- Issue #26802: Optimize function calls only using unpacking like

``func(*tuple)`` (no other positional argument, no keyword): avoid copying

the tuple. Patch written by Joe Jevnik.

- Issue #26659: Make the builtin slice type support cycle collection.

- Issue #26718: super.__init__ no longer leaks memory if called multiple times.

NOTE: A direct call of super.__init__ is not endorsed!

- Issue #27138: Fix the doc comment for FileFinder.find_spec().

- Issue #27147: Mention PEP 420 in the importlib docs.

- Issue #25339: PYTHONIOENCODING now has priority over locale in setting the

error handler for stdin and stdout.

- Issue #26494: Fixed crash on iterating exhausting iterators.

Affected classes are generic sequence iterators, iterators of str, bytes,

bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding

views and os.scandir() iterator.

- Issue #26574: Optimize ``bytes.replace(b'', b'.')`` and

``bytearray.replace(b'', b'.')``. Patch written by Josh Snider.

- Issue #26581: If coding cookie is specified multiple times on a line in

Python source code file, only the first one is taken to account.

- Issue #19711: Add tests for reloading namespace packages.

- Issue #21099: Switch applicable importlib tests to use PEP 451 API.

- Issue #26563: Debug hooks on Python memory allocators now raise a fatal

error if functions of the :c:func:`PyMem_Malloc` family are called without

holding the GIL.

- Issue #26564: On error, the debug hooks on Python memory allocators now use

the :mod:`tracemalloc` module to get the traceback where a memory block was

allocated.

- Issue #26558: The debug hooks on Python memory allocator

:c:func:`PyObject_Malloc` now detect when functions are called without

holding the GIL.

- Issue #26516: Add :envvar:`PYTHONMALLOC` environment variable to set the

Python memory allocators and/or install debug hooks.

- Issue #26516: The :c:func:`PyMem_SetupDebugHooks` function can now also be

used on Python compiled in release mode.

- Issue #26516: The :envvar:`PYTHONMALLOCSTATS` environment variable can now

also be used on Python compiled in release mode. It now has no effect if

set to an empty string.

- Issue #26516: In debug mode, debug hooks are now also installed on Python

memory allocators when Python is configured without pymalloc.

- Issue #26464: Fix str.translate() when string is ASCII and first replacements

removes character, but next replacement uses a non-ASCII character or a

string longer than 1 character. Regression introduced in Python 3.5.0.

- Issue #22836: Ensure exception reports from PyErr_Display() and

PyErr_WriteUnraisable() are sensible even when formatting them produces

secondary errors. This affects the reports produced by

sys.__excepthook__() and when __del__() raises an exception.

- Issue #26302: Correct behavior to reject comma as a legal character for

cookie names.

- Issue #26136: Upgrade the warning when a generator raises StopIteration

from PendingDeprecationWarning to DeprecationWarning. Patch by Anish

Shah.

- Issue #26204: The compiler now ignores all constant statements: bytes, str,

int, float, complex, name constants (None, False, True), Ellipsis

and ast.Constant; not only str and int. For example, ``1.0`` is now ignored

in ``def f(): 1.0``.

- Issue #4806: Avoid masking the original TypeError exception when using star

(*) unpacking in function calls. Based on patch by Hagen Fürstenau and

Daniel Urban.

- Issue #26146: Add a new kind of AST node: ``ast.Constant``. It can be used

by external AST optimizers, but the compiler does not emit directly such

node.

- Issue #23601: Sped-up allocation of dict key objects by using Python's

small object allocator. (Contributed by Julian Taylor.)

- Issue #18018: Import raises ImportError instead of SystemError if a relative

import is attempted without a known parent package.

- Issue #25843: When compiling code, don't merge constants if they are equal

but have a different types. For example, ``f1, f2 = lambda: 1, lambda: 1.0``

is now correctly compiled to two different functions: ``f1()`` returns ``1``

(``int``) and ``f2()`` returns ``1.0`` (``float``), even if ``1`` and ``1.0``

are equal.

- Issue #26107: The format of the ``co_lnotab`` attribute of code objects

changes to support negative line number delta.

- Issue #26154: Add a new private _PyThreadState_UncheckedGet() function to get

the current Python thread state, but don't issue a fatal error if it is NULL.

This new function must be used instead of accessing directly the

_PyThreadState_Current variable. The variable is no more exposed since

Python 3.5.1 to hide the exact implementation of atomic C types, to avoid

compiler issues.

- Issue #25791: If __package__ != __spec__.parent or if neither __package__ or

__spec__ are defined then ImportWarning is raised.

- Issue #22995: [UPDATE] Comment out the one of the pickleability tests in

_PyObject_GetState() due to regressions observed in Cython-based projects.

- Issue #25961: Disallowed null characters in the type name.

- Issue #25973: Fix segfault when an invalid nonlocal statement binds a name

starting with two underscores.

- Issue #22995: Instances of extension types with a state that aren't

subclasses of list or dict and haven't implemented any pickle-related

methods (__reduce__, __reduce_ex__, __getnewargs__, __getnewargs_ex__,

or __getstate__), can no longer be pickled. Including memoryview.

- Issue #20440: Massive replacing unsafe attribute setting code with special

macro Py_SETREF.

- Issue #25766: Special method __bytes__() now works in str subclasses.

- Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size.

This allows sys.getsize() to work correctly with their subclasses with

__slots__ defined.

- Issue #25709: Fixed problem with in-place string concatenation and utf-8 cache.

- Issue #5319: New Py_FinalizeEx() API allowing Python to set an exit status

of 120 on failure to flush buffered streams.

- Issue #25485: telnetlib.Telnet is now a context manager.

- Issue #24097: Fixed crash in object.__reduce__() if slot name is freed inside

__getattr__.

- Issue #24731: Fixed crash on converting objects with special methods

__bytes__, __trunc__, and __float__ returning instances of subclasses of

bytes, int, and float to subclasses of bytes, int, and float correspondingly.

- Issue #25630: Fix a possible segfault during argument parsing in functions

that accept filesystem paths.

- Issue #23564: Fixed a partially broken sanity check in the _posixsubprocess

internals regarding how fds_to_pass were passed to the child. The bug had

no actual impact as subprocess.py already avoided it.

- Issue #25388: Fixed tokenizer crash when processing undecodable source code

with a null byte.

- Issue #25462: The hash of the key now is calculated only once in most

operations in C implementation of OrderedDict.

- Issue #22995: Default implementation of __reduce__ and __reduce_ex__ now

rejects builtin types with not defined __new__.

- Issue #24802: Avoid buffer overreads when int(), float(), compile(), exec()

and eval() are passed bytes-like objects. These objects are not

necessarily terminated by a null byte, but the functions assumed they were.

- Issue #25555: Fix parser and AST: fill lineno and col_offset of "arg" node

when compiling AST from Python objects.

- Issue #24726: Fixed a crash and leaking NULL in repr() of OrderedDict that

was mutated by direct calls of dict methods.

- Issue #25449: Iterating OrderedDict with keys with unstable hash now raises

KeyError in C implementations as well as in Python implementation.

- Issue #25395: Fixed crash when highly nested OrderedDict structures were

garbage collected.

- Issue #25401: Optimize bytes.fromhex() and bytearray.fromhex(): they are now

between 2x and 3.5x faster.

- Issue #25399: Optimize bytearray % args using the new private _PyBytesWriter

API. Formatting is now between 2.5 and 5 times faster.

- Issue #25274: sys.setrecursionlimit() now raises a RecursionError if the new

recursion limit is too low depending at the current recursion depth. Modify

also the "lower-water mark" formula to make it monotonic. This mark is used

to decide when the overflowed flag of the thread state is reset.

- Issue #24402: Fix input() to prompt to the redirected stdout when

sys.stdout.fileno() fails.

- Issue #25349: Optimize bytes % args using the new private _PyBytesWriter API.

Formatting is now up to 2 times faster.

- Issue #24806: Prevent builtin types that are not allowed to be subclassed from

being subclassed through multiple inheritance.

- Issue #25301: The UTF-8 decoder is now up to 15 times as fast for error

handlers: ``ignore``, ``replace`` and ``surrogateescape``.

- Issue #24848: Fixed a number of bugs in UTF-7 decoding of misformed data.

- Issue #25267: The UTF-8 encoder is now up to 75 times as fast for error

handlers: ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass``.

Patch co-written with Serhiy Storchaka.

- Issue #25280: Import trace messages emitted in verbose (-v) mode are no

longer formatted twice.

- Issue #25227: Optimize ASCII and latin1 encoders with the ``surrogateescape``

error handler: the encoders are now up to 3 times as fast. Initial patch

written by Serhiy Storchaka.

- Issue #25003: On Solaris 11.3 or newer, os.urandom() now uses the

getrandom() function instead of the getentropy() function. The getentropy()

function is blocking to generate very good quality entropy, os.urandom()

doesn't need such high-quality entropy.

- Issue #9232: Modify Python's grammar to allow trailing commas in the

argument list of a function declaration. For example, "def f(\*, a =

3,): pass" is now legal. Patch from Mark Dickinson.

- Issue #24965: Implement PEP 498 "Literal String Interpolation". This

allows you to embed expressions inside f-strings, which are

converted to normal strings at run time. Given x=3, then

f'value={x}' == 'value=3'. Patch by Eric V. Smith.

- Issue #26478: Fix semantic bugs when using binary operators with dictionary

views and tuples.

- Issue #26171: Fix possible integer overflow and heap corruption in

zipimporter.get_data().

- Issue #25660: Fix TAB key behaviour in REPL with readline.

- Issue #26288: Optimize PyLong_AsDouble.

- Issues #26289 and #26315: Optimize floor and modulo division for

single-digit longs. Microbenchmarks show 2-2.5x improvement. Built-in

'divmod' function is now also ~10% faster.

- Issue #25887: Raise a RuntimeError when a coroutine object is awaited

more than once.

Library

-------

- Issue #27057: Fix os.set_inheritable() on Android, ioctl() is blocked by

SELinux and fails with EACCESS. The function now falls back to fcntl().

Patch written by Michał Bednarski.

- Issue #27014: Fix infinite recursion using typing.py. Thanks to Kalle Tuure!

- Issue #27031: Removed dummy methods in Tkinter widget classes: tk_menuBar()

and tk_bindForTraversal().

- Issue #14132: Fix urllib.request redirect handling when the target only has

a query string. Original fix by Ján Janech.

- Issue #17214: The "urllib.request" module now percent-encodes non-ASCII

bytes found in redirect target URLs. Some servers send Location header

fields with non-ASCII bytes, but "http.client" requires the request target

to be ASCII-encodable, otherwise a UnicodeEncodeError is raised. Based on

patch by Christian Heimes.

- Issue #27033: The default value of the decode_data parameter for

smtpd.SMTPChannel and smtpd.SMTPServer constructors is changed to False.

- Issue #27034: Removed deprecated class asynchat.fifo.

- Issue #26870: Added readline.set_auto_history(), which can stop entries

being automatically added to the history list. Based on patch by Tyler

Crompton.

- Issue #26039: zipfile.ZipFile.open() can now be used to write data into a ZIP

file, as well as for extracting data. Patch by Thomas Kluyver.

- Issue #26892: Honor debuglevel flag in urllib.request.HTTPHandler. Patch

contributed by Chi Hsuan Yen.

- Issue #22274: In the subprocess module, allow stderr to be redirected to

stdout even when stdout is not redirected. Patch by Akira Li.

- Issue #26807: mock_open 'files' no longer error on readline at end of file.

Patch from Yolanda Robla.

- Issue #25745: Fixed leaking a userptr in curses panel destructor.

- Issue #26977: Removed unnecessary, and ignored, call to sum of squares helper

in statistics.pvariance.

- Issue #26002: Use bisect in statistics.median instead of a linear search.

Patch by Upendra Kuma.

- Issue #25974: Make use of new Decimal.as_integer_ratio() method in statistics

module. Patch by Stefan Krah.

- Issue #26996: Add secrets module as described in PEP 506.

- Issue #26881: The modulefinder module now supports extended opcode arguments.

- Issue #23815: Fixed crashes related to directly created instances of types in

_tkinter and curses.panel modules.

- Issue #17765: weakref.ref() no longer silently ignores keyword arguments.

Patch by Georg Brandl.

- Issue #26873: xmlrpc now raises ResponseError on unsupported type tags

instead of silently return incorrect result.

- Issue #26915: The __contains__ methods in the collections ABCs now check

for identity before checking equality. This better matches the behavior

of the concrete classes, allows sensible handling of NaNs, and makes it

easier to reason about container invariants.

- Issue #26711: Fixed the comparison of plistlib.Data with other types.

- Issue #24114: Fix an uninitialized variable in `ctypes.util`.

The bug only occurs on SunOS when the ctypes implementation searches

for the `crle` program. Patch by Xiang Zhang. Tested on SunOS by

Kees Bos.

- Issue #26864: In urllib.request, change the proxy bypass host checking

against no_proxy to be case-insensitive, and to not match unrelated host

names that happen to have a bypassed hostname as a suffix. Patch by Xiang

Zhang.

- Issue #24902: Print server URL on http.server startup. Initial patch by

Felix Kaiser.

- Issue #25788: fileinput.hook_encoded() now supports an "errors" argument

for passing to open. Original patch by Joseph Hackman.

- Issue #26634: recursive_repr() now sets __qualname__ of wrapper. Patch by

Xiang Zhang.

- Issue #26804: urllib.request will prefer lower_case proxy environment

variables over UPPER_CASE or Mixed_Case ones. Patch contributed by Hans-Peter

Jansen.

- Issue #26837: assertSequenceEqual() now correctly outputs non-stringified

differing items (like bytes in the -b mode). This affects assertListEqual()

and assertTupleEqual().

- Issue #26041: Remove "will be removed in Python 3.7" from deprecation

messages of platform.dist() and platform.linux_distribution().

Patch by Kumaripaba Miyurusara Athukorala.

- Issue #26822: itemgetter, attrgetter and methodcaller objects no longer

silently ignore keyword arguments.