+++++++++++

Python News

+++++++++++

What's New in Python 3.2 Beta 1?

================================

*Release date: XX-Nov-2010*

Core and Builtins

-----------------

- Issue #10293: Remove obsolete field in the PyMemoryView structure,

unused undocumented value PyBUF_SHADOW, and strangely-looking code in

PyMemoryView_GetContiguous.

- Issue #6081: Add str.format_map, similar to str.format(**mapping).

- If FileIO.__init__ fails, close the file descriptor.

- Issue #10221: dict.pop(k) now has a key error message that includes the

missing key (same message d[k] returns for missing keys).

- Issue #5437: A preallocated MemoryError instance should not hold traceback

data (including local variables caught in the stack trace) alive infinitely.

- Issue #10186: Fix the SyntaxError caret when the offset is equal to the length

of the offending line.

- Issue #6011: sysconfig and distutils.sysconfig use the surrogateescape error

handler to parse the Makefile file. Avoid a UnicodeDecodeError if the source

code directory name contains a non-ASCII character and the locale encoding is

ASCII.

- Issue #10089: Add support for arbitrary -X options on the command-line.

They can be retrieved through a new attribute ``sys._xoptions``.

- Issue #4388: On Mac OS X, decode command line arguments from UTF-8, instead

of the locale encoding. If the LANG (and LC_ALL and LC_CTYPE) environment

variable is not set, the locale encoding is ISO-8859-1, whereas most programs

(including Python) expect UTF-8. Python already uses UTF-8 for the filesystem

encoding and to encode command line arguments on this OS.

- Issue #9713, #10114: Parser functions (eg. PyParser_ASTFromFile) expects

filenames encoded to the filesystem encoding with surrogateescape error

handler (to support undecodable bytes), instead of UTF-8 in strict mode.

- Issue #9997: Don't let the name "top" have special significance in scope

resolution.

- Issue #9862: Compensate for broken PIPE_BUF in AIX by hard coding

its value as the default 512 when compiling on AIX.

- Use locale encoding instead of UTF-8 to encode and decode filenames if

Py_FileSystemDefaultEncoding is not set.

- Issue #10095: fp_setreadl() doesn't reopen the file, reuse instead the file

descriptor.

- Issue #9418: Moved private string methods ``_formatter_parser`` and

``_formatter_field_name_split`` into a new ``_string`` module.

- Issue #9992: Remove PYTHONFSENCODING environment variable.

Library

-------

- Issue #10180: Pickling file objects is now explicitly forbidden, since

unpickling them produced nonsensical results.

- Issue #10311: The signal module now restores errno before returning from

its low-level signal handler. Patch by Hallvard B Furuseth.

- Issue #10282: Add a ``nntp_implementation`` attribute to NNTP objects.

- Issue #10283: Add a ``group_pattern`` argument to NNTP.list().

- Issue #10155: Add IISCGIHandler to wsgiref.handlers to support IIS

CGI environment better, and to correct unicode environment values

for WSGI 1.0.1.

- Issue #10281: nntplib now returns None for absent fields in the OVER/XOVER

response, instead of raising an exception.

- wsgiref now implements and validates PEP 3333, rather than an experimental

extension of PEP 333. (Note: earlier versions of Python 3.x may have

incorrectly validated some non-compliant applications as WSGI compliant;

if your app validates with Python <3.2b1+, but not on this version, it is

likely the case that your app was not compliant.)

- Issue #10280: NNTP.nntp_version should reflect the highest version

advertised by the server.

- Issue #10184: Touch directories only once when extracting a tarfile.

- Issue #10199: New package, ``turtledemo`` now contains selected demo

scripts that were formerly found under Demo/turtle.

- Issue #10265: Close file objects explicitly in sunau. Patch by Brian Brazil.

- Issue #10266: uu.decode didn't close in_file explicitly when it was given

as a filename. Patch by Brian Brazil.

- Issue #10110: Queue objects didn't recognize full queues when the

maxsize parameter had been reduced.

- Issue #10160: Speed up operator.attrgetter. Patch by Christos Georgiou.

- logging: Added style option to basicConfig() to allow %, {} or $-formatting.

- Issue #5729: json.dumps() now supports using a string such as '\t'

for pretty-printing multilevel objects.

- Issue #10253: FileIO leaks a file descriptor when trying to open a file

for append that isn't seekable. Patch by Brian Brazil.

- Support context manager protocol for file-like objects returned by

mailbox ``get_file()`` methods.

- Issue #10246: uu.encode didn't close file objects explicitly when filenames

were given to it. Patch by Brian Brazil.

- Issue #10198: fix duplicate header written to wave files when writeframes()

is called without data.

- Close file objects in modulefinder in a timely manner.

- Close a io.TextIOWrapper object in email.parser in a timely manner.

- Close a file object in distutils.sysconfig in a timely manner.

- Close a file object in pkgutil in a timely manner.

- Issue #10233: Close file objects in a timely manner in the tarfile module

and its test suite.

- Issue #10093: ResourceWarnings are now issued when files and sockets are

deallocated without explicit closing. These warnings are silenced by

default, except in pydebug mode.

- tarfile.py: Add support for all missing variants of the GNU sparse

extensions and create files with holes when extracting sparse members.

- Issue #10218: Return timeout status from ``Condition.wait`` in threading.

- Issue #7351: Add ``zipfile.BadZipFile`` spelling of the exception name

and deprecate the old name ``zipfile.BadZipfile``.

- Issue #5027: The standard ``xml`` namespace is now understood by

xml.sax.saxutils.XMLGenerator as being bound to

http://www.w3.org/XML/1998/namespace. Patch by Troy J. Farrell.

- Issue #5975: Add csv.unix_dialect class.

- Issue #7761: telnetlib.interact failures on Windows fixed.

- logging: Added style option to Formatter to allow %, {} or $-formatting.

- Issue #5178: Added tempfile.TemporaryDirectory class that can be used

as a context manager.

- Issue #1349106: Generator (and BytesGenerator) flatten method and Header

encode method now support a 'linesep' argument.

- Issue #5639: Add a *server_hostname* argument to ``SSLContext.wrap_socket``

in order to support the TLS SNI extension. ``HTTPSConnection`` and

``urlopen()`` also use this argument, so that HTTPS virtual hosts are now

supported.

- Issue #10166: Avoid recursion in pstats Stats.add() for many stats items.

- Issue #10163: Skip unreadable registry keys during mimetypes

initialization.

- logging: Made StreamHandler terminator configurable.

- logging: Allowed filters to be just callables.

- logging: Added tests for _logRecordClass changes.

- Issue #10092: Properly reset locale in calendar.Locale*Calendar classes.

- logging: Added _logRecordClass, getLogRecordClass, setLogRecordClass to

increase flexibility of LogRecord creation.

- Issue #5117: Case normalization was needed on ntpath.relpath(). And

fixed root directory issue on posixpath.relpath(). (Ported working fixes

from ntpath)

- Issue #1343: xml.sax.saxutils.XMLGenerator now has an option

short_empty_elements to direct it to use self-closing tags when appropriate.

- Issue #9807 (part 1): Expose the ABI flags in sys.abiflags. Add --abiflags

switch to python-config for command line access.

- Issue #6098: Don't claim DOM level 3 conformance in minidom.

- Issue #5762: Fix AttributeError raised by ``xml.dom.minidom`` when an empty

XML namespace attribute is encountered.

- Issue #2830: Add the ``html.escape()`` function, which quotes all problematic

characters by default. Deprecate ``cgi.escape()``.

- Issue #9409: Fix the regex to match all kind of filenames, for interactive

debugging in doctests.

- Issue #9183: ``datetime.timezone(datetime.timedelta(0))`` will now

return the same instance as ``datetime.timezone.utc``.

- Issue #7523: Add SOCK_CLOEXEC and SOCK_NONBLOCK to the socket module,

where supported by the system. Patch by Nikita Vetoshkin.

- Issue #10063: file:// scheme will stop accessing remote hosts via ftp

protocol. file:// urls had fallback to access remote hosts via ftp. This was

not correct, change is made to raise a URLError when a remote host is tried

to access via file:// scheme.

- Issue #1710703: Write structures for an empty ZIP archive when a ZipFile is

created in modes 'a' or 'w' and then closed without adding any files. Raise

BadZipfile (rather than IOError) when opening small non-ZIP files.

- Issue #10041: The signature of optional arguments in socket.makefile()

didn't match that of io.open(), and they also didn't get forwarded

properly to TextIOWrapper in text mode. Patch by Kai Zhu.

- Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and

urllib.request.urlopen now take optional arguments to allow for

server certificate checking, as recommended in public uses of HTTPS.

- Issue #6612: Fix site and sysconfig to catch os.getcwd() error, eg. if the

current directory was deleted. Patch written by W. Trevor King.

- Issue #3873: Speed up unpickling from file objects which have a peek()

method.

- Issue #10075: Add a session_stats() method to SSLContext objects.

- Issue #9948: Fixed problem of losing filename case information.

Extensions

----------

- Issue #9377: Use Unicode API for gethostname on Windows.

- Issue #10143: Update "os.pathconf" values.

- Issue #6518: Support context manager protcol for ossaudiodev types.

- Issue #678250: Make mmap flush a noop on ACCESS_READ and ACCESS_COPY.

- Issue #9054: Fix a crash occurring when using the pyexpat module

with expat version 2.0.1.

- Issue #5355: Provide mappings from Expat error numbers to string

descriptions and backwards, in order to actually make it possible

to analyze error codes provided by ExpatError.

- The Unicode database was updated to 6.0.0.

C-API

-----

- Issue #10288: The deprecated family of "char"-handling macros

(ISLOWER()/ISUPPER()/etc) have now been removed: use Py_ISLOWER() etc

instead.

- Issue #9778: Hash values are now always the size of pointers. A new Py_hash_t

type has been introduced.

Tools/Demos

-----------

- Issue #10117: Tools/scripts/reindent.py now accepts source files

that use encoding other than ASCII or UTF-8. Source encoding is

preserved when reindented code is written to a file.

- Issue #7287: Demo/imputil/knee.py was removed.

Tests

-----

- Issue #3699: Fix test_bigaddrspace and extend it to test bytestrings

as well as unicode strings. Initial patch by Sandro Tosi.

- Issue #10294: Remove dead code form test_unicode_file.

- Issue #10123: Don't use non-ascii filenames in test_doctest tests. Add a

new test specific to unicode (non-ascii name and filename).

Build

-----

- Issue #10268: Add a --enable-loadable-sqlite-extensions option to configure.

- Issue #8852: Allow the socket module to build on OpenSolaris.

- Drop -OPT:Olimit compiler option.

- Issue #10094: Use versioned .so files on GNU/kfreeBSD and the GNU Hurd.

- Accept Oracle Berkeley DB 5.0 and 5.1 as backend for the dbm extension.

- Issue #7473: avoid link errors when building a framework with a different

set of architectures than the one that is currently installed.

What's New in Python 3.2 Alpha 3?

=================================

*Release date: 09-Oct-2010*

Core and Builtins

-----------------

- Issue #10068: Global objects which have reference cycles with their module's

dict are now cleared again. This causes issue #7140 to appear again.

- Issue #9738: Document PyErr_SetString() and PyErr_SetFromErrnoWithFilename()

encodings.

- ast.literal_eval() can now handle negative numbers. It is also a little more

liberal in what it accepts without compromising the safety of the evaluation.

For example, 3j+4 and 3+4+5 are both accepted.

- Issue #10006: type.__abstractmethods__ now raises an AttributeError. As a

result metaclasses can now be ABCs (see #9533).

- Issue #8670: ctypes.c_wchar supports non-BMP characters with 32 bits wchar_t.

- Issue #8670: PyUnicode_AsWideChar() and PyUnicode_AsWideCharString() replace

UTF-16 surrogate pairs by single non-BMP characters for 16 bits Py_UNICODE and

32 bits wchar_t (eg. Linux in narrow build).

- Issue #10003: Allow handling of SIGBREAK on Windows. Fixes a regression

introduced by issue #9324.

- Issue #9979: Create function PyUnicode_AsWideCharString().

- Issue #7397: Mention that importlib.import_module() is probably what someone

really wants to be using in __import__'s docstring.

- Issue #8521: Allow CreateKeyEx, OpenKeyEx, and DeleteKeyEx functions of winreg

to use named arguments.

- Issue #9930: Remove bogus subtype check that was causing (e.g.)

float.__rdiv__(2.0, 3) to return NotImplemented instead of the expected 1.5.

- Issue #9808: Implement os.getlogin for Windows. Patch by Jon Anglin.

- Issue #9901: Destroying the GIL in Py_Finalize() can fail if some other

threads are still running. Instead, reinitialize the GIL on a second call to

Py_Initialize().

- All SyntaxErrors now have a column offset and therefore a caret when the error

is printed.

- Issue #9252: PyImport_Import no longer uses a fromlist hack to return the

module that was imported, but instead gets the module from sys.modules.

- Issue #9212: The range type_items now provides index() and count() methods, to

conform to the Sequence ABC. Patch by Daniel Urban and Daniel Stutzbach.

- Issue #7994: Issue a PendingDeprecationWarning if object.__format__ is called

with a non-empty format string. This is an effort to future-proof user

code. If a derived class does not currently implement __format__ but later

adds its own __format__, it would most likely break user code that had

supplied a format string. This will be changed to a DeprecationWaring in

Python 3.3 and it will be an error in Python 3.4.

- Issue #9828: Destroy the GIL in Py_Finalize(), so that it gets properly

re-created on a subsequent call to Py_Initialize(). The problem (a crash)

wouldn't appear in 3.1 or 2.7 where the GIL's structure is more trivial.

- Issue #9210: Configure option --with-wctype-functions was removed. Using the

functions from the libc caused the methods .upper() and lower() to become

locale aware and created subtly wrong results.

- Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on a

non-ASCII byte in the format string.

- Issue #4617: Previously it was illegal to delete a name from the local

namespace if it occurs as a free variable in a nested block. This limitation

of the compiler has been lifted, and a new opcode introduced (DELETE_DEREF).

- Issue #9804: ascii() now always represents unicode surrogate pairs as a single

``\UXXXXXXXX``, regardless of whether the character is printable or not.

Also, the "backslashreplace" error handler now joins surrogate pairs into a

single character on UCS-2 builds.

- Issue #9757: memoryview objects get a release() method to release the

underlying buffer (previously this was only done when deallocating the

memoryview), and gain support for the context management protocol.

- Issue #9797: pystate.c wrongly assumed that zero couldn't be a valid

thread-local storage key.

Library

-------

- Issue #9437: Fix building C extensions with non-default LDFLAGS.

- Issue #4661: email can now parse bytes input and generate either converted

7bit output or bytes output. Email version bumped to 5.1.0.

- Issue #1589: Add ssl.match_hostname(), to help implement server identity

verification for higher-level protocols.

- Issue #9759: GzipFile now raises ValueError when an operation is attempted

after the file is closed. Patch by Jeffrey Finkelstein.

- Issue #9042: Fix interaction of custom translation classes and caching in

gettext.

- Issue #6706: asyncore.dispatcher now provides a handle_accepted() method

returning a (sock, addr) pair which is called when a connection has been

established with a new remote endpoint. This is supposed to be used as a

replacement for old handle_accept() and avoids the user to call accept()

directly.

- Issue #9065: tarfile no longer uses "root" as the default for the uname and

gname field.

- Issue #8980: Fixed a failure in distutils.command check that was shadowed by

an environment that does not have docutils. Patch by Arfrever.

- Issue #1050268: parseaddr now correctly quotes double quote and backslash

characters that appear inside quoted strings in email addresses.

- Issue #10004: quoprimime no longer generates a traceback when confronted with

invalid characters after '=' in a Q-encoded word.

- Issue #1491: BaseHTTPServer nows send a ``100 Continue`` response before

sending a 200 OK for the Expect: 100-continue request header.

- Issue #9360: Cleanup and improvements to the nntplib module. The API now

conforms to the philosophy of bytes and unicode separation in Python 3. A

test suite has also been added.

- Issue #9962: GzipFile now has the peek() method.

- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or EAGAIN,

retry the select() loop instead of bailing out. This is because select() can

incorrectly report a socket as ready for reading (for example, if it received

some data with an invalid checksum).

- Issue #3612: Added new types to ctypes.wintypes. (CHAR and pointers)

- Issue #9950: Fix socket.sendall() crash or misbehaviour when a signal is

received. Now sendall() properly calls signal handlers if necessary, and

retries sending if these returned successfully, including on sockets with a

timeout.

- Issue #9947: logging: Fixed locking bug in stopListening.

- Issue #9945: logging: Fixed locking bugs in addHandler/removeHandler.

- Issue #9936: Fixed executable lines' search in the trace module.

- Issue #9790: Rework imports necessary for samefile and sameopenfile

in ntpath.

- Issue #9928: Properly initialize the types exported by the bz2 module.

- Issue #1675951: Allow GzipFile to work with unseekable file objects. Patch by

Florian Festi.

- Logging: Added QueueListener class to facilitate logging usage for

performance-critical threads.

- Issue #9916: Add some missing errno symbols.

- Issue #9877: Expose sysconfig.get_makefile_filename()

- logging: Added hasHandlers() method to Logger and LoggerAdapter.

- Issue #9908: Fix os.stat() on bytes paths under Windows 7.

- Issue #2643: msync() is not called anymore when deallocating an open mmap

object, only munmap().

- logging: Changed LoggerAdapter implementation internally, to make it easier to

subclass in a useful way.

- logging: hasHandlers method was added to Logger, and isEnabledFor,

getEffectiveLevel, hasHandlers and setLevel were added to LoggerAdapter.

LoggerAdapter was introduced into the unit tests for logging.

- Issue #1686: Fix string.Template when overriding the pattern attribute.

- Issue #9854: SocketIO objects now observe the RawIOBase interface in

non-blocking mode: they return None when an operation would block (instead of

raising an exception).

- Issue #1730136: Fix the comparison between a tk.font.Font and an object of

another kind.

- Issue #9441: logging has better coverage for rotating file handlers.

- Issue #9865: collections.OrderedDict now has a __sizeof__ method.

- Issue #9854: The default read() implementation in io.RawIOBase now handles

non-blocking readinto() returning None correctly.

- Issue #1552: socket.socketpair() now returns regular socket.socket objects

supporting the whole socket API (rather than the "raw" _socket.socket

objects).

- Issue #9853: Fix the signature of SSLSocket.recvfrom() and SSLSocket.sendto()

to match the corresponding socket methods.

- Issue 9840: Added a decorator to reprlib for wrapping __repr__ methods to make

them handle recursive calls within the same thread.

- logging: Enhanced HTTPHandler with secure and credentials initializers.

- Issue #767645: Set os.path.supports_unicode_filenames to True on Mac OS X.

- Issue #9837: The read() method of ZipExtFile objects (as returned by

ZipFile.open()) could return more bytes than requested.

- Issue #9826: OrderedDict.__repr__ can now handle self-referential values:

d['x'] = d.

- Issue #9825: Using __del__ in the definition of collections.OrderedDict made

it possible for the user to create self-referencing ordered dictionaries which

become permanently uncollectable GC garbage. Reinstated the Python 3.1

approach of using weakref proxies so that reference cycles never get created

in the first place.

- Issue #9579, #9580: Fix os.confstr() for value longer than 255 bytes and

encode the value with filesystem encoding and surrogateescape (instead of

utf-8 in strict mode) . Patch written by David Watson.

- Issue #9632: Remove sys.setfilesystemencoding() function: use PYTHONFSENCODING

environment variable to set the filesystem encoding at Python startup.

sys.setfilesystemencoding() creates inconsistencies because it is unable to

reencode all filenames in all objects.

- Issue #9410: Various optimizations to the pickle module, leading to speedups

up to 4x (depending on the benchmark). Mostly ported from Unladen Swallow;

initial patch by Alexandre Vassalotti.

- The pprint module now supports printing OrderedDicts in their given order

(formerly, it would sort the keys).

- Logging: Added QueueHandler class to facilitate logging usage with

multiprocessing.

- Issue #9707: Rewritten reference implementation of threading.local which is

friendlier towards reference cycles. This change is not normally visible

since an optimized C implementation (_thread._local) is used instead.

- Issue #6394: os.getppid() is now supported on Windows. Note that it will

still return the id of the parent process after it has exited. This process

id may even have been reused by another unrelated process.

- Issue #9792: In case of connection failure, socket.create_connection() would

swallow the exception and raise a new one, making it impossible to fetch the

original errno, or to filter timeout errors. Now the original error is

re-raised.

- Issue #9758: When fcntl.ioctl() was called with mutable_flag set to True, and

the passed buffer was exactly 1024 bytes long, the buffer wouldn't be updated

back after the system call. Original patch by Brian Brazil.

- Updates to the random module:

* Document which parts of the module are guaranteed to stay the same across

versions and which parts are subject to change.

* Update the seed() method to use all of the bits in a string instead of just

the hash value. This makes better use of the seed value and assures the

seeding is platform independent. Issue #7889.

* Improved the random()-->integer algorithm used in choice(), shuffle(),

sample(), randrange(), and randint(). Formerly, it used int(n*random())

which has a slight bias whenever n is not a power of two. Issue #9025.

* Improved documentation of arguments to randrange(). Issue #9379.

- collections.OrderedDict now supports a new method for repositioning keys to

either end.

- Issue #9754: Similarly to assertRaises and assertRaisesRegexp, unittest test

cases now also have assertWarns and assertWarnsRegexp methods to check that a

given warning type was triggered by the code under test.

- Issue #5506: BytesIO objects now have a getbuffer() method exporting a view of

their contents without duplicating them. The view is both readable and

writable.

- Issue #7566: Implement os.path.sameopenfile for Windows.

- Issue #9293: I/O streams now raise ``io.UnsupportedOperation`` when an

unsupported operation is attempted (for example, writing to a file open only

for reading).

- hashlib has two new constant attributes: algorithms_guaranteed and

algorithms_avaiable that respectively list the names of hash algorithms

guaranteed to exist in all Python implementations and the names of hash

algorithms available in the current process.

- A new package ``concurrent.futures`` as defined by PEP 3148.

C-API

-----

- Add PyErr_SyntaxLocationEx, which supports passing a column offset.

- Issue #9834: Don't segfault in PySequence_GetSlice, PySequence_SetSlice, or

PySequence_DelSlice when the object doesn't have any mapping operations

defined.

Tools/Demos

-----------

- Issue #9188: The gdb extension now handles correctly narrow (UCS2) as well as

wide (UCS4) unicode builds for both the host interpreter (embedded inside gdb)

and the interpreter under test.

Tests

-----

- Issue #9308: Added tests for importing encoded modules that do not

depend on specific stdlib modules being encoded in a certain way.

- Issue #1051: Add a script (Lib/test/make_ssl_certs.py) to generate the custom

certificate and private key files used by SSL-related certs.

- Issue #9978: Wait until subprocess completes initialization. (Win32KillTests

in test_os)

- Issue #7110: regrtest now sends test failure reports and single-failure

tracebacks to stderr rather than stdout.

- Issue #9628: fix runtests.sh -x option so more than one test can be excluded.

- Issue #9899: Fix test_tkinter.test_font on various platforms. Patch by Ned

Deily.

- Issue #9894: Do not hardcode ENOENT in test_subprocess.

- Issue #9315: Added tests for the trace module. Patch by Eli Bendersky.

- Issue #9323: Make test.regrtest.__file__ absolute, this was not always the

case when running profile or trace, for example.

- Issue #9568: Fix test_urllib2_localnet on OS X 10.3.

Build

-----

- Issue #10062: Allow building on platforms which do not have sem_timedwait.

- Issue #10054: Some platforms provide uintptr_t in inttypes.h. Patch by Akira

Kitada.

- Issue #10055: Make json C89-compliant in UCS4 mode.

- Issue #9552: Avoid unnecessary rebuild of OpenSSL. (Windows)

- Issue #1633863: Don't ignore $CC under AIX.

- Issue #9810: Compile bzip2 source files in Python's project file directly. It

used to be built with bzip2's makefile.

- Issue #9848: Stopping trying to build _weakref in setup.py as it is a built-in

module.

- Issue #9806: python-config now has an ``--extension-suffix`` option that

outputs the suffix for dynamic libraries including the ABI version name

defined by PEP 3149.

- Issue #941346: Improve the build process under AIX and allow Python to be

built as a shared library. Patch by Sébastien Sablé.

- Issue #4026: Make the fcntl extension build under AIX. Patch by Sébastien

Sablé.

- Issue #9701: The MacOSX installer can patch the shell profile to ensure that

the "bin" directory inside the framework is on the shell's search path. This

feature now also supports the ZSH shell.

What's New in Python 3.2 Alpha 2?

=================================

*Release date: 05-Sep-2010*

Core and Builtins

-----------------

- Issue #9225: Remove the ROT_FOUR and DUP_TOPX opcode, the latter replaced by

the new (and simpler) DUP_TOP_TWO. Performance isn't changed, but our

bytecode is a bit simplified. Patch by Demur Rumed.

- Issue #9766: Rename poorly named variables exposed by _warnings to prevent

confusion with the proper variables names from 'warnings' itself.

- Issue #9212: dict_keys and dict_items now provide the isdisjoint() method, to

conform to the Set ABC. Patch by Daniel Urban.

- Issue #9737: Fix a crash when trying to delete a slice or an item from a

memoryview object.

- Issue #9549: sys.setdefaultencoding() and PyUnicode_SetDefaultEncoding() are

now removed, since their effect was inexistent in 3.x (the default encoding is

hardcoded to utf-8 and cannot be changed).

- Issue #7415: PyUnicode_FromEncodedObject() now uses the new buffer API

properly. Patch by Stefan Behnel.

- Issue #5553: The Py_LOCAL_INLINE macro now results in inlining on most

platforms. Previously, it inlined only when using Microsoft Visual C.

- Issue #9712: Fix tokenize on identifiers that start with non-ascii names.

- Issue #9688: __basicsize__ and __itemsize__ must be accessed as Py_ssize_t.

- Issue #9684: Added a definition for SIZEOF_WCHAR_T to PC/pyconfig.h, to match

the pyconfig.h generated by configure on other systems.

- Issue #9666: Only catch AttributeError in hasattr(). All other exceptions that

occur during attribute lookup are now propagated to the caller.

- Issue #8622: Add PYTHONFSENCODING environment variable to override the

filesystem encoding.

- Issue #5127: The C functions that access the Unicode Database now accept and

return characters from the full Unicode range, even on narrow unicode builds

(Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others). A visible difference

in Python is that unicodedata.numeric() now returns the correct value for

large code points, and repr() may consider more characters as printable.

- Issue #9425: Create PyModule_GetFilenameObject() function to get the filename

as a unicode object, instead of a byte string. Function needed to support

unencodable filenames. Deprecate PyModule_GetFilename() in favor on the new

function.

- Issue #8063: Call _PyGILState_Init() earlier in Py_InitializeEx().

- Issue #9612: The set object is now 64-bit clean under Windows.

- Issue #8202: sys.argv[0] is now set to '-m' instead of '-c' when searching for

the module file to be executed with the -m command line option.

- Issue #9599: Create PySys_FormatStdout() and PySys_FormatStderr() functions to

write a message formatted by PyUnicode_FromFormatV() to sys.stdout and

sys.stderr.

- Issue #9542: Create PyUnicode_FSDecoder() function, a ParseTuple converter:

decode bytes objects to unicode using PyUnicode_DecodeFSDefaultAndSize(); str

objects are output as-is.

- Issue #9203: Computed gotos are now enabled by default on supported compilers

(which are detected by the configure script). They can still be disable

selectively by specifying --without-computed-gotos.

- Issue #9425: Create PyErr_WarnFormat() function, similar to PyErr_WarnEx() but

use PyUnicode_FromFormatV() to format the warning message.

- Issue #8530: Prevent stringlib fastsearch from reading beyond the front of an

array.

- Issue #5319: Print an error if flushing stdout fails at interpreter shutdown.

- Issue #9337: The str() of a float or complex number is now identical to its

repr().

- Issue #9416: Fix some issues with complex formatting where the output with no

type specifier failed to match the str output:

- format(complex(-0.0, 2.0), '-') omitted the real part from the output,

- format(complex(0.0, 2.0), '-') included a sign and parentheses.

Extensions

----------

- Issue #6608: time.asctime is now checking struct tm fields its input

before passing it to the system asctime. Patch by MunSic Jeong.

- Issue #8734: Avoid crash in msvcrt.get_osfhandle() when an invalid file

descriptor is provided. Patch by Pascal Chambon.

- Issue #7736: Release the GIL around calls to opendir() and closedir() in the

posix module. Patch by Marcin Bachry.

- Issue #4835: make PyLong_FromSocket_t() and PyLong_AsSocket_t() private to the

socket module, and fix the width of socket descriptors to be correctly

detected under 64-bit Windows.

- Issue #1027206: Support IDNA in gethostbyname, gethostbyname_ex, getaddrinfo

and gethostbyaddr. getnameinfo is now restricted to numeric addresses as

input.

- Issue #9214: Set operations on a KeysView or ItemsView in collections now

correctly return a set. Patch by Eli Bendersky.

- Issue #5737: Add Solaris-specific mnemonics in the errno module. Patch by

Matthew Ahrens.

- Restore GIL in nis_cat in case of error. Decode NIS data to fs encoding, using

the surrogate error handler.

- Issue #665761: ``functools.reduce()`` will no longer mask exceptions other

than ``TypeError`` raised by the iterator argument.

- Issue #9570: Use PEP 383 decoding in os.mknod and os.mkfifo.

- Issue #6915: Under Windows, os.listdir() didn't release the Global Interpreter

Lock around all system calls. Original patch by Ryan Kelly.

- Issue #8524: Add a detach() method to socket objects, so as to put the socket

into the closed state without closing the underlying file descriptor.

- Issue #477863: Emit a ResourceWarning at shutdown if gc.garbage is not empty.

- Issue #6869: Fix a refcount problem in the _ctypes extension.

- Issue #5504: ctypes should now work with systems where mmap can't be

PROT_WRITE and PROT_EXEC.

- Issue #9507: Named tuple repr will now automatically display the right name in

a tuple subclass.

- Issue #9324: Add parameter validation to signal.signal on Windows in order to

prevent crashes.

- Issue #9526: Remove some outdated (int) casts that were preventing the array

module from working correctly with arrays of more than 2**31 elements.

- Fix memory leak in ssl._ssl._test_decode_cert.

- Issue #8065: Fix memory leak in readline module (from failure to free the

result of history_get_history_state()).

- Issue #9450: Fix memory leak in readline.replace_history_item and

readline.remove_history_item for readline version >= 5.0.

- Issue #8105: Validate file descriptor passed to mmap.mmap on Windows.

- Issue #8046: Add context manager protocol support and .closed property to mmap

objects.

Library

-------

- Issue #7451: Improve decoding performance of JSON objects, and reduce the

memory consumption of said decoded objects when they use the same strings as

keys.

- Issue #1100562: Fix deep-copying of objects derived from the list and dict

types. Patch by Michele Orrù and Björn Lindqvist.

- Issue #9753: Fixed socket.dup, which did not always work correctly on Windows.

- Issue #9421: Made the get<type> methods consistently accept the vars and

default arguments on all parser classes.

- Issue #7005: Fixed output of None values for RawConfigParser.write and

ConfigParser.write.

- Issue #8990: array.fromstring() and array.tostring() get renamed to

frombytes() and tobytes(), respectively, to avoid confusion. Furthermore,

array.frombytes(), array.extend() as well as the array.array() constructor now

accept bytearray objects. Patch by Thomas Jollans.

- Issue #808164: Fixed socket.close to avoid references to globals, to avoid

issues when socket.close is called from a __del__ method.

- Issue #9706: ssl module provides a better error handling in various

circumstances.

- Issue #1868: Eliminate subtle timing issues in thread-local objects by getting

rid of the cached copy of thread-local attribute dictionary.

- Issue #1512791: In setframerate() in the wave module, non-integral frame rates

are rounded to the nearest integer.

- Issue #8797: urllib2 does a retry for Basic Authentication failure instead of

falling into recursion.

- Issue #1194222: email.utils.parsedate now returns RFC2822 compliant four

character years even if the message contains RFC822 two character years.

- Issue #8750: Fixed MutableSet's methods to correctly handle reflexive

operations on its self, namely x -= x and x ^= x.

- Issue #9129: smtpd.py is vulnerable to DoS attacks deriving from missing error

handling when accepting a new connection.

- Issue #9601: ftplib now provides a workaround for non-compliant

implementations such as IIS shipped with Windows server 2003 returning invalid

response codes for MKD and PWD commands.

- Issue #658749: asyncore's connect() method now correctly interprets winsock

errors.

- Issue #9501: Fixed logging regressions in cleanup code.

- Fix functools.total_ordering() to skip methods inherited from object.

- Issue #9572: Importlib should not raise an exception if a directory it thought

it needed to create was done concurrently by another process.

- Issue #9617: Signals received during a low-level write operation aren't

ignored by the buffered IO layer anymore.

- Issue #843590: Make "macintosh" an alias to the "mac_roman" encoding.

- Create os.fsdecode(): decode from the filesystem encoding with surrogateescape

error handler, or strict error handler on Windows.

- Issue #3488: Provide convenient shorthand functions ``gzip.compress`` and

``gzip.decompress``. Original patch by Anand B. Pillai.

- Issue #8807: poplib.POP3_SSL class now accepts a context parameter, which is a

ssl.SSLContext object allowing bundling SSL configuration options,

certificates and private keys into a single (potentially long-lived)

structure.

- Issue #8866: parameters passed to socket.getaddrinfo can now be specified as

single keyword arguments.

- Address XXX comment in dis.py by having inspect.py prefer to reuse the dis.py

compiler flag values over defining its own.

- Issue #9147: Added dis.code_info() which is similar to show_code() but returns

formatted code information in a string rather than displaying on screen.

- Issue #9567: functools.update_wrapper now adds a __wrapped__ attribute

pointing to the original callable.

- Issue #3445: functools.update_wrapper now tolerates missing attributes on

wrapped callables.

- Issue #5867: Add abc.abstractclassmethod and abc.abstractstaticmethod.

- Issue #9605: posix.getlogin() decodes the username with file filesystem

encoding and surrogateescape error handler. Patch written by David Watson.

- Issue #9604: posix.initgroups() encodes the username using the fileystem

encoding and surrogateescape error handler. Patch written by David Watson.

- Issue #9603: posix.ttyname() and posix.ctermid() decode the terminal name

using the filesystem encoding and surrogateescape error handler. Patch written

by David Watson.

- Issue #7647: The posix module now has the ST_RDONLY and ST_NOSUID constants,

for use with the statvfs() function. Patch by Adam Jackson.

- Issue #8688: MANIFEST files created by distutils now include a magic comment

indicating they are generated. Manually maintained MANIFESTs without this

marker will not be overwritten or removed.

- Issue #7467: when reading a file from a ZIP archive, its CRC is checked and a

BadZipfile error is raised if it doesn't match (as used to be the case in

Python 2.5 and earlier).

- Issue #9550: a BufferedReader could issue an additional read when the original

read request had been satisfied, which could block indefinitely when the

underlying raw IO channel was e.g. a socket. Report and original patch by

Jason V. Miller.

- Issue #3757: thread-local objects now support cyclic garbage collection.

Thread-local objects involved in reference cycles will be deallocated timely

by the cyclic GC, even if the underlying thread is still running.

- Issue #9452: Add read_file, read_string, and read_dict to the configparser

API; new source attribute to exceptions.

- Issue #6231: Fix xml.etree.ElementInclude to include the tail of the current

node.

- Issue #8047: Fix the xml.etree serializer to return bytes by default. Use

``encoding="unicode"`` to generate a Unicode string.

- Issue #8280: urllib2's Request method will remove fragments in the url. This

is how it is supposed to work, wget and curl do the same. Previous behavior

was wrong.

- Issue #6683: For SMTP logins we now try all authentication methods advertised

by the server. Many servers are buggy and advertise authentication methods

they do not support in reality.

- Issue #8814: function annotations (the ``__annotations__`` attribute) are now

included in the set of attributes copied by default by functools.wraps and

functools.update_wrapper. Patch by Terrence Cole.

- Issue #2944: asyncore doesn't handle connection refused correctly.

- Issue #4184: Private attributes on smtpd.SMTPChannel made public and deprecate

the private attributes. Add tests for smtpd module.

- Issue #3196: email header decoding is now forgiving if an RFC2047 encoded word

encoded in base64 is lacking padding.

- Issue #9444: Argparse now uses the first element of prefix_chars as the option

character for the added 'h/help' option if prefix_chars does not contain a

'-', instead of raising an error.