Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 92843e3

Browse files
benjaminpbenjaminp
committed
merge heads
2 parents 774f83c + 37d72b7 commit 92843e3

15 files changed

Lines changed: 440 additions & 145 deletions

File tree

‎Doc/library/smtplib.rst‎

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -348,21 +348,32 @@ An :class:`SMTP` instance has the following methods:
348348
.. versionchanged:: 3.2 *msg* may be a byte string.
349349

350350

351-
.. method:: SMTP.send_message(msg, from_addr=None, to_addrs=None, mail_options=[], rcpt_options=[])
351+
.. method:: SMTP.send_message(msg, from_addr=None, to_addrs=None, \
352+
mail_options=[], rcpt_options=[])
352353

353354
This is a convenience method for calling :meth:`sendmail` with the message
354355
represented by an :class:`email.message.Message` object. The arguments have
355356
the same meaning as for :meth:`sendmail`, except that *msg* is a ``Message``
356357
object.
357358

358-
If *from_addr* is ``None``, ``send_message`` sets its value to the value of
359-
the :mailheader:`From` header from *msg*. If *to_addrs* is ``None``,
360-
``send_message`` combines the values (if any) of the :mailheader:`To`,
361-
:mailheader:`CC`, and :mailheader:`Bcc` fields from *msg*. Regardless of
362-
the values of *from_addr* and *to_addrs*, ``send_message`` deletes any Bcc
363-
field from *msg*. It then serializes *msg* using
359+
If *from_addr* is ``None`` or *to_addrs* is ``None``, ``send_message`` fills
360+
those arguments with addresses extracted from the headers of *msg* as
361+
specified in :rfc:`2822`\: *from_addr* is set to the :mailheader:`Sender`
362+
field if it is present, and otherwise to the :mailheader:`From` field.
363+
*to_adresses* combines the values (if any) of the :mailheader:`To`,
364+
:mailheader:`Cc`, and :mailheader:`Bcc` fields from *msg*. If exactly one
365+
set of :mailheader:`Resent-*` headers appear in the message, the regular
366+
headers are ignored and the :mailheader:`Resent-*` headers are used instead.
367+
If the message contains more than one set of :mailheader:`Resent-*` headers,
368+
a :exc:`ValueError` is raised, since there is no way to unambiguously detect
369+
the most recent set of :mailheader:`Resent-` headers.
370+
371+
``send_message`` serializes *msg* using
364372
:class:`~email.generator.BytesGenerator` with ``\r\n`` as the *linesep*, and
365-
calls :meth:`sendmail` to transmit the resulting message.
373+
calls :meth:`sendmail` to transmit the resulting message. Regardless of the
374+
values of *from_addr* and *to_addrs*, ``send_message`` does not transmit any
375+
:mailheader:`Bcc` or :mailheader:`Resent-Bcc` headers that may appear
376+
in *msg*.
366377

367378
.. versionadded:: 3.2
368379

‎Doc/library/sqlite3.rst‎

Lines changed: 38 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -615,43 +615,43 @@ Row Objects
615615

616616
Let's assume we initialize a table as in the example given above::
617617

618-
conn = sqlite3.connect(":memory:")
619-
c = conn.cursor()
620-
c.execute('''create table stocks
621-
(date text, trans text, symbol text,
622-
qty real, price real)''')
623-
c.execute("""insert into stocks
624-
values ('2006-01-05','BUY','RHAT',100,35.14)""")
625-
conn.commit()
626-
c.close()
618+
conn = sqlite3.connect(":memory:")
619+
c = conn.cursor()
620+
c.execute('''create table stocks
621+
(date text, trans text, symbol text,
622+
qty real, price real)''')
623+
c.execute("""insert into stocks
624+
values ('2006-01-05','BUY','RHAT',100,35.14)""")
625+
conn.commit()
626+
c.close()
627627

628628
Now we plug :class:`Row` in::
629629

630-
>>> conn.row_factory = sqlite3.Row
631-
>>> c = conn.cursor()
632-
>>> c.execute('select * from stocks')
633-
<sqlite3.Cursor object at 0x7f4e7dd8fa80>
634-
>>> r = c.fetchone()
635-
>>> type(r)
636-
<class 'sqlite3.Row'>
637-
>>> tuple(r)
638-
('2006-01-05', 'BUY', 'RHAT', 100.0, 35.14)
639-
>>> len(r)
640-
5
641-
>>> r[2]
642-
'RHAT'
643-
>>> r.keys()
644-
['date', 'trans', 'symbol', 'qty', 'price']
645-
>>> r['qty']
646-
100.0
647-
>>> for member in r:
648-
... print(member)
649-
...
650-
2006-01-05
651-
BUY
652-
RHAT
653-
100.0
654-
35.14
630+
>>> conn.row_factory = sqlite3.Row
631+
>>> c = conn.cursor()
632+
>>> c.execute('select * from stocks')
633+
<sqlite3.Cursor object at 0x7f4e7dd8fa80>
634+
>>> r = c.fetchone()
635+
>>> type(r)
636+
<class 'sqlite3.Row'>
637+
>>> tuple(r)
638+
('2006-01-05', 'BUY', 'RHAT', 100.0, 35.14)
639+
>>> len(r)
640+
5
641+
>>> r[2]
642+
'RHAT'
643+
>>> r.keys()
644+
['date', 'trans', 'symbol', 'qty', 'price']
645+
>>> r['qty']
646+
100.0
647+
>>> for member in r:
648+
... print(member)
649+
...
650+
2006-01-05
651+
BUY
652+
RHAT
653+
100.0
654+
35.14
655655

656656

657657
.. _sqlite3-types:
@@ -902,6 +902,7 @@ only makes sense to call from a different thread.
902902
.. rubric:: Footnotes
903903

904904
.. [#f1] The sqlite3 module is not built with loadable extension support by
905-
default, because some platforms (notably Mac OS X) have SQLite libraries which
906-
are compiled without this feature. To get loadable extension support, you must
907-
pass --enable-loadable-sqlite-extensions to configure.
905+
default, because some platforms (notably Mac OS X) have SQLite
906+
libraries which are compiled without this feature. To get loadable
907+
extension support, you must pass --enable-loadable-sqlite-extensions to
908+
configure.

‎Doc/whatsnew/3.3.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ handle NAT with non-secure FTP without opening fixed ports.
206206
shutil
207207
------
208208

209-
The :mod:`shutil` module has a new :func:`~shutil.disk_usage` providing total,
209+
The :mod:`shutil` module has a new :func:`~shutil.disk_usage` function providing total,
210210
used and free disk space statistics.
211211

212212
(Contributed by Giampaolo Rodolà in :issue:`12442`)

‎Lib/concurrent/futures/process.py‎

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@
5050
from concurrent.futures import _base
5151
import queue
5252
import multiprocessing
53-
from multiprocessing.queues import SimpleQueue, SentinelReady
53+
from multiprocessing.queues import SimpleQueue, SentinelReady, Full
5454
import threading
5555
import weakref
5656

@@ -195,15 +195,18 @@ def _queue_management_worker(executor_reference,
195195
result_queue: A multiprocessing.Queue of _ResultItems generated by the
196196
process workers.
197197
"""
198+
executor = None
199+
200+
def shutting_down():
201+
return _shutdown or executor is None or executor._shutdown_thread
198202

199203
def shutdown_worker():
200204
# This is an upper bound
201205
nb_children_alive = sum(p.is_alive() for p in processes.values())
202206
for i in range(0, nb_children_alive):
203-
call_queue.put(None)
207+
call_queue.put_nowait(None)
204208
# If .join() is not called on the created processes then
205-
# some multiprocessing.Queue methods may deadlock on Mac OS
206-
# X.
209+
# some multiprocessing.Queue methods may deadlock on Mac OS X.
207210
for p in processes.values():
208211
p.join()
209212

@@ -222,7 +225,7 @@ def shutdown_worker():
222225
if executor is not None:
223226
executor._broken = True
224227
executor._shutdown_thread = True
225-
del executor
228+
executor = None
226229
# All futures in flight must be marked failed
227230
for work_id, work_item in pending_work_items.items():
228231
work_item.future.set_exception(
@@ -242,7 +245,11 @@ def shutdown_worker():
242245
if isinstance(result_item, int):
243246
# Clean shutdown of a worker using its PID
244247
# (avoids marking the executor broken)
248+
assert shutting_down()
245249
del processes[result_item]
250+
if not processes:
251+
shutdown_worker()
252+
return
246253
elif result_item is not None:
247254
work_item = pending_work_items.pop(result_item.work_id, None)
248255
# work_item can be None if another process terminated (see above)
@@ -257,16 +264,21 @@ def shutdown_worker():
257264
# - The interpreter is shutting down OR
258265
# - The executor that owns this worker has been collected OR
259266
# - The executor that owns this worker has been shutdown.
260-
if _shutdown or executor is None or executor._shutdown_thread:
261-
# Since no new work items can be added, it is safe to shutdown
262-
# this thread if there are no pending work items.
263-
if not pending_work_items:
264-
shutdown_worker()
265-
return
266-
else:
267-
# Start shutting down by telling a process it can exit.
268-
call_queue.put(None)
269-
del executor
267+
if shutting_down():
268+
try:
269+
# Since no new work items can be added, it is safe to shutdown
270+
# this thread if there are no pending work items.
271+
if not pending_work_items:
272+
shutdown_worker()
273+
return
274+
else:
275+
# Start shutting down by telling a process it can exit.
276+
call_queue.put_nowait(None)
277+
except Full:
278+
# This is not a problem: we will eventually be woken up (in
279+
# result_queue.get()) and be able to send a sentinel again.
280+
pass
281+
executor = None
270282

271283
_system_limits_checked = False
272284
_system_limited = None

‎Lib/importlib/test/source/test_file_loader.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ def _test_bad_marshal(self, *, del_source=False):
214214
lambda bc: bc[:8] + b'<test>',
215215
del_source=del_source)
216216
file_path = mapping['_temp'] if not del_source else bytecode_path
217-
with self.assertRaises(ValueError):
217+
with self.assertRaises(EOFError):
218218
self.import_(file_path, '_temp')
219219

220220
def _test_bad_magic(self, test, *, del_source=False):

‎Lib/smtplib.py‎

100755100644
Lines changed: 39 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@
4949
import email.generator
5050
import base64
5151
import hmac
52+
import copy
5253
from email.base64mime import body_encode as encode_base64
5354
from sys import stderr
5455

@@ -676,7 +677,7 @@ def sendmail(self, from_addr, to_addrs, msg, mail_options=[],
676677
677678
msg may be a string containing characters in the ASCII range, or a byte
678679
string. A string is encoded to bytes using the ascii codec, and lone
679-
\r and \n characters are converted to \r\n characters.
680+
\\r and \\n characters are converted to \\r\\n characters.
680681
681682
If there has been no previous EHLO or HELO command this session, this
682683
method tries ESMTP EHLO first. If the server does ESMTP, message size
@@ -759,24 +760,49 @@ def send_message(self, msg, from_addr=None, to_addrs=None,
759760
"""Converts message to a bytestring and passes it to sendmail.
760761
761762
The arguments are as for sendmail, except that msg is an
762-
email.message.Message object. If from_addr is None, the from_addr is
763-
taken from the 'From' header of the Message. If to_addrs is None, its
764-
value is composed from the addresses listed in the 'To', 'CC', and
765-
'Bcc' fields. Regardless of the values of from_addr and to_addr, any
766-
Bcc field in the Message object is deleted. The Message object is then
767-
serialized using email.generator.BytesGenerator and sendmail is called
768-
to transmit the message.
763+
email.message.Message object. If from_addr is None or to_addrs is
764+
None, these arguments are taken from the headers of the Message as
765+
described in RFC 2822 (a ValueError is raised if there is more than
766+
one set of 'Resent-' headers). Regardless of the values of from_addr and
767+
to_addr, any Bcc field (or Resent-Bcc field, when the Message is a
768+
resent) of the Message object won't be transmitted. The Message
769+
object is then serialized using email.generator.BytesGenerator and
770+
sendmail is called to transmit the message.
771+
769772
"""
773+
# 'Resent-Date' is a mandatory field if the Message is resent (RFC 2822
774+
# Section 3.6.6). In such a case, we use the 'Resent-*' fields. However,
775+
# if there is more than one 'Resent-' block there's no way to
776+
# unambiguously determine which one is the most recent in all cases,
777+
# so rather than guess we raise a ValueError in that case.
778+
#
779+
# TODO implement heuristics to guess the correct Resent-* block with an
780+
# option allowing the user to enable the heuristics. (It should be
781+
# possible to guess correctly almost all of the time.)
782+
resent =msg.get_all('Resent-Date')
783+
if resent is None:
784+
header_prefix = ''
785+
elif len(resent) == 1:
786+
header_prefix = 'Resent-'
787+
else:
788+
raise ValueError("message has more than one 'Resent-' header block")
770789
if from_addr is None:
771-
from_addr = msg['From']
790+
# Prefer the sender field per RFC 2822:3.6.2.
791+
from_addr = (msg[header_prefix+'Sender']
792+
if (header_prefix+'Sender') in msg
793+
else msg[header_prefix+'From'])
772794
if to_addrs is None:
773-
addr_fields = [f for f in (msg['To'], msg['Bcc'], msg['CC'])
774-
if f is not None]
795+
addr_fields = [f for f in (msg[header_prefix+'To'],
796+
msg[header_prefix+'Bcc'],
797+
msg[header_prefix+'Cc']) if f is not None]
775798
to_addrs = [a[1] for a in email.utils.getaddresses(addr_fields)]
776-
del msg['Bcc']
799+
# Make a local copy so we can delete the bcc headers.
800+
msg_copy = copy.copy(msg)
801+
del msg_copy['Bcc']
802+
del msg_copy['Resent-Bcc']
777803
with io.BytesIO() as bytesmsg:
778804
g = email.generator.BytesGenerator(bytesmsg)
779-
g.flatten(msg, linesep='\r\n')
805+
g.flatten(msg_copy, linesep='\r\n')
780806
flatmsg = bytesmsg.getvalue()
781807
return self.sendmail(from_addr, to_addrs, flatmsg, mail_options,
782808
rcpt_options)

‎Lib/test/test_concurrent_futures.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -367,6 +367,13 @@ def test_map_timeout(self):
367367

368368
self.assertEqual([None, None], results)
369369

370+
def test_shutdown_race_issue12456(self):
371+
# Issue #12456: race condition at shutdown where trying to post a
372+
# sentinel in the call queue blocks (the queue is full while processes
373+
# have exited).
374+
self.executor.map(str, [2] * (self.worker_count + 1))
375+
self.executor.shutdown()
376+
370377

371378
class ThreadPoolExecutorTest(ThreadPoolMixin, ExecutorTest):
372379
def test_map_submits_without_iteration(self):

‎Lib/test/test_csv.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -459,20 +459,20 @@ def test_quoted(self):
459459
'5', '6']])
460460

461461
def test_quoted_quote(self):
462-
self.readerAssertEqual('1,2,3,"""I see,"" said the blind man","as he picked up his hammer and saw"',
462+
self.readerAssertEqual('1,2,3,"""I see,"" said the happy man","as he picked up his hammer and saw"',
463463
[['1', '2', '3',
464-
'"I see," said the blind man',
464+
'"I see," said the happy man',
465465
'as he picked up his hammer and saw']])
466466

467467
def test_quoted_nl(self):
468468
input = '''\
469469
1,2,3,"""I see,""
470-
said the blind man","as he picked up his
470+
said the happy man","as he picked up his
471471
hammer and saw"
472472
9,8,7,6'''
473473
self.readerAssertEqual(input,
474474
[['1', '2', '3',
475-
'"I see,"\nsaid the blind man',
475+
'"I see,"\nsaid the happy man',
476476
'as he picked up his\nhammer and saw'],
477477
['9','8','7','6']])
478478

‎Lib/test/test_marshal.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,6 +228,30 @@ def test_invalid_longs(self):
228228
invalid_string = b'l\x02\x00\x00\x00\x00\x00\x00\x00'
229229
self.assertRaises(ValueError, marshal.loads, invalid_string)
230230

231+
def test_multiple_dumps_and_loads(self):
232+
# Issue 12291: marshal.load() should be callable multiple times
233+
# with interleaved data written by non-marshal code
234+
# Adapted from a patch by Engelbert Gruber.
235+
data = (1, 'abc', b'def', 1.0, (2, 'a', ['b', b'c']))
236+
for interleaved in (b'', b'0123'):
237+
ilen = len(interleaved)
238+
positions = []
239+
try:
240+
with open(support.TESTFN, 'wb') as f:
241+
for d in data:
242+
marshal.dump(d, f)
243+
if ilen:
244+
f.write(interleaved)
245+
positions.append(f.tell())
246+
with open(support.TESTFN, 'rb') as f:
247+
for i, d in enumerate(data):
248+
self.assertEqual(d, marshal.load(f))
249+
if ilen:
250+
f.read(ilen)
251+
self.assertEqual(positions[i], f.tell())
252+
finally:
253+
support.unlink(support.TESTFN)
254+
231255

232256
def test_main():
233257
support.run_unittest(IntTestCase,

‎Lib/test/test_shutil.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -732,11 +732,11 @@ def _boo(filename, extract_dir, extra):
732732
"disk_usage not available on this platform")
733733
def test_disk_usage(self):
734734
usage = shutil.disk_usage(os.getcwd())
735-
self.assertTrue(usage.total > 0)
736-
self.assertTrue(usage.used > 0)
737-
self.assertTrue(usage.free >= 0)
738-
self.assertTrue(usage.total >= usage.used)
739-
self.assertTrue(usage.total > usage.free)
735+
self.assertGreater(usage.total, 0)
736+
self.assertGreater(usage.used, 0)
737+
self.assertGreaterEqual(usage.free, 0)
738+
self.assertGreaterEqual(usage.total, usage.used)
739+
self.assertGreater(usage.total, usage.free)
740740

741741

742742
class TestMove(unittest.TestCase):

0 commit comments

Comments
 (0)