Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 6166519

Browse files
florentxflorentx
committed
Closes #13297: use bytes type to send and receive binary data through XMLRPC.
1 parent 1d8f3f4 commit 6166519

4 files changed

Lines changed: 130 additions & 46 deletions

File tree

‎Doc/library/xmlrpc.client.rst‎

Lines changed: 35 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99

1010
.. XXX Not everything is documented yet. It might be good to describe
11-
Marshaller, Unmarshaller, getparser, dumps, loads, and Transport.
11+
Marshaller, Unmarshaller, getparser and Transport.
1212
1313
**Source code:** :source:`Lib/xmlrpc/client.py`
1414

@@ -21,7 +21,12 @@ supports writing XML-RPC client code; it handles all the details of translating
2121
between conformable Python objects and XML on the wire.
2222

2323

24-
.. class:: ServerProxy(uri, transport=None, encoding=None, verbose=False, allow_none=False, use_datetime=False)
24+
.. class:: ServerProxy(uri, transport=None, encoding=None, verbose=False, \
25+
allow_none=False, use_datetime=False, \
26+
use_builtin_types=False)
27+
28+
.. versionchanged:: 3.3
29+
The *use_builtin_types* flag was added.
2530

2631
A :class:`ServerProxy` instance is an object that manages communication with a
2732
remote XML-RPC server. The required first argument is a URI (Uniform Resource
@@ -34,9 +39,13 @@ between conformable Python objects and XML on the wire.
3439
XML; the default behaviour is for ``None`` to raise a :exc:`TypeError`. This is
3540
a commonly-used extension to the XML-RPC specification, but isn't supported by
3641
all clients and servers; see http://ontosys.com/xml-rpc/extensions.php for a
37-
description. The *use_datetime* flag can be used to cause date/time values to
38-
be presented as :class:`datetime.datetime` objects; this is false by default.
39-
:class:`datetime.datetime` objects may be passed to calls.
42+
description. The *use_builtin_types* flag can be used to cause date/time values
43+
to be presented as :class:`datetime.datetime` objects and binary data to be
44+
presented as :class:`bytes` objects; this flag is false by default.
45+
:class:`datetime.datetime` and :class:`bytes` objects may be passed to calls.
46+
47+
The obsolete *use_datetime* flag is similar to *use_builtin_types* but it
48+
applies only to date/time values.
4049

4150
Both the HTTP and HTTPS transports support the URL syntax extension for HTTP
4251
Basic Authentication: ``http://user:pass@host:port/path``. The ``user:pass``
@@ -78,12 +87,12 @@ between conformable Python objects and XML on the wire.
7887
| | only their *__dict__* attribute is |
7988
| | transmitted. |
8089
+---------------------------------+---------------------------------------------+
81-
| :const:`dates` | in seconds since the epoch (pass in an |
82-
| | instance of the :class:`DateTime` class) or |
90+
| :const:`dates` | In seconds since the epoch. Pass in an |
91+
| | instance of the :class:`DateTime` class or |
8392
| | a :class:`datetime.datetime` instance. |
8493
+---------------------------------+---------------------------------------------+
85-
| :const:`binary data` | pass in an instance of the :class:`Binary` |
86-
| | wrapper class |
94+
| :const:`binary data` | Pass in an instance of the :class:`Binary` |
95+
| | wrapper class or a :class:`bytes` instance. |
8796
+---------------------------------+---------------------------------------------+
8897

8998
This is the full set of data types supported by XML-RPC. Method calls may also
@@ -98,8 +107,9 @@ between conformable Python objects and XML on the wire.
98107
ensure that the string is free of characters that aren't allowed in XML, such as
99108
the control characters with ASCII values between 0 and 31 (except, of course,
100109
tab, newline and carriage return); failing to do this will result in an XML-RPC
101-
request that isn't well-formed XML. If you have to pass arbitrary strings via
102-
XML-RPC, use the :class:`Binary` wrapper class described below.
110+
request that isn't well-formed XML. If you have to pass arbitrary bytes
111+
via XML-RPC, use the :class:`bytes` class or the class:`Binary` wrapper class
112+
described below.
103113

104114
:class:`Server` is retained as an alias for :class:`ServerProxy` for backwards
105115
compatibility. New code should use :class:`ServerProxy`.
@@ -249,23 +259,23 @@ The client code for the preceding server::
249259
Binary Objects
250260
--------------
251261

252-
This class may be initialized from string data (which may include NULs). The
262+
This class may be initialized from bytes data (which may include NULs). The
253263
primary access to the content of a :class:`Binary` object is provided by an
254264
attribute:
255265

256266

257267
.. attribute:: Binary.data
258268

259269
The binary data encapsulated by the :class:`Binary` instance. The data is
260-
provided as an 8-bit string.
270+
provided as a :class:`bytes` object.
261271

262272
:class:`Binary` objects have the following methods, supported mainly for
263273
internal use by the marshalling/unmarshalling code:
264274

265275

266-
.. method:: Binary.decode(string)
276+
.. method:: Binary.decode(bytes)
267277

268-
Accept a base64 string and decode it as the instance's new data.
278+
Accept a base64 :class:`bytes` object and decode it as the instance's new data.
269279

270280

271281
.. method:: Binary.encode(out)
@@ -471,14 +481,21 @@ Convenience Functions
471481
it via an extension, provide a true value for *allow_none*.
472482

473483

474-
.. function:: loads(data, use_datetime=False)
484+
.. function:: loads(data, use_datetime=False, use_builtin_types=False)
475485

476486
Convert an XML-RPC request or response into Python objects, a ``(params,
477487
methodname)``. *params* is a tuple of argument; *methodname* is a string, or
478488
``None`` if no method name is present in the packet. If the XML-RPC packet
479489
represents a fault condition, this function will raise a :exc:`Fault` exception.
480-
The *use_datetime* flag can be used to cause date/time values to be presented as
481-
:class:`datetime.datetime` objects; this is false by default.
490+
The *use_builtin_types* flag can be used to cause date/time values to be
491+
presented as :class:`datetime.datetime` objects and binary data to be
492+
presented as :class:`bytes` objects; this flag is false by default.
493+
494+
The obsolete *use_datetime* flag is similar to *use_builtin_types* but it
495+
applies only to date/time values.
496+
497+
.. versionchanged:: 3.3
498+
The *use_builtin_types* flag was added.
482499

483500

484501
.. _xmlrpc-client-example:

‎Lib/test/test_xmlrpc.py‎

Lines changed: 57 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@
2424
'ashortlong': 2,
2525
'anotherlist': ['.zyx.41'],
2626
'abase64': xmlrpclib.Binary(b"my dog has fleas"),
27+
'b64bytes': b"my dog has fleas",
28+
'b64bytearray': bytearray(b"my dog has fleas"),
2729
'boolean': False,
2830
'unicode': '\u4000\u6000\u8000',
2931
'ukey\u4000': 'regular value',
@@ -44,27 +46,54 @@ def test_dump_load(self):
4446
def test_dump_bare_datetime(self):
4547
# This checks that an unwrapped datetime.date object can be handled
4648
# by the marshalling code. This can't be done via test_dump_load()
47-
# since with use_datetime set to 1 the unmarshaller would create
49+
# since with use_builtin_types set to 1 the unmarshaller would create
4850
# datetime objects for the 'datetime[123]' keys as well
4951
dt = datetime.datetime(2005, 2, 10, 11, 41, 23)
52+
self.assertEqual(dt, xmlrpclib.DateTime('20050210T11:41:23'))
5053
s = xmlrpclib.dumps((dt,))
51-
(newdt,), m = xmlrpclib.loads(s, use_datetime=1)
54+
55+
result, m = xmlrpclib.loads(s, use_builtin_types=True)
56+
(newdt,) = result
5257
self.assertEqual(newdt, dt)
53-
self.assertEqual(m, None)
58+
self.assertIs(type(newdt), datetime.datetime)
59+
self.assertIsNone(m)
60+
61+
result, m = xmlrpclib.loads(s, use_builtin_types=False)
62+
(newdt,) = result
63+
self.assertEqual(newdt, dt)
64+
self.assertIs(type(newdt), xmlrpclib.DateTime)
65+
self.assertIsNone(m)
66+
67+
result, m = xmlrpclib.loads(s, use_datetime=True)
68+
(newdt,) = result
69+
self.assertEqual(newdt, dt)
70+
self.assertIs(type(newdt), datetime.datetime)
71+
self.assertIsNone(m)
72+
73+
result, m = xmlrpclib.loads(s, use_datetime=False)
74+
(newdt,) = result
75+
self.assertEqual(newdt, dt)
76+
self.assertIs(type(newdt), xmlrpclib.DateTime)
77+
self.assertIsNone(m)
5478

55-
(newdt,), m = xmlrpclib.loads(s, use_datetime=0)
56-
self.assertEqual(newdt, xmlrpclib.DateTime('20050210T11:41:23'))
5779

5880
def test_datetime_before_1900(self):
5981
# same as before but with a date before 1900
6082
dt = datetime.datetime(1, 2, 10, 11, 41, 23)
83+
self.assertEqual(dt, xmlrpclib.DateTime('00010210T11:41:23'))
6184
s = xmlrpclib.dumps((dt,))
62-
(newdt,), m = xmlrpclib.loads(s, use_datetime=1)
85+
86+
result, m = xmlrpclib.loads(s, use_builtin_types=True)
87+
(newdt,) = result
6388
self.assertEqual(newdt, dt)
64-
self.assertEqual(m, None)
89+
self.assertIs(type(newdt), datetime.datetime)
90+
self.assertIsNone(m)
6591

66-
(newdt,), m = xmlrpclib.loads(s, use_datetime=0)
67-
self.assertEqual(newdt, xmlrpclib.DateTime('00010210T11:41:23'))
92+
result, m = xmlrpclib.loads(s, use_builtin_types=False)
93+
(newdt,) = result
94+
self.assertEqual(newdt, dt)
95+
self.assertIs(type(newdt), xmlrpclib.DateTime)
96+
self.assertIsNone(m)
6897

6998
def test_bug_1164912 (self):
7099
d = xmlrpclib.DateTime()
@@ -133,16 +162,32 @@ def test_dump_none(self):
133162
xmlrpclib.loads(strg)[0][0])
134163
self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,))
135164

165+
def test_dump_bytes(self):
166+
sample = b"my dog has fleas"
167+
self.assertEqual(sample, xmlrpclib.Binary(sample))
168+
for type_ in bytes, bytearray, xmlrpclib.Binary:
169+
value = type_(sample)
170+
s = xmlrpclib.dumps((value,))
171+
172+
result, m = xmlrpclib.loads(s, use_builtin_types=True)
173+
(newvalue,) = result
174+
self.assertEqual(newvalue, sample)
175+
self.assertIs(type(newvalue), bytes)
176+
self.assertIsNone(m)
177+
178+
result, m = xmlrpclib.loads(s, use_builtin_types=False)
179+
(newvalue,) = result
180+
self.assertEqual(newvalue, sample)
181+
self.assertIs(type(newvalue), xmlrpclib.Binary)
182+
self.assertIsNone(m)
183+
136184
def test_get_host_info(self):
137185
# see bug #3613, this raised a TypeError
138186
transp = xmlrpc.client.Transport()
139187
self.assertEqual(transp.get_host_info("[email protected]"),
140188
('host.tld',
141189
[('Authorization', 'Basic dXNlcg==')], {}))
142190

143-
def test_dump_bytes(self):
144-
self.assertRaises(TypeError, xmlrpclib.dumps, (b"my dog has fleas",))
145-
146191
def test_ssl_presence(self):
147192
try:
148193
import ssl

‎Lib/xmlrpc/client.py‎

Lines changed: 36 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -386,8 +386,8 @@ def __init__(self, data=None):
386386
if data is None:
387387
data = b""
388388
else:
389-
if not isinstance(data, bytes):
390-
raise TypeError("expected bytes, not %s" %
389+
if not isinstance(data, (bytes, bytearray)):
390+
raise TypeError("expected bytes or bytearray, not %s" %
391391
data.__class__.__name__)
392392
data = bytes(data) # Make a copy of the bytes!
393393
self.data = data
@@ -559,6 +559,14 @@ def dump_unicode(self, value, write, escape=escape):
559559
write("</string></value>\n")
560560
dispatch[str] = dump_unicode
561561

562+
def dump_bytes(self, value, write):
563+
write("<value><base64>\n")
564+
encoded = base64.encodebytes(value)
565+
write(encoded.decode('ascii'))
566+
write("</base64></value>\n")
567+
dispatch[bytes] = dump_bytes
568+
dispatch[bytearray] = dump_bytes
569+
562570
def dump_array(self, value, write):
563571
i = id(value)
564572
if i in self.memo:
@@ -629,15 +637,16 @@ class Unmarshaller:
629637
# and again, if you don't understand what's going on in here,
630638
# that's perfectly ok.
631639

632-
def __init__(self, use_datetime=False):
640+
def __init__(self, use_datetime=False, use_builtin_types=False):
633641
self._type = None
634642
self._stack = []
635643
self._marks = []
636644
self._data = []
637645
self._methodname = None
638646
self._encoding = "utf-8"
639647
self.append = self._stack.append
640-
self._use_datetime = use_datetime
648+
self._use_datetime = use_builtin_types or use_datetime
649+
self._use_bytes = use_builtin_types
641650

642651
def close(self):
643652
# return response tuple and target method
@@ -749,6 +758,8 @@ def end_struct(self, data):
749758
def end_base64(self, data):
750759
value = Binary()
751760
value.decode(data.encode("ascii"))
761+
if self._use_bytes:
762+
value = value.data
752763
self.append(value)
753764
self._value = 0
754765
dispatch["base64"] = end_base64
@@ -860,21 +871,26 @@ def __call__(self):
860871
#
861872
# return A (parser, unmarshaller) tuple.
862873

863-
def getparser(use_datetime=False):
874+
def getparser(use_datetime=False, use_builtin_types=False):
864875
"""getparser() -> parser, unmarshaller
865876
866877
Create an instance of the fastest available parser, and attach it
867878
to an unmarshalling object. Return both objects.
868879
"""
869880
if FastParser and FastUnmarshaller:
870-
if use_datetime:
881+
if use_builtin_types:
882+
mkdatetime = _datetime_type
883+
mkbytes = base64.decodebytes
884+
elif use_datetime:
871885
mkdatetime = _datetime_type
886+
mkbytes = _binary
872887
else:
873888
mkdatetime = _datetime
874-
target = FastUnmarshaller(True, False, _binary, mkdatetime, Fault)
889+
mkbytes = _binary
890+
target = FastUnmarshaller(True, False, mkbytes, mkdatetime, Fault)
875891
parser = FastParser(target)
876892
else:
877-
target = Unmarshaller(use_datetime=use_datetime)
893+
target = Unmarshaller(use_datetime=use_datetime, use_builtin_types=use_builtin_types)
878894
if FastParser:
879895
parser = FastParser(target)
880896
else:
@@ -912,7 +928,7 @@ def dumps(params, methodname=None, methodresponse=None, encoding=None,
912928
913929
encoding: the packet encoding (default is UTF-8)
914930
915-
All 8-bit strings in the data structure are assumed to use the
931+
All byte strings in the data structure are assumed to use the
916932
packet encoding. Unicode strings are automatically converted,
917933
where necessary.
918934
"""
@@ -971,7 +987,7 @@ def dumps(params, methodname=None, methodresponse=None, encoding=None,
971987
# (None if not present).
972988
# @see Fault
973989

974-
def loads(data, use_datetime=False):
990+
def loads(data, use_datetime=False, use_builtin_types=False):
975991
"""data -> unmarshalled data, method name
976992
977993
Convert an XML-RPC packet to unmarshalled data plus a method
@@ -980,7 +996,7 @@ def loads(data, use_datetime=False):
980996
If the XML-RPC packet represents a fault condition, this function
981997
raises a Fault exception.
982998
"""
983-
p, u = getparser(use_datetime=use_datetime)
999+
p, u = getparser(use_datetime=use_datetime, use_builtin_types=use_builtin_types)
9841000
p.feed(data)
9851001
p.close()
9861002
return u.close(), u.getmethodname()
@@ -1092,8 +1108,9 @@ class Transport:
10921108
# that they can decode such a request
10931109
encode_threshold = None #None = don't encode
10941110

1095-
def __init__(self, use_datetime=False):
1111+
def __init__(self, use_datetime=False, use_builtin_types=False):
10961112
self._use_datetime = use_datetime
1113+
self._use_builtin_types = use_builtin_types
10971114
self._connection = (None, None)
10981115
self._extra_headers = []
10991116

@@ -1154,7 +1171,8 @@ def single_request(self, host, handler, request_body, verbose=False):
11541171

11551172
def getparser(self):
11561173
# get parser and unmarshaller
1157-
return getparser(use_datetime=self._use_datetime)
1174+
return getparser(use_datetime=self._use_datetime,
1175+
use_builtin_types=self._use_builtin_types)
11581176

11591177
##
11601178
# Get authorization info from host parameter
@@ -1361,7 +1379,7 @@ class ServerProxy:
13611379
"""
13621380

13631381
def __init__(self, uri, transport=None, encoding=None, verbose=False,
1364-
allow_none=False, use_datetime=False):
1382+
allow_none=False, use_datetime=False, use_builtin_types=False):
13651383
# establish a "logical" server connection
13661384

13671385
# get the url
@@ -1375,9 +1393,11 @@ def __init__(self, uri, transport=None, encoding=None, verbose=False,
13751393

13761394
if transport is None:
13771395
if type == "https":
1378-
transport = SafeTransport(use_datetime=use_datetime)
1396+
handler = SafeTransport
13791397
else:
1380-
transport = Transport(use_datetime=use_datetime)
1398+
handler = Transport
1399+
transport = handler(use_datetime=use_datetime,
1400+
use_builtin_types=use_builtin_types)
13811401
self.__transport = transport
13821402

13831403
self.__encoding = encoding or 'utf-8'

‎Misc/NEWS‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -374,6 +374,8 @@ Core and Builtins
374374
Library
375375
-------
376376

377+
- Issue #13297: Use bytes type to send and receive binary data through XMLRPC.
378+
377379
- Issue #6397: Support "/dev/poll" polling objects in select module,
378380
under Solaris & derivatives.
379381

0 commit comments

Comments
 (0)