Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 687c577

Browse files
msullivanmsullivan
authored
Support subclassing python classes (mypyc/mypyc#339)
I think we are going to want to support both options in mypyc/mypyc#296, and so this is work on option 1 of it. This is not fully finished code, but I wanted to ask for thoughts on it now. My approach for this is moving class creation into genops, evaluating the base classes, calling a prim to create the class, and saving it with a newly added `InitStatic` op. In addition to supporting subclassing python classes during the migration, we also want to be able to subclass `Generic[T]`. This turns out to be a huge pain, because pre-3.7 `Generic` is implemented using metaclasses and 3.7 `Generic` is implemented using some other language features we didn't support previously. We can't actually create out class using the metaclass constructor, so we allocate a *parallel* class with the metaclass and then copy the attributes over. This is pretty awful.
1 parent 6064133 commit 687c577

22 files changed

Lines changed: 867 additions & 139 deletions

‎LICENSE‎

Lines changed: 202 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Mypyc is licensed under the terms of the MIT license, reproduced below.
44

55
The MIT License
66

7-
Copyright (c) 2017 Jukka Lehtosalo and contributors
7+
Copyright (c) 2017-2018 Jukka Lehtosalo and contributors
88

99
Permission is hereby granted, free of charge, to any person obtaining a
1010
copy of this software and associated documentation files (the "Software"),
@@ -25,3 +25,204 @@ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
2525
DEALINGS IN THE SOFTWARE.
2626

2727
= = = = =
28+
29+
Portions of mypyc are licensed under different licenses. The file
30+
lib-rt/pythonsupport.h is licensed under the PSF 2 License, reproduced below.
31+
32+
= = = = =
33+
34+
PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
35+
--------------------------------------------
36+
37+
1. This LICENSE AGREEMENT is between the Python Software Foundation
38+
("PSF"), and the Individual or Organization ("Licensee") accessing and
39+
otherwise using this software ("Python") in source or binary form and
40+
its associated documentation.
41+
42+
2. Subject to the terms and conditions of this License Agreement, PSF hereby
43+
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
44+
analyze, test, perform and/or display publicly, prepare derivative works,
45+
distribute, and otherwise use Python alone or in any derivative version,
46+
provided, however, that PSF's License Agreement and PSF's notice of copyright,
47+
i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
48+
2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018 Python Software Foundation; All
49+
Rights Reserved" are retained in Python alone or in any derivative version
50+
prepared by Licensee.
51+
52+
3. In the event Licensee prepares a derivative work that is based on
53+
or incorporates Python or any part thereof, and wants to make
54+
the derivative work available to others as provided herein, then
55+
Licensee hereby agrees to include in any such work a brief summary of
56+
the changes made to Python.
57+
58+
4. PSF is making Python available to Licensee on an "AS IS"
59+
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
60+
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
61+
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
62+
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
63+
INFRINGE ANY THIRD PARTY RIGHTS.
64+
65+
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
66+
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
67+
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
68+
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
69+
70+
6. This License Agreement will automatically terminate upon a material
71+
breach of its terms and conditions.
72+
73+
7. Nothing in this License Agreement shall be deemed to create any
74+
relationship of agency, partnership, or joint venture between PSF and
75+
Licensee. This License Agreement does not grant permission to use PSF
76+
trademarks or trade name in a trademark sense to endorse or promote
77+
products or services of Licensee, or any third party.
78+
79+
8. By copying, installing or otherwise using Python, Licensee
80+
agrees to be bound by the terms and conditions of this License
81+
Agreement.
82+
83+
84+
BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
85+
-------------------------------------------
86+
87+
BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
88+
89+
1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
90+
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
91+
Individual or Organization ("Licensee") accessing and otherwise using
92+
this software in source or binary form and its associated
93+
documentation ("the Software").
94+
95+
2. Subject to the terms and conditions of this BeOpen Python License
96+
Agreement, BeOpen hereby grants Licensee a non-exclusive,
97+
royalty-free, world-wide license to reproduce, analyze, test, perform
98+
and/or display publicly, prepare derivative works, distribute, and
99+
otherwise use the Software alone or in any derivative version,
100+
provided, however, that the BeOpen Python License is retained in the
101+
Software, alone or in any derivative version prepared by Licensee.
102+
103+
3. BeOpen is making the Software available to Licensee on an "AS IS"
104+
basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
105+
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
106+
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
107+
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
108+
INFRINGE ANY THIRD PARTY RIGHTS.
109+
110+
4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
111+
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
112+
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
113+
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
114+
115+
5. This License Agreement will automatically terminate upon a material
116+
breach of its terms and conditions.
117+
118+
6. This License Agreement shall be governed by and interpreted in all
119+
respects by the law of the State of California, excluding conflict of
120+
law provisions. Nothing in this License Agreement shall be deemed to
121+
create any relationship of agency, partnership, or joint venture
122+
between BeOpen and Licensee. This License Agreement does not grant
123+
permission to use BeOpen trademarks or trade names in a trademark
124+
sense to endorse or promote products or services of Licensee, or any
125+
third party. As an exception, the "BeOpen Python" logos available at
126+
http://www.pythonlabs.com/logos.html may be used according to the
127+
permissions granted on that web page.
128+
129+
7. By copying, installing or otherwise using the software, Licensee
130+
agrees to be bound by the terms and conditions of this License
131+
Agreement.
132+
133+
134+
CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
135+
---------------------------------------
136+
137+
1. This LICENSE AGREEMENT is between the Corporation for National
138+
Research Initiatives, having an office at 1895 Preston White Drive,
139+
Reston, VA 20191 ("CNRI"), and the Individual or Organization
140+
("Licensee") accessing and otherwise using Python 1.6.1 software in
141+
source or binary form and its associated documentation.
142+
143+
2. Subject to the terms and conditions of this License Agreement, CNRI
144+
hereby grants Licensee a nonexclusive, royalty-free, world-wide
145+
license to reproduce, analyze, test, perform and/or display publicly,
146+
prepare derivative works, distribute, and otherwise use Python 1.6.1
147+
alone or in any derivative version, provided, however, that CNRI's
148+
License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
149+
1995-2001 Corporation for National Research Initiatives; All Rights
150+
Reserved" are retained in Python 1.6.1 alone or in any derivative
151+
version prepared by Licensee. Alternately, in lieu of CNRI's License
152+
Agreement, Licensee may substitute the following text (omitting the
153+
quotes): "Python 1.6.1 is made available subject to the terms and
154+
conditions in CNRI's License Agreement. This Agreement together with
155+
Python 1.6.1 may be located on the Internet using the following
156+
unique, persistent identifier (known as a handle): 1895.22/1013. This
157+
Agreement may also be obtained from a proxy server on the Internet
158+
using the following URL: http://hdl.handle.net/1895.22/1013".
159+
160+
3. In the event Licensee prepares a derivative work that is based on
161+
or incorporates Python 1.6.1 or any part thereof, and wants to make
162+
the derivative work available to others as provided herein, then
163+
Licensee hereby agrees to include in any such work a brief summary of
164+
the changes made to Python 1.6.1.
165+
166+
4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
167+
basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
168+
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
169+
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
170+
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
171+
INFRINGE ANY THIRD PARTY RIGHTS.
172+
173+
5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
174+
1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
175+
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
176+
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
177+
178+
6. This License Agreement will automatically terminate upon a material
179+
breach of its terms and conditions.
180+
181+
7. This License Agreement shall be governed by the federal
182+
intellectual property law of the United States, including without
183+
limitation the federal copyright law, and, to the extent such
184+
U.S. federal law does not apply, by the law of the Commonwealth of
185+
Virginia, excluding Virginia's conflict of law provisions.
186+
Notwithstanding the foregoing, with regard to derivative works based
187+
on Python 1.6.1 that incorporate non-separable material that was
188+
previously distributed under the GNU General Public License (GPL), the
189+
law of the Commonwealth of Virginia shall govern this License
190+
Agreement only as to issues arising under or with respect to
191+
Paragraphs 4, 5, and 7 of this License Agreement. Nothing in this
192+
License Agreement shall be deemed to create any relationship of
193+
agency, partnership, or joint venture between CNRI and Licensee. This
194+
License Agreement does not grant permission to use CNRI trademarks or
195+
trade name in a trademark sense to endorse or promote products or
196+
services of Licensee, or any third party.
197+
198+
8. By clicking on the "ACCEPT" button where indicated, or by copying,
199+
installing or otherwise using Python 1.6.1, Licensee agrees to be
200+
bound by the terms and conditions of this License Agreement.
201+
202+
ACCEPT
203+
204+
205+
CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
206+
--------------------------------------------------
207+
208+
Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
209+
The Netherlands. All rights reserved.
210+
211+
Permission to use, copy, modify, and distribute this software and its
212+
documentation for any purpose and without fee is hereby granted,
213+
provided that the above copyright notice appear in all copies and that
214+
both that copyright notice and this permission notice appear in
215+
supporting documentation, and that the name of Stichting Mathematisch
216+
Centrum or CWI not be used in advertising or publicity pertaining to
217+
distribution of the software without specific, written prior
218+
permission.
219+
220+
STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
221+
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
222+
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
223+
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
224+
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
225+
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
226+
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
227+
228+
= = = = =

‎lib-rt/CPy.h‎

Lines changed: 102 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
#include <Python.h>
66
#include <frameobject.h>
77
#include <assert.h>
8+
#include "pythonsupport.h"
89

910
#ifdef __cplusplus
1011
extern "C" {
@@ -69,24 +70,118 @@ static inline void CPy_FixupTraitVtable(CPyVTableItem *vtable, int count) {
6970

7071
// Create a heap type based on a template non-heap type.
7172
// This is super hacky and maybe we should suck it up and use PyType_FromSpec instead.
72-
static inline PyTypeObject *CPyType_FromTemplate(PyTypeObject *template_,
73-
PyObject *modname) {
74-
PyHeapTypeObject *t = (PyHeapTypeObject*)PyType_GenericAlloc(&PyType_Type, 0);
73+
// We allow bases to be NULL to represent just inheriting from object.
74+
// We don't support NULL bases and a non-type metaclass.
75+
static inline PyObject *CPyType_FromTemplate(PyTypeObject *template_,
76+
PyObject *orig_bases,
77+
PyObject *modname) {
78+
PyHeapTypeObject *t = NULL;
79+
PyTypeObject *dummy_class = NULL;
80+
PyObject *name = NULL;
81+
PyObject *bases = NULL;
82+
83+
PyTypeObject *metaclass = Py_TYPE(template_);
84+
85+
if (orig_bases) {
86+
bases = update_bases(orig_bases);
87+
// update_bases doesn't increment the refcount if nothing changes,
88+
// so we do it to make sure we have distinct "references" to both
89+
if (bases == orig_bases)
90+
Py_INCREF(bases);
91+
92+
// Find the appropriate metaclass from our base classes. We
93+
// care about this because Generic uses a metaclass prior to
94+
// Python 3.7.
95+
metaclass = _PyType_CalculateMetaclass(metaclass, bases);
96+
if (!metaclass)
97+
goto error;
98+
}
99+
100+
name = PyUnicode_FromString(template_->tp_name);
101+
if (!name)
102+
goto error;
103+
104+
// If there is a metaclass other than type, we would like to call
105+
// its __new__ function. Unfortunately there doesn't seem to be a
106+
// good way to mix a C extension class and creating it via a
107+
// metaclass. We need to do it anyways, though, in order to
108+
// support subclassing Generic[T] prior to Python 3.7.
109+
//
110+
// We solve this with a kind of atrocious hack: create a parallel
111+
// class using the metaclass, determine the bases of the real
112+
// class by pulling them out of the parallel class, creating the
113+
// real class, and then merging its dict back into the original
114+
// class. There are lots of cases where this won't really work,
115+
// but for the case of GenericMeta setting a bunch of properties
116+
// on the class we should be fine.
117+
if (metaclass != &PyType_Type) {
118+
assert(bases && "non-type metaclasses require non-NULL bases");
119+
120+
PyObject *ns = PyDict_New();
121+
if (!ns)
122+
goto error;
123+
124+
dummy_class = (PyTypeObject *)PyObject_CallFunctionObjArgs(
125+
(PyObject *)metaclass, name, bases, ns, NULL);
126+
Py_DECREF(ns);
127+
if (!dummy_class)
128+
goto error;
129+
130+
Py_DECREF(bases);
131+
bases = dummy_class->tp_bases;
132+
Py_INCREF(bases);
133+
}
134+
135+
// Allocate the type and then copy the main stuff in.
136+
t = (PyHeapTypeObject*)PyType_GenericAlloc(metaclass, 0);
137+
if (!t)
138+
goto error;
75139
memcpy((char *)t + sizeof(PyVarObject),
76140
(char *)template_ + sizeof(PyVarObject),
77141
sizeof(PyTypeObject) - sizeof(PyVarObject));
78142

79-
PyObject *name = PyUnicode_FromString(template_->tp_name);
143+
if (bases != orig_bases) {
144+
if (PyObject_SetAttrString((PyObject *)t, "__orig_bases__", orig_bases) < 0)
145+
goto error;
146+
}
147+
80148
t->ht_name = name;
81149
Py_INCREF(name);
82150
t->ht_qualname = name;
151+
t->ht_type.tp_bases = bases;
152+
// references stolen so NULL these out
153+
bases = name = NULL;
83154

84155
if (PyType_Ready((PyTypeObject *)t) < 0)
85-
return NULL;
156+
goto error;
157+
158+
if (dummy_class) {
159+
if (PyDict_Merge(t->ht_type.tp_dict, dummy_class->tp_dict, 0) != 0)
160+
goto error;
161+
// This is the *really* tasteless bit. GenericMeta's __new__
162+
// in certain versions of typing sets _gorg to point back to
163+
// the class. We need to override it to keep it from pointing
164+
// to the proxy.
165+
if (PyDict_SetItemString(t->ht_type.tp_dict, "_gorg", (PyObject *)t) < 0)
166+
goto error;
167+
}
168+
169+
if (PyObject_SetAttrString((PyObject *)t, "__module__", modname) < 0)
170+
goto error;
171+
172+
if (init_subclass((PyTypeObject *)t, NULL))
173+
goto error;
174+
175+
Py_XDECREF(dummy_class);
86176

87-
PyObject_SetAttrString((PyObject *)t, "__module__", modname);
177+
return (PyObject *)t;
88178

89-
return (PyTypeObject *)t;
179+
error:
180+
Py_XDECREF(t);
181+
Py_XDECREF(bases);
182+
Py_XDECREF(dummy_class);
183+
Py_XDECREF(name);
184+
return NULL;
90185
}
91186

92187
// Get attribute value using vtable (may return an undefined value)

0 commit comments

Comments
 (0)