The public C API PyGen_New() is implemented by calling gen_new_with_qualname().
gen_new_with_qualname() code evolved in Python 3.11:
- Py_INCREF(f->f_code);
- gen->gi_code = (PyObject *)(f->f_code);
+ gen->gi_code = PyFrame_GetCode(f);
gen->gi_code = PyFrame_GetCode(f);
+ Py_INCREF(gen->gi_code);
+ Py_DECREF(f);
I'm surprised by the added Py_INCREF(gen->gi_code); since PyFrame_GetCode() already returns a strong reference. For me, it's a reference leak.
I tried but failed to write an unit test on the PyGen_New() function. I wrote the test:
static PyObject *
test_gen_new(PyObject *self, PyObject *Py_UNUSED(args))
{
PyCodeObject *code = PyCode_NewEmpty("filename", "func_name", 1);
if (code == NULL) {
return NULL;
}
PyObject *globals = PyDict_New();
if (globals == NULL) {
return NULL;
}
PyObject *locals = globals; // borrowed ref
PyThreadState *tstate = PyThreadState_Get();
PyFrameObject *frame = PyFrame_New(tstate, code, globals, locals);
if (frame == NULL) {
return NULL;
}
PyObject *gen = PyGen_New(frame);
Py_DECREF(gen);
Py_DECREF(code);
Py_DECREF(globals);
Py_DECREF(frame);
Py_RETURN_NONE;
}But when I run it, it fails with an assertion error:
python: Objects/genobject.c:937: gen_new_with_qualname: Assertion `frame->frame_obj == f' failed.
Extract of the code. I added newlines and "HERE" comment around the code causing the assertion error:
/* Copy the frame */
assert(f->f_frame->frame_obj == NULL);
assert(f->f_frame->owner == FRAME_OWNED_BY_FRAME_OBJECT);
_PyInterpreterFrame *frame = (_PyInterpreterFrame *)gen->gi_iframe;
_PyFrame_Copy((_PyInterpreterFrame *)f->_f_frame_data, frame);
gen->gi_frame_state = FRAME_CREATED;
assert(frame->frame_obj == f); // <=== HERE
f->f_frame = frame;
frame->owner = FRAME_OWNED_BY_GENERATOR;
assert(PyObject_GC_IsTracked((PyObject *)f));
gen->gi_code = PyFrame_GetCode(f);
Py_DECREF(f);But the problem doesn't come from _PyFrame_Copy(). If I added the following assertion, it also fails:
assert(f->f_frame->frame_obj == f);
I am not sure if PyFrame_New() is supposed to create a frame with frame->f_frame->frame_obj == frame or not.
Linked PRs
The public C API PyGen_New() is implemented by calling gen_new_with_qualname().
gen_new_with_qualname() code evolved in Python 3.11:
I'm surprised by the added
Py_INCREF(gen->gi_code);sincePyFrame_GetCode()already returns a strong reference. For me, it's a reference leak.I tried but failed to write an unit test on the PyGen_New() function. I wrote the test:
But when I run it, it fails with an assertion error:
Extract of the code. I added newlines and "HERE" comment around the code causing the assertion error:
But the problem doesn't come from
_PyFrame_Copy(). If I added the following assertion, it also fails:I am not sure if PyFrame_New() is supposed to create a frame with
frame->f_frame->frame_obj == frameor not.Linked PRs