Codestin Search App

History

1389 lines (1326 loc) · 138 KB

Raw

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

854

855

856

857

858

859

860

861

862

863

864

865

866

867

868

869

870

871

872

873

874

875

876

877

878

879

880

881

882

883

884

885

886

887

888

889

890

891

892

893

894

895

896

897

898

899

900

901

902

903

904

905

906

907

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969

970

971

972

973

974

975

976

977

978

979

980

981

982

983

984

985

986

987

988

989

990

991

992

993

994

995

996

997

998

999

1000

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<head>

<title>1. 以 C 或 C++ 擴充 Python — Python 3.7.0 說明文件</title>

title="在 Python 3.7.0 說明文件中搜尋"

href="../_static/opensearch.xml"/>

</head><body>

<ul>

<a href="../genindex.html" title="General Index"

accesskey="I">索引</a></li>

<a href="../py-modindex.html" title="Python 模組索引"

>模組</a> |</li>

<a href="newtypes_tutorial.html" title="2. Defining Extension Types: Tutorial"

accesskey="N">下一頁</a> |</li>

<a href="index.html" title="Extending and Embedding the Python Interpreter"

accesskey="P">上一頁</a> |</li>

<li><img src="../_static/py.png" alt=""

style="vertical-align: middle; margin-top: -1px"/></li>

<li><a href="https://www.python.org/">Python</a> »</li>

<li>

zh_TW

3.7.0

<a href="../index.html">Documentation </a> »

</li>

<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Extending and Embedding the Python Interpreter</a> »</li>

</form>

</div>

</li>

</ul>

</div>

<h1>1. 以 C 或 C++ 擴充 Python<a class="headerlink" href="#extending-python-with-c-or-c" title="本標題的永久連結">¶</a></h1>

It is quite easy to add new built-in modules to Python, if you know how to

program in C. Such extension modules can do two things that can’t be

done directly in Python: they can implement new built-in object types, and they

can call C library functions and system calls.

To support extensions, the Python API (Application Programmers Interface)

defines a set of functions, macros and variables that provide access to most

aspects of the Python run-time system. The Python API is incorporated in a C

source file by including the header <code class="docutils literal notranslate">"Python.h"</code>.

The compilation of an extension module depends on its intended use as well as on

your system setup; details are given in later chapters.

備註

The C extension interface is specific to CPython, and extension modules do

not work on other Python implementations. In many cases, it is possible to

avoid writing C extensions and preserve portability to other implementations.

For example, if your use case is calling C library functions or system calls,

you should consider using the <a class="reference internal" href="../library/ctypes.html#module-ctypes" title="ctypes: A foreign function library for Python."><code class="xref py py-mod docutils literal notranslate">ctypes</code></a> module or the <a class="reference external" href="https://cffi.readthedocs.io/">cffi</a> library rather than writing

custom C code.

These modules let you write Python code to interface with C code and are more

portable between implementations of Python than writing and compiling a C

extension module.

</div>

<h2>1.1. A Simple Example<a class="headerlink" href="#a-simple-example" title="本標題的永久連結">¶</a></h2>

Let’s create an extension module called <code class="docutils literal notranslate">spam</code> (the favorite food of Monty

Python fans…) and let’s say we want to create a Python interface to the C

library function <code class="xref c c-func docutils literal notranslate">system()</code> <a class="footnote-reference" href="#id5" id="id1">[1]</a>. This function takes a null-terminated

character string as argument and returns an integer. We want this function to

be callable from Python as follows:

<div class="highlight-pycon notranslate"><div class="highlight"><pre>>>> import spam

>>> status = spam.system("ls -l")

</pre></div>

</div>

Begin by creating a file <code class="file docutils literal notranslate">spammodule.c</code>. (Historically, if a module is

called <code class="docutils literal notranslate">spam</code>, the C file containing its implementation is called

<code class="file docutils literal notranslate">spammodule.c</code>; if the module name is very long, like <code class="docutils literal notranslate">spammify</code>, the

module name can be just <code class="file docutils literal notranslate">spammify.c</code>.)

The first line of our file can be:

<div class="highlight-c notranslate"><div class="highlight"><pre>#include <Python.h>

</pre></div>

</div>

which pulls in the Python API (you can add a comment describing the purpose of

the module and a copyright notice if you like).

備註

Since Python may define some pre-processor definitions which affect the standard

headers on some systems, you must include <code class="file docutils literal notranslate">Python.h</code> before any standard

headers are included.

</div>

All user-visible symbols defined by <code class="file docutils literal notranslate">Python.h</code> have a prefix of <code class="docutils literal notranslate">Py</code> or

<code class="docutils literal notranslate">PY</code>, except those defined in standard header files. For convenience, and

since they are used extensively by the Python interpreter, <code class="docutils literal notranslate">"Python.h"</code>

includes a few standard header files: <code class="docutils literal notranslate"><stdio.h></code>, <code class="docutils literal notranslate"><string.h></code>,

<code class="docutils literal notranslate"><errno.h></code>, and <code class="docutils literal notranslate"><stdlib.h></code>. If the latter header file does not exist on

your system, it declares the functions <code class="xref c c-func docutils literal notranslate">malloc()</code>, <code class="xref c c-func docutils literal notranslate">free()</code> and

<code class="xref c c-func docutils literal notranslate">realloc()</code> directly.

The next thing we add to our module file is the C function that will be called

when the Python expression <code class="docutils literal notranslate">spam.system(string)</code> is evaluated (we’ll see

shortly how it ends up being called):

<div class="highlight-c notranslate"><div class="highlight"><pre>static PyObject *

spam_system(PyObject *self, PyObject *args)

{

const char *command;

int sts;

if (!PyArg_ParseTuple(args, "s", &command))

return NULL;

sts = system(command);

return PyLong_FromLong(sts);

}

</pre></div>

</div>

There is a straightforward translation from the argument list in Python (for

example, the single expression <code class="docutils literal notranslate">"ls -l"</code>) to the arguments passed to the C

function. The C function always has two arguments, conventionally named self

and args.

The self argument points to the module object for module-level functions;

for a method it would point to the object instance.

The args argument will be a pointer to a Python tuple object containing the

arguments. Each item of the tuple corresponds to an argument in the call’s

argument list. The arguments are Python objects — in order to do anything

with them in our C function we have to convert them to C values. The function

<a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a> in the Python API checks the argument types and

converts them to C values. It uses a template string to determine the required

types of the arguments as well as the types of the C variables into which to

store the converted values. More about this later.

<a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a> returns true (nonzero) if all arguments have the right

type and its components have been stored in the variables whose addresses are

passed. It returns false (zero) if an invalid argument list was passed. In the

latter case it also raises an appropriate exception so the calling function can

return NULL immediately (as we saw in the example).

</div>

<h2>1.2. Intermezzo: Errors and Exceptions<a class="headerlink" href="#intermezzo-errors-and-exceptions" title="本標題的永久連結">¶</a></h2>

An important convention throughout the Python interpreter is the following: when

a function fails, it should set an exception condition and return an error value

(usually a NULL pointer). Exceptions are stored in a static global variable

inside the interpreter; if this variable is NULL no exception has occurred. A

second global variable stores the 「associated value」 of the exception (the

second argument to <a class="reference internal" href="../reference/simple_stmts.html#raise"><code class="xref std std-keyword docutils literal notranslate">raise</code></a>). A third variable contains the stack

traceback in case the error originated in Python code. These three variables

are the C equivalents of the result in Python of <a class="reference internal" href="../library/sys.html#sys.exc_info" title="sys.exc_info"><code class="xref py py-meth docutils literal notranslate">sys.exc_info()</code></a> (see the

section on module <a class="reference internal" href="../library/sys.html#module-sys" title="sys: Access system-specific parameters and functions."><code class="xref py py-mod docutils literal notranslate">sys</code></a> in the Python Library Reference). It is important

to know about them to understand how errors are passed around.

The Python API defines a number of functions to set various types of exceptions.

The most common one is <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_SetString" title="PyErr_SetString"><code class="xref c c-func docutils literal notranslate">PyErr_SetString()</code></a>. Its arguments are an exception

object and a C string. The exception object is usually a predefined object like

<code class="xref c c-data docutils literal notranslate">PyExc_ZeroDivisionError</code>. The C string indicates the cause of the error

and is converted to a Python string object and stored as the 「associated value」

of the exception.

Another useful function is <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_SetFromErrno" title="PyErr_SetFromErrno"><code class="xref c c-func docutils literal notranslate">PyErr_SetFromErrno()</code></a>, which only takes an

exception argument and constructs the associated value by inspection of the

global variable <code class="xref c c-data docutils literal notranslate">errno</code>. The most general function is

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_SetObject" title="PyErr_SetObject"><code class="xref c c-func docutils literal notranslate">PyErr_SetObject()</code></a>, which takes two object arguments, the exception and

its associated value. You don’t need to <a class="reference internal" href="../c-api/refcounting.html#c.Py_INCREF" title="Py_INCREF"><code class="xref c c-func docutils literal notranslate">Py_INCREF()</code></a> the objects passed

to any of these functions.

You can test non-destructively whether an exception has been set with

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Occurred" title="PyErr_Occurred"><code class="xref c c-func docutils literal notranslate">PyErr_Occurred()</code></a>. This returns the current exception object, or NULL

if no exception has occurred. You normally don’t need to call

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Occurred" title="PyErr_Occurred"><code class="xref c c-func docutils literal notranslate">PyErr_Occurred()</code></a> to see whether an error occurred in a function call,

since you should be able to tell from the return value.

When a function f that calls another function g detects that the latter

fails, f should itself return an error value (usually NULL or <code class="docutils literal notranslate">-1</code>). It

should not call one of the <code class="xref c c-func docutils literal notranslate">PyErr_*()</code> functions — one has already

been called by g. f’s caller is then supposed to also return an error

indication to its caller, again without calling <code class="xref c c-func docutils literal notranslate">PyErr_*()</code>, and so on

— the most detailed cause of the error was already reported by the function

that first detected it. Once the error reaches the Python interpreter’s main

loop, this aborts the currently executing Python code and tries to find an

exception handler specified by the Python programmer.

(There are situations where a module can actually give a more detailed error

message by calling another <code class="xref c c-func docutils literal notranslate">PyErr_*()</code> function, and in such cases it is

fine to do so. As a general rule, however, this is not necessary, and can cause

information about the cause of the error to be lost: most operations can fail

for a variety of reasons.)

To ignore an exception set by a function call that failed, the exception

condition must be cleared explicitly by calling <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Clear" title="PyErr_Clear"><code class="xref c c-func docutils literal notranslate">PyErr_Clear()</code></a>. The only

time C code should call <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Clear" title="PyErr_Clear"><code class="xref c c-func docutils literal notranslate">PyErr_Clear()</code></a> is if it doesn’t want to pass the

error on to the interpreter but wants to handle it completely by itself

(possibly by trying something else, or pretending nothing went wrong).

Every failing <code class="xref c c-func docutils literal notranslate">malloc()</code> call must be turned into an exception — the

direct caller of <code class="xref c c-func docutils literal notranslate">malloc()</code> (or <code class="xref c c-func docutils literal notranslate">realloc()</code>) must call

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_NoMemory" title="PyErr_NoMemory"><code class="xref c c-func docutils literal notranslate">PyErr_NoMemory()</code></a> and return a failure indicator itself. All the

object-creating functions (for example, <a class="reference internal" href="../c-api/long.html#c.PyLong_FromLong" title="PyLong_FromLong"><code class="xref c c-func docutils literal notranslate">PyLong_FromLong()</code></a>) already do

this, so this note is only relevant to those who call <code class="xref c c-func docutils literal notranslate">malloc()</code> directly.

Also note that, with the important exception of <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a> and

friends, functions that return an integer status usually return a positive value

or zero for success and <code class="docutils literal notranslate">-1</code> for failure, like Unix system calls.

Finally, be careful to clean up garbage (by making <a class="reference internal" href="../c-api/refcounting.html#c.Py_XDECREF" title="Py_XDECREF"><code class="xref c c-func docutils literal notranslate">Py_XDECREF()</code></a> or

<a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a> calls for objects you have already created) when you return

an error indicator!

The choice of which exception to raise is entirely yours. There are predeclared

C objects corresponding to all built-in Python exceptions, such as

<code class="xref c c-data docutils literal notranslate">PyExc_ZeroDivisionError</code>, which you can use directly. Of course, you

should choose exceptions wisely — don’t use <code class="xref c c-data docutils literal notranslate">PyExc_TypeError</code> to mean

that a file couldn’t be opened (that should probably be <code class="xref c c-data docutils literal notranslate">PyExc_IOError</code>).

If something’s wrong with the argument list, the <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a>

function usually raises <code class="xref c c-data docutils literal notranslate">PyExc_TypeError</code>. If you have an argument whose

value must be in a particular range or must satisfy other conditions,

<code class="xref c c-data docutils literal notranslate">PyExc_ValueError</code> is appropriate.

You can also define a new exception that is unique to your module. For this, you

usually declare a static object variable at the beginning of your file:

<div class="highlight-c notranslate"><div class="highlight"><pre>static PyObject *SpamError;

</pre></div>

</div>

and initialize it in your module’s initialization function (<code class="xref c c-func docutils literal notranslate">PyInit_spam()</code>)

with an exception object (leaving out the error checking for now):

<div class="highlight-c notranslate"><div class="highlight"><pre>PyMODINIT_FUNC

PyInit_spam(void)

{

PyObject *m;

m = PyModule_Create(&spammodule);

if (m == NULL)

return NULL;

SpamError = PyErr_NewException("spam.error", NULL, NULL);

Py_INCREF(SpamError);

PyModule_AddObject(m, "error", SpamError);

return m;

}

</pre></div>

</div>

Note that the Python name for the exception object is <code class="xref py py-exc docutils literal notranslate">spam.error</code>. The

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_NewException" title="PyErr_NewException"><code class="xref c c-func docutils literal notranslate">PyErr_NewException()</code></a> function may create a class with the base class

being <a class="reference internal" href="../library/exceptions.html#Exception" title="Exception"><code class="xref py py-exc docutils literal notranslate">Exception</code></a> (unless another class is passed in instead of NULL),

described in <a class="reference internal" href="../library/exceptions.html#bltin-exceptions">內建的例外</a>.

Note also that the <code class="xref c c-data docutils literal notranslate">SpamError</code> variable retains a reference to the newly

created exception class; this is intentional! Since the exception could be

removed from the module by external code, an owned reference to the class is

needed to ensure that it will not be discarded, causing <code class="xref c c-data docutils literal notranslate">SpamError</code> to

become a dangling pointer. Should it become a dangling pointer, C code which

raises the exception could cause a core dump or other unintended side effects.

We discuss the use of <code class="docutils literal notranslate">PyMODINIT_FUNC</code> as a function return type later in this

sample.

The <code class="xref py py-exc docutils literal notranslate">spam.error</code> exception can be raised in your extension module using a

call to <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_SetString" title="PyErr_SetString"><code class="xref c c-func docutils literal notranslate">PyErr_SetString()</code></a> as shown below:

<div class="highlight-c notranslate"><div class="highlight"><pre>static PyObject *

{

const char *command;

int sts;

return NULL;

sts = system(command);

if (sts < 0) {

PyErr_SetString(SpamError, "System command failed");

return NULL;

}

return PyLong_FromLong(sts);

}

</pre></div>

</div>

<h2>1.3. Back to the Example<a class="headerlink" href="#back-to-the-example" title="本標題的永久連結">¶</a></h2>

Going back to our example function, you should now be able to understand this

statement:

<div class="highlight-c notranslate"><div class="highlight"><pre>if (!PyArg_ParseTuple(args, "s", &command))

return NULL;

</pre></div>

</div>

It returns NULL (the error indicator for functions returning object pointers)

if an error is detected in the argument list, relying on the exception set by

copied to the local variable <code class="xref c c-data docutils literal notranslate">command</code>. This is a pointer assignment and

you are not supposed to modify the string to which it points (so in Standard C,

the variable <code class="xref c c-data docutils literal notranslate">command</code> should properly be declared as <code class="docutils literal notranslate">const char

*command</code>).

The next statement is a call to the Unix function <code class="xref c c-func docutils literal notranslate">system()</code>, passing it

the string we just got from <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a>:

<div class="highlight-c notranslate"><div class="highlight"><pre>sts = system(command);

</pre></div>

</div>

Our <code class="xref py py-func docutils literal notranslate">spam.system()</code> function must return the value of <code class="xref c c-data docutils literal notranslate">sts</code> as a

Python object. This is done using the function <a class="reference internal" href="../c-api/long.html#c.PyLong_FromLong" title="PyLong_FromLong"><code class="xref c c-func docutils literal notranslate">PyLong_FromLong()</code></a>.

<div class="highlight-c notranslate"><div class="highlight"><pre>return PyLong_FromLong(sts);

</pre></div>

</div>

In this case, it will return an integer object. (Yes, even integers are objects

on the heap in Python!)

If you have a C function that returns no useful argument (a function returning

<code class="xref c c-type docutils literal notranslate">void</code>), the corresponding Python function must return <code class="docutils literal notranslate">None</code>. You

need this idiom to do so (which is implemented by the <a class="reference internal" href="../c-api/none.html#c.Py_RETURN_NONE" title="Py_RETURN_NONE"><code class="xref c c-macro docutils literal notranslate">Py_RETURN_NONE</code></a>

macro):

<div class="highlight-c notranslate"><div class="highlight"><pre>Py_INCREF(Py_None);

return Py_None;

</pre></div>

</div>

<a class="reference internal" href="../c-api/none.html#c.Py_None" title="Py_None"><code class="xref c c-data docutils literal notranslate">Py_None</code></a> is the C name for the special Python object <code class="docutils literal notranslate">None</code>. It is a

genuine Python object rather than a NULL pointer, which means 「error」 in most

contexts, as we have seen.

</div>

<h2>1.4. The Module’s Method Table and Initialization Function<a class="headerlink" href="#the-module-s-method-table-and-initialization-function" title="本標題的永久連結">¶</a></h2>

I promised to show how <code class="xref c c-func docutils literal notranslate">spam_system()</code> is called from Python programs.

First, we need to list its name and address in a 「method table」:

<div class="highlight-c notranslate"><div class="highlight"><pre>static PyMethodDef SpamMethods[] = {

...

{"system", spam_system, METH_VARARGS,

"Execute a shell command."},

...

{NULL, NULL, 0, NULL} /* Sentinel */

};

</pre></div>

</div>

Note the third entry (<code class="docutils literal notranslate">METH_VARARGS</code>). This is a flag telling the interpreter

the calling convention to be used for the C function. It should normally always

be <code class="docutils literal notranslate">METH_VARARGS</code> or <code class="docutils literal notranslate">METH_VARARGS | METH_KEYWORDS</code>; a value of <code class="docutils literal notranslate">0</code> means

that an obsolete variant of <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a> is used.

When using only <code class="docutils literal notranslate">METH_VARARGS</code>, the function should expect the Python-level

parameters to be passed in as a tuple acceptable for parsing via

The <a class="reference internal" href="../c-api/structures.html#METH_KEYWORDS" title="METH_KEYWORDS"><code class="xref py py-const docutils literal notranslate">METH_KEYWORDS</code></a> bit may be set in the third field if keyword

arguments should be passed to the function. In this case, the C function should

accept a third <code class="docutils literal notranslate">PyObject *</code> parameter which will be a dictionary of keywords.

Use <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTupleAndKeywords" title="PyArg_ParseTupleAndKeywords"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code></a> to parse the arguments to such a

function.

The method table must be referenced in the module definition structure:

<div class="highlight-c notranslate"><div class="highlight"><pre>static struct PyModuleDef spammodule = {

PyModuleDef_HEAD_INIT,

"spam", /* name of module */

spam_doc, /* module documentation, may be NULL */

-1, /* size of per-interpreter state of the module,

or -1 if the module keeps state in global variables. */

SpamMethods

};

</pre></div>

</div>

This structure, in turn, must be passed to the interpreter in the module’s

initialization function. The initialization function must be named

<code class="xref c c-func docutils literal notranslate">PyInit_name()</code>, where name is the name of the module, and should be the

only non-<code class="docutils literal notranslate">static</code> item defined in the module file:

<div class="highlight-c notranslate"><div class="highlight"><pre>PyMODINIT_FUNC

PyInit_spam(void)

{

return PyModule_Create(&spammodule);

}

</pre></div>

</div>

Note that PyMODINIT_FUNC declares the function as <code class="docutils literal notranslate">PyObject *</code> return type,

declares any special linkage declarations required by the platform, and for C++

declares the function as <code class="docutils literal notranslate">extern "C"</code>.

When the Python program imports module <code class="xref py py-mod docutils literal notranslate">spam</code> for the first time,

<code class="xref c c-func docutils literal notranslate">PyInit_spam()</code> is called. (See below for comments about embedding Python.)

It calls <a class="reference internal" href="../c-api/module.html#c.PyModule_Create" title="PyModule_Create"><code class="xref c c-func docutils literal notranslate">PyModule_Create()</code></a>, which returns a module object, and

inserts built-in function objects into the newly created module based upon the

table (an array of <a class="reference internal" href="../c-api/structures.html#c.PyMethodDef" title="PyMethodDef"><code class="xref c c-type docutils literal notranslate">PyMethodDef</code></a> structures) found in the module definition.

<a class="reference internal" href="../c-api/module.html#c.PyModule_Create" title="PyModule_Create"><code class="xref c c-func docutils literal notranslate">PyModule_Create()</code></a> returns a pointer to the module object

that it creates. It may abort with a fatal error for

certain errors, or return NULL if the module could not be initialized

satisfactorily. The init function must return the module object to its caller,

so that it then gets inserted into <code class="docutils literal notranslate">sys.modules</code>.

When embedding Python, the <code class="xref c c-func docutils literal notranslate">PyInit_spam()</code> function is not called

automatically unless there’s an entry in the <code class="xref c c-data docutils literal notranslate">PyImport_Inittab</code> table.

To add the module to the initialization table, use <a class="reference internal" href="../c-api/import.html#c.PyImport_AppendInittab" title="PyImport_AppendInittab"><code class="xref c c-func docutils literal notranslate">PyImport_AppendInittab()</code></a>,

optionally followed by an import of the module:

<div class="highlight-c notranslate"><div class="highlight"><pre>int

main(int argc, char *argv[])

{

wchar_t *program = Py_DecodeLocale(argv[0], NULL);

if (program == NULL) {

fprintf(stderr, "Fatal error: cannot decode argv[0]\n");

exit(1);

}

/* Add a built-in module, before Py_Initialize */

PyImport_AppendInittab("spam", PyInit_spam);

/* Pass argv[0] to the Python interpreter */

Py_SetProgramName(program);

/* Initialize the Python interpreter. Required. */

Py_Initialize();

/* Optionally import the module; alternatively,

import can be deferred until the embedded script

imports it. */

PyImport_ImportModule("spam");

...

PyMem_RawFree(program);

return 0;

}

</pre></div>

</div>

備註

Removing entries from <code class="docutils literal notranslate">sys.modules</code> or importing compiled modules into

multiple interpreters within a process (or following a <code class="xref c c-func docutils literal notranslate">fork()</code> without an

intervening <code class="xref c c-func docutils literal notranslate">exec()</code>) can create problems for some extension modules.

Extension module authors should exercise caution when initializing internal data

structures.

</div>

A more substantial example module is included in the Python source distribution

as <code class="file docutils literal notranslate">Modules/xxmodule.c</code>. This file may be used as a template or simply

read as an example.

備註

Unlike our <code class="docutils literal notranslate">spam</code> example, <code class="docutils literal notranslate">xxmodule</code> uses multi-phase initialization

(new in Python 3.5), where a PyModuleDef structure is returned from

<code class="docutils literal notranslate">PyInit_spam</code>, and creation of the module is left to the import machinery.

For details on multi-phase initialization, see <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0489">PEP 489</a>.

</div>

<h2>1.5. Compilation and Linkage<a class="headerlink" href="#compilation-and-linkage" title="本標題的永久連結">¶</a></h2>

There are two more things to do before you can use your new extension: compiling

and linking it with the Python system. If you use dynamic loading, the details

may depend on the style of dynamic loading your system uses; see the chapters

about building extension modules (chapter <a class="reference internal" href="building.html#building">Building C and C++ Extensions</a>) and additional

information that pertains only to building on Windows (chapter

<a class="reference internal" href="windows.html#building-on-windows">Building C and C++ Extensions on Windows</a>) for more information about this.

If you can’t use dynamic loading, or if you want to make your module a permanent

part of the Python interpreter, you will have to change the configuration setup

and rebuild the interpreter. Luckily, this is very simple on Unix: just place

your file (<code class="file docutils literal notranslate">spammodule.c</code> for example) in the <code class="file docutils literal notranslate">Modules/</code> directory

of an unpacked source distribution, add a line to the file

<code class="file docutils literal notranslate">Modules/Setup.local</code> describing your file:

<div class="highlight-sh notranslate"><div class="highlight"><pre>spam spammodule.o

</pre></div>

</div>

and rebuild the interpreter by running make in the toplevel

directory. You can also run make in the <code class="file docutils literal notranslate">Modules/</code>

subdirectory, but then you must first rebuild <code class="file docutils literal notranslate">Makefile</code> there by running

『make Makefile』. (This is necessary each time you change the

<code class="file docutils literal notranslate">Setup</code> file.)

If your module requires additional libraries to link with, these can be listed

on the line in the configuration file as well, for instance:

<div class="highlight-sh notranslate"><div class="highlight"><pre>spam spammodule.o -lX11

</pre></div>

</div>

<h2>1.6. Calling Python Functions from C<a class="headerlink" href="#calling-python-functions-from-c" title="本標題的永久連結">¶</a></h2>

So far we have concentrated on making C functions callable from Python. The

reverse is also useful: calling Python functions from C. This is especially the

case for libraries that support so-called 「callback」 functions. If a C

interface makes use of callbacks, the equivalent Python often needs to provide a

callback mechanism to the Python programmer; the implementation will require

calling the Python callback functions from a C callback. Other uses are also

imaginable.

Fortunately, the Python interpreter is easily called recursively, and there is a

standard interface to call a Python function. (I won’t dwell on how to call the

Python parser with a particular string as input — if you’re interested, have a

look at the implementation of the <a class="reference internal" href="../using/cmdline.html#cmdoption-c"><code class="xref std std-option docutils literal notranslate">-c</code></a> command line option in

<code class="file docutils literal notranslate">Modules/main.c</code> from the Python source code.)

Calling a Python function is easy. First, the Python program must somehow pass

you the Python function object. You should provide a function (or some other

interface) to do this. When this function is called, save a pointer to the

Python function object (be careful to <a class="reference internal" href="../c-api/refcounting.html#c.Py_INCREF" title="Py_INCREF"><code class="xref c c-func docutils literal notranslate">Py_INCREF()</code></a> it!) in a global

variable — or wherever you see fit. For example, the following function might

be part of a module definition:

static PyObject *

my_set_callback(PyObject *dummy, PyObject *args)

{

PyObject *result = NULL;

PyObject *temp;

if (PyArg_ParseTuple(args, "O:set_callback", &temp)) {

if (!PyCallable_Check(temp)) {

PyErr_SetString(PyExc_TypeError, "parameter must be callable");

return NULL;

}

Py_XINCREF(temp); /* Add a reference to new callback */

Py_XDECREF(my_callback); /* Dispose of previous callback */

my_callback = temp; /* Remember new callback */

/* Boilerplate to return "None" */

Py_INCREF(Py_None);

result = Py_None;

}

return result;

}

</pre></div>

</div>

This function must be registered with the interpreter using the

<a class="reference internal" href="../c-api/structures.html#METH_VARARGS" title="METH_VARARGS"><code class="xref py py-const docutils literal notranslate">METH_VARARGS</code></a> flag; this is described in section <a class="reference internal" href="#methodtable">The Module’s Method Table and Initialization Function</a>. The

<a class="reference internal" href="#parsetuple">Extracting Parameters in Extension Functions</a>.

The macros <a class="reference internal" href="../c-api/refcounting.html#c.Py_XINCREF" title="Py_XINCREF"><code class="xref c c-func docutils literal notranslate">Py_XINCREF()</code></a> and <a class="reference internal" href="../c-api/refcounting.html#c.Py_XDECREF" title="Py_XDECREF"><code class="xref c c-func docutils literal notranslate">Py_XDECREF()</code></a> increment/decrement the

reference count of an object and are safe in the presence of NULL pointers

(but note that temp will not be NULL in this context). More info on them

in section <a class="reference internal" href="#refcounts">Reference Counts</a>.

Later, when it is time to call the function, you call the C function

<a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a>. This function has two arguments, both pointers to

arbitrary Python objects: the Python function, and the argument list. The

argument list must always be a tuple object, whose length is the number of

arguments. To call the Python function with no arguments, pass in NULL, or

an empty tuple; to call it with one argument, pass a singleton tuple.

<a class="reference internal" href="../c-api/arg.html#c.Py_BuildValue" title="Py_BuildValue"><code class="xref c c-func docutils literal notranslate">Py_BuildValue()</code></a> returns a tuple when its format string consists of zero

or more format codes between parentheses. For example:

<div class="highlight-c notranslate"><div class="highlight"><pre>int arg;

PyObject *arglist;

PyObject *result;

...

arg = 123;

...

/* Time to call the callback */

arglist = Py_BuildValue("(i)", arg);

result = PyObject_CallObject(my_callback, arglist);

Py_DECREF(arglist);

</pre></div>

</div>

<a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a> returns a Python object pointer: this is the return

value of the Python function. <a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a> is

「reference-count-neutral」 with respect to its arguments. In the example a new

tuple was created to serve as the argument list, which is <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a>-ed immediately after the <a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a> call.

The return value of <a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a> is 「new」: either it is a brand

new object, or it is an existing object whose reference count has been

incremented. So, unless you want to save it in a global variable, you should

somehow <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a> the result, even (especially!) if you are not

interested in its value.

Before you do this, however, it is important to check that the return value

isn’t NULL. If it is, the Python function terminated by raising an exception.

If the C code that called <a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a> is called from Python, it

should now return an error indication to its Python caller, so the interpreter

can print a stack trace, or the calling Python code can handle the exception.

If this is not possible or desirable, the exception should be cleared by calling

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Clear" title="PyErr_Clear"><code class="xref c c-func docutils literal notranslate">PyErr_Clear()</code></a>. For example:

<div class="highlight-c notranslate"><div class="highlight"><pre>if (result == NULL)

return NULL; /* Pass error back */

...use result...

Py_DECREF(result);

</pre></div>

</div>

Depending on the desired interface to the Python callback function, you may also

have to provide an argument list to <a class="reference internal" href="../c-api/object.html#c.PyObject_CallObject" title="PyObject_CallObject"><code class="xref c c-func docutils literal notranslate">PyObject_CallObject()</code></a>. In some cases

the argument list is also provided by the Python program, through the same

interface that specified the callback function. It can then be saved and used

in the same manner as the function object. In other cases, you may have to

construct a new tuple to pass as the argument list. The simplest way to do this

is to call <a class="reference internal" href="../c-api/arg.html#c.Py_BuildValue" title="Py_BuildValue"><code class="xref c c-func docutils literal notranslate">Py_BuildValue()</code></a>. For example, if you want to pass an integral

event code, you might use the following code:

<div class="highlight-c notranslate"><div class="highlight"><pre>PyObject *arglist;

...

arglist = Py_BuildValue("(l)", eventcode);

Py_DECREF(arglist);

if (result == NULL)

return NULL; /* Pass error back */

/* Here maybe use the result */

Py_DECREF(result);

</pre></div>

</div>

Note the placement of <code class="docutils literal notranslate">Py_DECREF(arglist)</code> immediately after the call, before

the error check! Also note that strictly speaking this code is not complete:

<a class="reference internal" href="../c-api/arg.html#c.Py_BuildValue" title="Py_BuildValue"><code class="xref c c-func docutils literal notranslate">Py_BuildValue()</code></a> may run out of memory, and this should be checked.

You may also call a function with keyword arguments by using

<a class="reference internal" href="../c-api/object.html#c.PyObject_Call" title="PyObject_Call"><code class="xref c c-func docutils literal notranslate">PyObject_Call()</code></a>, which supports arguments and keyword arguments. As in

the above example, we use <a class="reference internal" href="../c-api/arg.html#c.Py_BuildValue" title="Py_BuildValue"><code class="xref c c-func docutils literal notranslate">Py_BuildValue()</code></a> to construct the dictionary.

<div class="highlight-c notranslate"><div class="highlight"><pre>PyObject *dict;

...

dict = Py_BuildValue("{s:i}", "name", val);

result = PyObject_Call(my_callback, NULL, dict);

Py_DECREF(dict);

if (result == NULL)

return NULL; /* Pass error back */

/* Here maybe use the result */

Py_DECREF(result);

</pre></div>

</div>

<h2>1.7. Extracting Parameters in Extension Functions<a class="headerlink" href="#extracting-parameters-in-extension-functions" title="本標題的永久連結">¶</a></h2>

The <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a> function is declared as follows:

<div class="highlight-c notranslate"><div class="highlight"><pre>int PyArg_ParseTuple(PyObject *arg, const char *format, ...);

</pre></div>

</div>

The arg argument must be a tuple object containing an argument list passed

from Python to a C function. The format argument must be a format string,

whose syntax is explained in <a class="reference internal" href="../c-api/arg.html#arg-parsing">Parsing arguments and building values</a> in the Python/C API Reference

Manual. The remaining arguments must be addresses of variables whose type is

determined by the format string.

Note that while <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a> checks that the Python arguments have

the required types, it cannot check the validity of the addresses of C variables

passed to the call: if you make mistakes there, your code will probably crash or

at least overwrite random bits in memory. So be careful!

Note that any Python object references which are provided to the caller are

borrowed references; do not decrement their reference count!

Some example calls:

<div class="highlight-c notranslate"><div class="highlight"><pre>#define PY_SSIZE_T_CLEAN /* Make "s#" use Py_ssize_t rather than int. */

#include <Python.h>

</pre></div>

</div>

<div class="highlight-c notranslate"><div class="highlight"><pre>int ok;

int i, j;

long k, l;

const char *s;

Py_ssize_t size;

/* Python call: f() */

</pre></div>

</div>

/* Possible Python call: f('whoops!') */

</pre></div>

</div>

<div class="highlight-c notranslate"><div class="highlight"><pre>ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */

/* Possible Python call: f(1, 2, 'three') */

</pre></div>

</div>

<div class="highlight-c notranslate"><div class="highlight"><pre>ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size);

/* A pair of ints and a string, whose size is also returned */

/* Possible Python call: f((1, 2), 'three') */

</pre></div>

</div>

<div class="highlight-c notranslate"><div class="highlight"><pre>{

const char *file;

const char *mode = "r";

int bufsize = 0;

ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize);

/* A string, and optionally another string and an integer */

/* Possible Python calls:

f('spam')

f('spam', 'w')

f('spam', 'wb', 100000) */

}

</pre></div>

</div>

<div class="highlight-c notranslate"><div class="highlight"><pre>{

int left, top, right, bottom, h, v;

&left, &top, &right, &bottom, &h, &v);

/* A rectangle and a point */

/* Possible Python call:

f(((0, 0), (400, 300)), (10, 10)) */

}

</pre></div>

</div>

<div class="highlight-c notranslate"><div class="highlight"><pre>{

Py_complex c;

/* a complex, also providing a function name for errors */

/* Possible Python call: myfunction(1+2j) */

}

</pre></div>

</div>

<h2>1.8. Keyword Parameters for Extension Functions<a class="headerlink" href="#keyword-parameters-for-extension-functions" title="本標題的永久連結">¶</a></h2>

The <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTupleAndKeywords" title="PyArg_ParseTupleAndKeywords"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code></a> function is declared as follows:

<div class="highlight-c notranslate"><div class="highlight"><pre>int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict,

const char *format, char *kwlist[], ...);

</pre></div>

</div>

The arg and format parameters are identical to those of the

keywords received as the third parameter from the Python runtime. The kwlist

parameter is a NULL-terminated list of strings which identify the parameters;

the names are matched with the type information from format from left to

right. On success, <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTupleAndKeywords" title="PyArg_ParseTupleAndKeywords"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTupleAndKeywords()</code></a> returns true, otherwise

it returns false and raises an appropriate exception.

備註

Nested tuples cannot be parsed when using keyword arguments! Keyword parameters

passed in which are not present in the kwlist will cause <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal notranslate">TypeError</code></a> to

be raised.

</div>

Here is an example module which uses keywords, based on an example by Geoff

Philbrick (<a class="reference external" href="mailto:philbrick%40hks.com">philbrick@hks.com</a>):

<div class="highlight-c notranslate"><div class="highlight"><pre>#include "Python.h"

static PyObject *

keywdarg_parrot(PyObject *self, PyObject *args, PyObject *keywds)

{

int voltage;

const char *state = "a stiff";

const char *action = "voom";

const char *type = "Norwegian Blue";

static char *kwlist[] = {"voltage", "state", "action", "type", NULL};

if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist,

&voltage, &state, &action, &type))

return NULL;

printf("-- This parrot wouldn't %s if you put %i Volts through it.\n",

action, voltage);

printf("-- Lovely plumage, the %s -- It's %s!\n", type, state);

Py_RETURN_NONE;

}

static PyMethodDef keywdarg_methods[] = {

/* The cast of the function is necessary since PyCFunction values

* only take two PyObject* parameters, and keywdarg_parrot() takes

* three.

*/

{"parrot", (PyCFunction)keywdarg_parrot, METH_VARARGS | METH_KEYWORDS,

"Print a lovely skit to standard output."},

};

static struct PyModuleDef keywdargmodule = {

PyModuleDef_HEAD_INIT,

"keywdarg",

NULL,

-1,

keywdarg_methods

};

PyMODINIT_FUNC

PyInit_keywdarg(void)

{

return PyModule_Create(&keywdargmodule);

}

</pre></div>

</div>

<h2>1.9. Building Arbitrary Values<a class="headerlink" href="#building-arbitrary-values" title="本標題的永久連結">¶</a></h2>

This function is the counterpart to <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a>. It is declared

as follows:

<div class="highlight-c notranslate"><div class="highlight"><pre>PyObject *Py_BuildValue(const char *format, ...);

</pre></div>

</div>

It recognizes a set of format units similar to the ones recognized by

not output) must not be pointers, just values. It returns a new Python object,

suitable for returning from a C function called from Python.

One difference with <a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal notranslate">PyArg_ParseTuple()</code></a>: while the latter requires its

first argument to be a tuple (since Python argument lists are always represented

as tuples internally), <a class="reference internal" href="../c-api/arg.html#c.Py_BuildValue" title="Py_BuildValue"><code class="xref c c-func docutils literal notranslate">Py_BuildValue()</code></a> does not always build a tuple. It

builds a tuple only if its format string contains two or more format units. If

the format string is empty, it returns <code class="docutils literal notranslate">None</code>; if it contains exactly one

format unit, it returns whatever object is described by that format unit. To

force it to return a tuple of size 0 or one, parenthesize the format string.

Examples (to the left the call, to the right the resulting Python value):

<div class="highlight-none notranslate"><div class="highlight"><pre>Py_BuildValue("") None

Py_BuildValue("i", 123) 123

Py_BuildValue("iii", 123, 456, 789) (123, 456, 789)

Py_BuildValue("s", "hello") 'hello'

Py_BuildValue("y", "hello") b'hello'

Py_BuildValue("ss", "hello", "world") ('hello', 'world')

Py_BuildValue("s#", "hello", 4) 'hell'

Py_BuildValue("y#", "hello", 4) b'hell'

Py_BuildValue("()") ()

Py_BuildValue("(i)", 123) (123,)

Py_BuildValue("(ii)", 123, 456) (123, 456)

Py_BuildValue("(i,i)", 123, 456) (123, 456)

Py_BuildValue("[i,i]", 123, 456) [123, 456]

Py_BuildValue("{s:i,s:i}",

"abc", 123, "def", 456) {'abc': 123, 'def': 456}

Py_BuildValue("((ii)(ii)) (ii)",

1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6))

</pre></div>

</div>

<h2>1.10. Reference Counts<a class="headerlink" href="#reference-counts" title="本標題的永久連結">¶</a></h2>

In languages like C or C++, the programmer is responsible for dynamic allocation

and deallocation of memory on the heap. In C, this is done using the functions

<code class="xref c c-func docutils literal notranslate">malloc()</code> and <code class="xref c c-func docutils literal notranslate">free()</code>. In C++, the operators <code class="docutils literal notranslate">new</code> and

<code class="docutils literal notranslate">delete</code> are used with essentially the same meaning and we’ll restrict

the following discussion to the C case.

Every block of memory allocated with <code class="xref c c-func docutils literal notranslate">malloc()</code> should eventually be

returned to the pool of available memory by exactly one call to <code class="xref c c-func docutils literal notranslate">free()</code>.

It is important to call <code class="xref c c-func docutils literal notranslate">free()</code> at the right time. If a block’s address

is forgotten but <code class="xref c c-func docutils literal notranslate">free()</code> is not called for it, the memory it occupies

cannot be reused until the program terminates. This is called a memory

leak. On the other hand, if a program calls <code class="xref c c-func docutils literal notranslate">free()</code> for a block and then

continues to use the block, it creates a conflict with re-use of the block

through another <code class="xref c c-func docutils literal notranslate">malloc()</code> call. This is called using freed memory.

It has the same bad consequences as referencing uninitialized data — core

dumps, wrong results, mysterious crashes.

Common causes of memory leaks are unusual paths through the code. For instance,

a function may allocate a block of memory, do some calculation, and then free

the block again. Now a change in the requirements for the function may add a

test to the calculation that detects an error condition and can return

prematurely from the function. It’s easy to forget to free the allocated memory

block when taking this premature exit, especially when it is added later to the

code. Such leaks, once introduced, often go undetected for a long time: the

error exit is taken only in a small fraction of all calls, and most modern

machines have plenty of virtual memory, so the leak only becomes apparent in a

long-running process that uses the leaking function frequently. Therefore, it’s

important to prevent leaks from happening by having a coding convention or

strategy that minimizes this kind of errors.

Since Python makes heavy use of <code class="xref c c-func docutils literal notranslate">malloc()</code> and <code class="xref c c-func docutils literal notranslate">free()</code>, it needs a

strategy to avoid memory leaks as well as the use of freed memory. The chosen

method is called reference counting. The principle is simple: every

object contains a counter, which is incremented when a reference to the object

is stored somewhere, and which is decremented when a reference to it is deleted.

When the counter reaches zero, the last reference to the object has been deleted

and the object is freed.

An alternative strategy is called automatic garbage collection.

(Sometimes, reference counting is also referred to as a garbage collection

strategy, hence my use of 「automatic」 to distinguish the two.) The big

advantage of automatic garbage collection is that the user doesn’t need to call

<code class="xref c c-func docutils literal notranslate">free()</code> explicitly. (Another claimed advantage is an improvement in speed

or memory usage — this is no hard fact however.) The disadvantage is that for

C, there is no truly portable automatic garbage collector, while reference

counting can be implemented portably (as long as the functions <code class="xref c c-func docutils literal notranslate">malloc()</code>

and <code class="xref c c-func docutils literal notranslate">free()</code> are available — which the C Standard guarantees). Maybe some

day a sufficiently portable automatic garbage collector will be available for C.

Until then, we’ll have to live with reference counts.

While Python uses the traditional reference counting implementation, it also

offers a cycle detector that works to detect reference cycles. This allows

applications to not worry about creating direct or indirect circular references;

these are the weakness of garbage collection implemented using only reference

counting. Reference cycles consist of objects which contain (possibly indirect)

references to themselves, so that each object in the cycle has a reference count

which is non-zero. Typical reference counting implementations are not able to

reclaim the memory belonging to any objects in a reference cycle, or referenced

from the objects in the cycle, even though there are no further references to

the cycle itself.

The cycle detector is able to detect garbage cycles and can reclaim them.

The <a class="reference internal" href="../library/gc.html#module-gc" title="gc: Interface to the cycle-detecting garbage collector."><code class="xref py py-mod docutils literal notranslate">gc</code></a> module exposes a way to run the detector (the

<a class="reference internal" href="../library/gc.html#gc.collect" title="gc.collect"><code class="xref py py-func docutils literal notranslate">collect()</code></a> function), as well as configuration

interfaces and the ability to disable the detector at runtime. The cycle

detector is considered an optional component; though it is included by default,

it can be disabled at build time using the <code class="xref std std-option docutils literal notranslate">--without-cycle-gc</code> option

to the configure script on Unix platforms (including Mac OS X). If

the cycle detector is disabled in this way, the <a class="reference internal" href="../library/gc.html#module-gc" title="gc: Interface to the cycle-detecting garbage collector."><code class="xref py py-mod docutils literal notranslate">gc</code></a> module will not be

available.

<h3>1.10.1. Reference Counting in Python<a class="headerlink" href="#reference-counting-in-python" title="本標題的永久連結">¶</a></h3>

There are two macros, <code class="docutils literal notranslate">Py_INCREF(x)</code> and <code class="docutils literal notranslate">Py_DECREF(x)</code>, which handle the

incrementing and decrementing of the reference count. <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a> also

frees the object when the count reaches zero. For flexibility, it doesn’t call

<code class="xref c c-func docutils literal notranslate">free()</code> directly — rather, it makes a call through a function pointer in

the object’s type object. For this purpose (and others), every object

also contains a pointer to its type object.

The big question now remains: when to use <code class="docutils literal notranslate">Py_INCREF(x)</code> and <code class="docutils literal notranslate">Py_DECREF(x)</code>?

Let’s first introduce some terms. Nobody 「owns」 an object; however, you can

own a reference to an object. An object’s reference count is now defined

as the number of owned references to it. The owner of a reference is

responsible for calling <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a> when the reference is no longer

needed. Ownership of a reference can be transferred. There are three ways to

dispose of an owned reference: pass it on, store it, or call <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a>.

Forgetting to dispose of an owned reference creates a memory leak.

It is also possible to borrow <a class="footnote-reference" href="#id6" id="id2">[2]</a> a reference to an object. The

borrower of a reference should not call <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a>. The borrower must

not hold on to the object longer than the owner from which it was borrowed.

Using a borrowed reference after the owner has disposed of it risks using freed

memory and should be avoided completely <a class="footnote-reference" href="#id7" id="id3">[3]</a>.

The advantage of borrowing over owning a reference is that you don’t need to

take care of disposing of the reference on all possible paths through the code

— in other words, with a borrowed reference you don’t run the risk of leaking

when a premature exit is taken. The disadvantage of borrowing over owning is

that there are some subtle situations where in seemingly correct code a borrowed

reference can be used after the owner from which it was borrowed has in fact

disposed of it.

A borrowed reference can be changed into an owned reference by calling

<a class="reference internal" href="../c-api/refcounting.html#c.Py_INCREF" title="Py_INCREF"><code class="xref c c-func docutils literal notranslate">Py_INCREF()</code></a>. This does not affect the status of the owner from which the

reference was borrowed — it creates a new owned reference, and gives full

owner responsibilities (the new owner must dispose of the reference properly, as

well as the previous owner).

</div>

<h3>1.10.2. Ownership Rules<a class="headerlink" href="#ownership-rules" title="本標題的永久連結">¶</a></h3>

Whenever an object reference is passed into or out of a function, it is part of

the function’s interface specification whether ownership is transferred with the

reference or not.

Most functions that return a reference to an object pass on ownership with the

reference. In particular, all functions whose function it is to create a new

object, such as <a class="reference internal" href="../c-api/long.html#c.PyLong_FromLong" title="PyLong_FromLong"><code class="xref c c-func docutils literal notranslate">PyLong_FromLong()</code></a> and <a class="reference internal" href="../c-api/arg.html#c.Py_BuildValue" title="Py_BuildValue"><code class="xref c c-func docutils literal notranslate">Py_BuildValue()</code></a>, pass

ownership to the receiver. Even if the object is not actually new, you still

receive ownership of a new reference to that object. For instance,

<a class="reference internal" href="../c-api/long.html#c.PyLong_FromLong" title="PyLong_FromLong"><code class="xref c c-func docutils literal notranslate">PyLong_FromLong()</code></a> maintains a cache of popular values and can return a

reference to a cached item.

Many functions that extract objects from other objects also transfer ownership

with the reference, for instance <a class="reference internal" href="../c-api/object.html#c.PyObject_GetAttrString" title="PyObject_GetAttrString"><code class="xref c c-func docutils literal notranslate">PyObject_GetAttrString()</code></a>. The picture

is less clear, here, however, since a few common routines are exceptions:

<a class="reference internal" href="../c-api/tuple.html#c.PyTuple_GetItem" title="PyTuple_GetItem"><code class="xref c c-func docutils literal notranslate">PyTuple_GetItem()</code></a>, <a class="reference internal" href="../c-api/list.html#c.PyList_GetItem" title="PyList_GetItem"><code class="xref c c-func docutils literal notranslate">PyList_GetItem()</code></a>, <a class="reference internal" href="../c-api/dict.html#c.PyDict_GetItem" title="PyDict_GetItem"><code class="xref c c-func docutils literal notranslate">PyDict_GetItem()</code></a>, and

<a class="reference internal" href="../c-api/dict.html#c.PyDict_GetItemString" title="PyDict_GetItemString"><code class="xref c c-func docutils literal notranslate">PyDict_GetItemString()</code></a> all return references that you borrow from the

tuple, list or dictionary.

The function <a class="reference internal" href="../c-api/import.html#c.PyImport_AddModule" title="PyImport_AddModule"><code class="xref c c-func docutils literal notranslate">PyImport_AddModule()</code></a> also returns a borrowed reference, even

though it may actually create the object it returns: this is possible because an

owned reference to the object is stored in <code class="docutils literal notranslate">sys.modules</code>.

When you pass an object reference into another function, in general, the

function borrows the reference from you — if it needs to store it, it will use

<a class="reference internal" href="../c-api/refcounting.html#c.Py_INCREF" title="Py_INCREF"><code class="xref c c-func docutils literal notranslate">Py_INCREF()</code></a> to become an independent owner. There are exactly two

important exceptions to this rule: <a class="reference internal" href="../c-api/tuple.html#c.PyTuple_SetItem" title="PyTuple_SetItem"><code class="xref c c-func docutils literal notranslate">PyTuple_SetItem()</code></a> and

<a class="reference internal" href="../c-api/list.html#c.PyList_SetItem" title="PyList_SetItem"><code class="xref c c-func docutils literal notranslate">PyList_SetItem()</code></a>. These functions take over ownership of the item passed

to them — even if they fail! (Note that <a class="reference internal" href="../c-api/dict.html#c.PyDict_SetItem" title="PyDict_SetItem"><code class="xref c c-func docutils literal notranslate">PyDict_SetItem()</code></a> and friends

don’t take over ownership — they are 「normal.」)

When a C function is called from Python, it borrows references to its arguments

from the caller. The caller owns a reference to the object, so the borrowed

reference’s lifetime is guaranteed until the function returns. Only when such a

borrowed reference must be stored or passed on, it must be turned into an owned

reference by calling <a class="reference internal" href="../c-api/refcounting.html#c.Py_INCREF" title="Py_INCREF"><code class="xref c c-func docutils literal notranslate">Py_INCREF()</code></a>.

The object reference returned from a C function that is called from Python must

be an owned reference — ownership is transferred from the function to its

caller.

</div>

There are a few situations where seemingly harmless use of a borrowed reference

can lead to problems. These all have to do with implicit invocations of the

interpreter, which can cause the owner of a reference to dispose of it.

The first and most important case to know about is using <a class="reference internal" href="../c-api/refcounting.html#c.Py_DECREF" title="Py_DECREF"><code class="xref c c-func docutils literal notranslate">Py_DECREF()</code></a> on

an unrelated object while borrowing a reference to a list item. For instance:

<div class="highlight-c notranslate"><div class="highlight"><pre>void

bug(PyObject *list)

{

PyObject *item = PyList_GetItem(list, 0);

PyList_SetItem(list, 1, PyLong_FromLong(0L));

PyObject_Print(item, stdout, 0); /* BUG! */

}

</pre></div>

</div>

This function first borrows a reference to <code class="docutils literal notranslate">list[0]</code>, then replaces

<code class="docutils literal notranslate">list[1]</code> with the value <code class="docutils literal notranslate">0</code>, and finally prints the borrowed reference.

Looks harmless, right? But it’s not!

Let’s follow the control flow into <a class="reference internal" href="../c-api/list.html#c.PyList_SetItem" title="PyList_SetItem"><code class="xref c c-func docutils literal notranslate">PyList_SetItem()</code></a>. The list owns

references to all its items, so when item 1 is replaced, it has to dispose of

the original item 1. Now let’s suppose the original item 1 was an instance of a

user-defined class, and let’s further suppose that the class defined a

<a class="reference internal" href="../reference/datamodel.html#object.__del__" title="object.__del__"><code class="xref py py-meth docutils literal notranslate">__del__()</code></a> method. If this class instance has a reference count of 1,

disposing of it will call its <a class="reference internal" href="../reference/datamodel.html#object.__del__" title="object.__del__"><code class="xref py py-meth docutils literal notranslate">__del__()</code></a> method.

Since it is written in Python, the <a class="reference internal" href="../reference/datamodel.html#object.__del__" title="object.__del__"><code class="xref py py-meth docutils literal notranslate">__del__()</code></a> method can execute arbitrary

Python code. Could it perhaps do something to invalidate the reference to

<code class="docutils literal notranslate">item</code> in <code class="xref c c-func docutils literal notranslate">bug()</code>? You bet! Assuming that the list passed into

<code class="xref c c-func docutils literal notranslate">bug()</code> is accessible to the <a class="reference internal" href="../reference/datamodel.html#object.__del__" title="object.__del__"><code class="xref py py-meth docutils literal notranslate">__del__()</code></a> method, it could execute a

statement to the effect of <code class="docutils literal notranslate">del list[0]</code>, and assuming this was the last

reference to that object, it would free the memory associated with it, thereby

invalidating <code class="docutils literal notranslate">item</code>.

The solution, once you know the source of the problem, is easy: temporarily

increment the reference count. The correct version of the function reads:

<div class="highlight-c notranslate"><div class="highlight"><pre>void

no_bug(PyObject *list)

{

Py_INCREF(item);

Py_DECREF(item);

}

</pre></div>

</div>

This is a true story. An older version of Python contained variants of this bug

and someone spent a considerable amount of time in a C debugger to figure out

why his <a class="reference internal" href="../reference/datamodel.html#object.__del__" title="object.__del__"><code class="xref py py-meth docutils literal notranslate">__del__()</code></a> methods would fail…

The second case of problems with a borrowed reference is a variant involving

threads. Normally, multiple threads in the Python interpreter can’t get in each

other’s way, because there is a global lock protecting Python’s entire object

space. However, it is possible to temporarily release this lock using the macro

<a class="reference internal" href="../c-api/init.html#c.Py_BEGIN_ALLOW_THREADS" title="Py_BEGIN_ALLOW_THREADS"><code class="xref c c-macro docutils literal notranslate">Py_BEGIN_ALLOW_THREADS</code></a>, and to re-acquire it using

<a class="reference internal" href="../c-api/init.html#c.Py_END_ALLOW_THREADS" title="Py_END_ALLOW_THREADS"><code class="xref c c-macro docutils literal notranslate">Py_END_ALLOW_THREADS</code></a>. This is common around blocking I/O calls, to

let other threads use the processor while waiting for the I/O to complete.

Obviously, the following function has the same problem as the previous one:

<div class="highlight-c notranslate"><div class="highlight"><pre>void

bug(PyObject *list)

{

Py_BEGIN_ALLOW_THREADS

...some blocking I/O call...

View remainder of file in raw view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

extending.html

Latest commit

History

extending.html

File metadata and controls