\documentclass{manual}

\usepackage{distutils}

% $Id$

% TODO

% Document extension.read_setup_file

% Document build_clib command

%

\title{Distributing Python Modules}

\input{boilerplate}

\author{Greg Ward\\

Anthony Baxter}

\authoraddress{

\strong{Python Software Foundation}\\

Email: \email{[email protected]}

}

\makeindex

\makemodindex

\begin{document}

\maketitle

\begin{abstract}

\noindent

This document describes the Python Distribution Utilities

(``Distutils'') from the module developer's point of view, describing

how to use the Distutils to make Python modules and extensions easily

available to a wider audience with very little overhead for

build/release/install mechanics.

\end{abstract}

% The ugly "%begin{latexonly}" pseudo-environment supresses the table

% of contents for HTML generation.

%

%begin{latexonly}

\tableofcontents

%end{latexonly}

\chapter{An Introduction to Distutils}

\label{intro}

This document covers using the Distutils to distribute your Python

modules, concentrating on the role of developer/distributor: if

you're looking for information on installing Python modules, you

should refer to the \citetitle[../inst/inst.html]{Installing Python

Modules} manual.

\section{Concepts \& Terminology}

\label{concepts}

Using the Distutils is quite simple, both for module developers and for

users/administrators installing third-party modules. As a developer,

your responsibilities (apart from writing solid, well-documented and

well-tested code, of course!) are:

\begin{itemize}

\item write a setup script (\file{setup.py} by convention)

\item (optional) write a setup configuration file

\item create a source distribution

\item (optional) create one or more built (binary) distributions

\end{itemize}

Each of these tasks is covered in this document.

Not all module developers have access to a multitude of platforms, so

it's not always feasible to expect them to create a multitude of built

distributions. It is hoped that a class of intermediaries, called

\emph{packagers}, will arise to address this need. Packagers will take

source distributions released by module developers, build them on one or

more platforms, and release the resulting built distributions. Thus,

users on the most popular platforms will be able to install most popular

Python module distributions in the most natural way for their platform,

without having to run a single setup script or compile a line of code.

\section{A Simple Example}

\label{simple-example}

The setup script is usually quite simple, although since it's written

in Python, there are no arbitrary limits to what you can do with it,

though you should be careful about putting arbitrarily expensive

operations in your setup script. Unlike, say, Autoconf-style configure

scripts, the setup script may be run multiple times in the course of

building and installing your module distribution.

If all you want to do is distribute a module called \module{foo},

contained in a file \file{foo.py}, then your setup script can be as

simple as this:

\begin{verbatim}

from distutils.core import setup

setup(name='foo',

version='1.0',

py_modules=['foo'],

)

\end{verbatim}

Some observations:

\begin{itemize}

\item most information that you supply to the Distutils is supplied as

keyword arguments to the \function{setup()} function

\item those keyword arguments fall into two categories: package

metadata (name, version number) and information about what's in the

package (a list of pure Python modules, in this case)

\item modules are specified by module name, not filename (the same will

hold true for packages and extensions)

\item it's recommended that you supply a little more metadata, in

particular your name, email address and a URL for the project

(see section~\ref{setup-script} for an example)

\end{itemize}

To create a source distribution for this module, you would create a

setup script, \file{setup.py}, containing the above code, and run:

\begin{verbatim}

python setup.py sdist

\end{verbatim}

which will create an archive file (e.g., tarball on \UNIX, ZIP file on

Windows) containing your setup script \file{setup.py}, and your module

\file{foo.py}. The archive file will be named \file{foo-1.0.tar.gz} (or

\file{.zip}), and will unpack into a directory \file{foo-1.0}.

If an end-user wishes to install your \module{foo} module, all she has

to do is download \file{foo-1.0.tar.gz} (or \file{.zip}), unpack it,

and---from the \file{foo-1.0} directory---run

\begin{verbatim}

python setup.py install

\end{verbatim}

which will ultimately copy \file{foo.py} to the appropriate directory

for third-party modules in their Python installation.

This simple example demonstrates some fundamental concepts of the

Distutils. First, both developers and installers have the same basic

user interface, i.e. the setup script. The difference is which

Distutils \emph{commands} they use: the \command{sdist} command is

almost exclusively for module developers, while \command{install} is

more often for installers (although most developers will want to install

their own code occasionally).

If you want to make things really easy for your users, you can create

one or more built distributions for them. For instance, if you are

running on a Windows machine, and want to make things easy for other

Windows users, you can create an executable installer (the most

appropriate type of built distribution for this platform) with the

\command{bdist\_wininst} command. For example:

\begin{verbatim}

python setup.py bdist_wininst

\end{verbatim}

will create an executable installer, \file{foo-1.0.win32.exe}, in the

current directory.

Other useful built distribution formats are RPM, implemented by the

\command{bdist\_rpm} command, Solaris \program{pkgtool}

(\command{bdist\_pkgtool}), and HP-UX \program{swinstall}

(\command{bdist_sdux}). For example, the following command will

create an RPM file called \file{foo-1.0.noarch.rpm}:

\begin{verbatim}

python setup.py bdist_rpm

\end{verbatim}

(The \command{bdist\_rpm} command uses the \command{rpm} executable,

therefore this has to be run on an RPM-based system such as Red Hat

Linux, SuSE Linux, or Mandrake Linux.)

You can find out what distribution formats are available at any time by

running

\begin{verbatim}

python setup.py bdist --help-formats

\end{verbatim}

\section{General Python terminology}

\label{python-terms}

If you're reading this document, you probably have a good idea of what

modules, extensions, and so forth are. Nevertheless, just to be sure

that everyone is operating from a common starting point, we offer the

following glossary of common Python terms:

\begin{description}

\item[module] the basic unit of code reusability in Python: a block of

code imported by some other code. Three types of modules concern us

here: pure Python modules, extension modules, and packages.

\item[pure Python module] a module written in Python and contained in a

single \file{.py} file (and possibly associated \file{.pyc} and/or

\file{.pyo} files). Sometimes referred to as a ``pure module.''

\item[extension module] a module written in the low-level language of

the Python implementation: C/\Cpp{} for Python, Java for Jython.

Typically contained in a single dynamically loadable pre-compiled

file, e.g. a shared object (\file{.so}) file for Python extensions on

\UNIX, a DLL (given the \file{.pyd} extension) for Python extensions

on Windows, or a Java class file for Jython extensions. (Note that

currently, the Distutils only handles C/\Cpp{} extensions for Python.)

\item[package] a module that contains other modules; typically contained

in a directory in the filesystem and distinguished from other

directories by the presence of a file \file{\_\_init\_\_.py}.

\item[root package] the root of the hierarchy of packages. (This isn't

really a package, since it doesn't have an \file{\_\_init\_\_.py}

file. But we have to call it something.) The vast majority of the

standard library is in the root package, as are many small, standalone

third-party modules that don't belong to a larger module collection.

Unlike regular packages, modules in the root package can be found in

many directories: in fact, every directory listed in \code{sys.path}

contributes modules to the root package.

\end{description}

\section{Distutils-specific terminology}

\label{distutils-term}

The following terms apply more specifically to the domain of

distributing Python modules using the Distutils:

\begin{description}

\item[module distribution] a collection of Python modules distributed

together as a single downloadable resource and meant to be installed

\emph{en masse}. Examples of some well-known module distributions are

Numeric Python, PyXML, PIL (the Python Imaging Library), or

mxBase. (This would be called a \emph{package}, except that term

is already taken in the Python context: a single module distribution

may contain zero, one, or many Python packages.)

\item[pure module distribution] a module distribution that contains only

pure Python modules and packages. Sometimes referred to as a ``pure

distribution.''

\item[non-pure module distribution] a module distribution that contains

at least one extension module. Sometimes referred to as a ``non-pure

distribution.''

\item[distribution root] the top-level directory of your source tree (or

source distribution); the directory where \file{setup.py} exists. Generally

\file{setup.py} will be run from this directory.

\end{description}

\chapter{Writing the Setup Script}

\label{setup-script}

The setup script is the centre of all activity in building,

distributing, and installing modules using the Distutils. The main

purpose of the setup script is to describe your module distribution to

the Distutils, so that the various commands that operate on your modules

do the right thing. As we saw in section~\ref{simple-example} above,

the setup script consists mainly of a call to \function{setup()}, and

most information supplied to the Distutils by the module developer is

supplied as keyword arguments to \function{setup()}.

Here's a slightly more involved example, which we'll follow for the next

couple of sections: the Distutils' own setup script. (Keep in mind that

although the Distutils are included with Python 1.6 and later, they also

have an independent existence so that Python 1.5.2 users can use them to

install other module distributions. The Distutils' own setup script,

shown here, is used to install the package into Python 1.5.2.)

\begin{verbatim}

#!/usr/bin/env python

from distutils.core import setup

setup(name='Distutils',

version='1.0',

description='Python Distribution Utilities',

author='Greg Ward',

author_email='[email protected]',

url='http://www.python.org/sigs/distutils-sig/',

packages=['distutils', 'distutils.command'],

)

\end{verbatim}

There are only two differences between this and the trivial one-file

distribution presented in section~\ref{simple-example}: more

metadata, and the specification of pure Python modules by package,

rather than by module. This is important since the Distutils consist of

a couple of dozen modules split into (so far) two packages; an explicit

list of every module would be tedious to generate and difficult to

maintain. For more information on the additional meta-data, see

section~\ref{meta-data}.

Note that any pathnames (files or directories) supplied in the setup

script should be written using the \UNIX{} convention, i.e.

slash-separated. The Distutils will take care of converting this

platform-neutral representation into whatever is appropriate on your

current platform before actually using the pathname. This makes your

setup script portable across operating systems, which of course is one

of the major goals of the Distutils. In this spirit, all pathnames in

this document are slash-separated. (Mac OS programmers should keep in

mind that the \emph{absence} of a leading slash indicates a relative

path, the opposite of the Mac OS convention with colons.)

This, of course, only applies to pathnames given to Distutils

functions. If you, for example, use standard Python functions such as

\function{glob.glob()} or \function{os.listdir()} to specify files, you

should be careful to write portable code instead of hardcoding path

separators:

\begin{verbatim}

glob.glob(os.path.join('mydir', 'subdir', '*.html'))

os.listdir(os.path.join('mydir', 'subdir'))

\end{verbatim}

\section{Listing whole packages}

\label{listing-packages}

The \option{packages} option tells the Distutils to process (build,

distribute, install, etc.) all pure Python modules found in each package

mentioned in the \option{packages} list. In order to do this, of

course, there has to be a correspondence between package names and

directories in the filesystem. The default correspondence is the most

obvious one, i.e. package \module{distutils} is found in the directory

\file{distutils} relative to the distribution root. Thus, when you say

\code{packages = ['foo']} in your setup script, you are promising that

the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might

be spelled differently on your system, but you get the idea) relative to

the directory where your setup script lives. If you break this

promise, the Distutils will issue a warning but still process the broken

package anyways.

If you use a different convention to lay out your source directory,

that's no problem: you just have to supply the \option{package\_dir}

option to tell the Distutils about your convention. For example, say

you keep all Python source under \file{lib}, so that modules in the

``root package'' (i.e., not in any package at all) are in

\file{lib}, modules in the \module{foo} package are in \file{lib/foo},

and so forth. Then you would put

\begin{verbatim}

package_dir = {'': 'lib'}

\end{verbatim}

in your setup script. The keys to this dictionary are package names,

and an empty package name stands for the root package. The values are

directory names relative to your distribution root. In this case, when

you say \code{packages = ['foo']}, you are promising that the file

\file{lib/foo/\_\_init\_\_.py} exists.

Another possible convention is to put the \module{foo} package right in

\file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc. This

would be written in the setup script as

\begin{verbatim}

package_dir = {'foo': 'lib'}

\end{verbatim}

A \code{\var{package}: \var{dir}} entry in the \option{package\_dir}

dictionary implicitly applies to all packages below \var{package}, so

the \module{foo.bar} case is automatically handled here. In this

example, having \code{packages = ['foo', 'foo.bar']} tells the Distutils

to look for \file{lib/\_\_init\_\_.py} and

\file{lib/bar/\_\_init\_\_.py}. (Keep in mind that although

\option{package\_dir} applies recursively, you must explicitly list all

packages in \option{packages}: the Distutils will \emph{not} recursively

scan your source tree looking for any directory with an

\file{\_\_init\_\_.py} file.)

\section{Listing individual modules}

\label{listing-modules}

For a small module distribution, you might prefer to list all modules

rather than listing packages---especially the case of a single module

that goes in the ``root package'' (i.e., no package at all). This

simplest case was shown in section~\ref{simple-example}; here is a

slightly more involved example:

\begin{verbatim}

py_modules = ['mod1', 'pkg.mod2']

\end{verbatim}

This describes two modules, one of them in the ``root'' package, the

other in the \module{pkg} package. Again, the default package/directory

layout implies that these two modules can be found in \file{mod1.py} and

\file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well.

And again, you can override the package/directory correspondence using

the \option{package\_dir} option.

\section{Describing extension modules}

\label{describing-extensions}

% XXX read over this section

Just as writing Python extension modules is a bit more complicated than

writing pure Python modules, describing them to the Distutils is a bit

more complicated. Unlike pure modules, it's not enough just to list

modules or packages and expect the Distutils to go out and find the

right files; you have to specify the extension name, source file(s), and

any compile/link requirements (include directories, libraries to link

with, etc.).

All of this is done through another keyword argument to

\function{setup()}, the \option{extensions} option. \option{extensions}

is just a list of \class{Extension} instances, each of which describes a

single extension module. Suppose your distribution includes a single

extension, called \module{foo} and implemented by \file{foo.c}. If no

additional instructions to the compiler/linker are needed, describing

this extension is quite simple:

\begin{verbatim}

Extension('foo', ['foo.c'])

\end{verbatim}

The \class{Extension} class can be imported from

\module{distutils.core} along with \function{setup()}. Thus, the setup

script for a module distribution that contains only this one extension

and nothing else might be:

\begin{verbatim}

from distutils.core import setup, Extension

setup(name='foo',

version='1.0',

ext_modules=[Extension('foo', ['foo.c'])],

)

\end{verbatim}

The \class{Extension} class (actually, the underlying extension-building

machinery implemented by the \command{build\_ext} command) supports a

great deal of flexibility in describing Python extensions, which is

explained in the following sections.

\subsection{Extension names and packages}

The first argument to the \class{Extension} constructor is always the

name of the extension, including any package names. For example,

\begin{verbatim}

Extension('foo', ['src/foo1.c', 'src/foo2.c'])

\end{verbatim}

describes an extension that lives in the root package, while

\begin{verbatim}

Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c'])

\end{verbatim}

describes the same extension in the \module{pkg} package. The source

files and resulting object code are identical in both cases; the only

difference is where in the filesystem (and therefore where in Python's

namespace hierarchy) the resulting extension lives.

If you have a number of extensions all in the same package (or all under

the same base package), use the \option{ext\_package} keyword argument

to \function{setup()}. For example,

\begin{verbatim}

setup(...

ext_package='pkg',

ext_modules=[Extension('foo', ['foo.c']),

Extension('subpkg.bar', ['bar.c'])],

)

\end{verbatim}

will compile \file{foo.c} to the extension \module{pkg.foo}, and

\file{bar.c} to \module{pkg.subpkg.bar}.

\subsection{Extension source files}

The second argument to the \class{Extension} constructor is a list of

source files. Since the Distutils currently only support C, \Cpp, and

Objective-C extensions, these are normally C/\Cpp/Objective-C source

files. (Be sure to use appropriate extensions to distinguish \Cpp\

source files: \file{.cc} and \file{.cpp} seem to be recognized by both

\UNIX{} and Windows compilers.)

However, you can also include SWIG interface (\file{.i}) files in the

list; the \command{build\_ext} command knows how to deal with SWIG

extensions: it will run SWIG on the interface file and compile the

resulting C/\Cpp{} file into your extension.

\XXX{SWIG support is rough around the edges and largely untested;

especially SWIG support for \Cpp{} extensions! Explain in more detail

here when the interface firms up.}

On some platforms, you can include non-source files that are processed

by the compiler and included in your extension. Currently, this just

means Windows message text (\file{.mc}) files and resource definition

(\file{.rc}) files for Visual \Cpp. These will be compiled to binary resource

(\file{.res}) files and linked into the executable.

\subsection{Preprocessor options}

Three optional arguments to \class{Extension} will help if you need to

specify include directories to search or preprocessor macros to

define/undefine: \code{include\_dirs}, \code{define\_macros}, and

\code{undef\_macros}.

For example, if your extension requires header files in the

\file{include} directory under your distribution root, use the

\code{include\_dirs} option:

\begin{verbatim}

Extension('foo', ['foo.c'], include_dirs=['include'])

\end{verbatim}

You can specify absolute directories there; if you know that your

extension will only be built on \UNIX{} systems with X11R6 installed to

\file{/usr}, you can get away with

\begin{verbatim}

Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11'])

\end{verbatim}

You should avoid this sort of non-portable usage if you plan to

distribute your code: it's probably better to write C code like

\begin{verbatim}

#include <X11/Xlib.h>

\end{verbatim}

If you need to include header files from some other Python extension,

you can take advantage of the fact that header files are installed in a

consistent way by the Distutils \command{install\_header} command. For

example, the Numerical Python header files are installed (on a standard

Unix installation) to \file{/usr/local/include/python1.5/Numerical}.

(The exact location will differ according to your platform and Python

installation.) Since the Python include

directory---\file{/usr/local/include/python1.5} in this case---is always

included in the search path when building Python extensions, the best

approach is to write C code like

\begin{verbatim}

#include <Numerical/arrayobject.h>

\end{verbatim}

If you must put the \file{Numerical} include directory right into your

header search path, though, you can find that directory using the

Distutils \refmodule{distutils.sysconfig} module:

\begin{verbatim}

from distutils.sysconfig import get_python_inc

incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical')

setup(...,

Extension(..., include_dirs=[incdir]),

)

\end{verbatim}

Even though this is quite portable---it will work on any Python

installation, regardless of platform---it's probably easier to just

write your C code in the sensible way.

You can define and undefine pre-processor macros with the

\code{define\_macros} and \code{undef\_macros} options.

\code{define\_macros} takes a list of \code{(name, value)} tuples, where

\code{name} is the name of the macro to define (a string) and

\code{value} is its value: either a string or \code{None}. (Defining a

macro \code{FOO} to \code{None} is the equivalent of a bare

\code{\#define FOO} in your C source: with most compilers, this sets

\code{FOO} to the string \code{1}.) \code{undef\_macros} is just

a list of macros to undefine.

For example:

\begin{verbatim}

Extension(...,

define_macros=[('NDEBUG', '1'),

('HAVE_STRFTIME', None)],

undef_macros=['HAVE_FOO', 'HAVE_BAR'])

\end{verbatim}

is the equivalent of having this at the top of every C source file:

\begin{verbatim}

#define NDEBUG 1

#define HAVE_STRFTIME

#undef HAVE_FOO

#undef HAVE_BAR

\end{verbatim}

\subsection{Library options}

You can also specify the libraries to link against when building your

extension, and the directories to search for those libraries. The

\code{libraries} option is a list of libraries to link against,

\code{library\_dirs} is a list of directories to search for libraries at

link-time, and \code{runtime\_library\_dirs} is a list of directories to

search for shared (dynamically loaded) libraries at run-time.

For example, if you need to link against libraries known to be in the

standard library search path on target systems

\begin{verbatim}

Extension(...,

libraries=['gdbm', 'readline'])

\end{verbatim}

If you need to link with libraries in a non-standard location, you'll

have to include the location in \code{library\_dirs}:

\begin{verbatim}

Extension(...,

library_dirs=['/usr/X11R6/lib'],

libraries=['X11', 'Xt'])

\end{verbatim}

(Again, this sort of non-portable construct should be avoided if you

intend to distribute your code.)

\XXX{Should mention clib libraries here or somewhere else!}

\subsection{Other options}

There are still some other options which can be used to handle special

cases.

The \option{extra\_objects} option is a list of object files to be passed

to the linker. These files must not have extensions, as the default

extension for the compiler is used.

\option{extra\_compile\_args} and \option{extra\_link\_args} can be used

to specify additional command line options for the respective compiler and

linker command lines.

\option{export\_symbols} is only useful on Windows. It can contain a list

of symbols (functions or variables) to be exported. This option

is not needed when building compiled extensions: Distutils

will automatically add \code{initmodule}

to the list of exported symbols.

\section{Installing Scripts}

So far we have been dealing with pure and non-pure Python modules,

which are usually not run by themselves but imported by scripts.

Scripts are files containing Python source code, intended to be

started from the command line. Scripts don't require Distutils to do

anything very complicated. The only clever feature is that if the

first line of the script starts with \code{\#!} and contains the word

``python'', the Distutils will adjust the first line to refer to the

current interpreter location.

The \option{scripts} option simply is a list of files to be handled

in this way. From the PyXML setup script:

\begin{verbatim}

setup(...

scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val']

)

\end{verbatim}

\section{Installing Package Data}

Often, additional files need to be installed into a package. These

files are often data that's closely related to the package's

implementation, or text files containing documentation that might be

of interest to programmers using the package. These files are called

\dfn{package data}.

Package data can be added to packages using the \code{package_data}

keyword argument to the \function{setup()} function. The value must

be a mapping from package name to a list of relative path names that

should be copied into the package. The paths are interpreted as

relative to the directory containing the package (information from the

\code{package_dir} mapping is used if appropriate); that is, the files

are expected to be part of the package in the source directories.

They may contain glob patterns as well.

The path names may contain directory portions; any necessary

directories will be created in the installation.

For example, if a package should contain a subdirectory with several

data files, the files can be arranged like this in the source tree:

\begin{verbatim}

setup.py

src/

mypkg/

__init__.py

module.py

data/

tables.dat

spoons.dat

forks.dat

\end{verbatim}

The corresponding call to \function{setup()} might be:

\begin{verbatim}

setup(...,

packages=['mypkg'],

package_dir={'mypkg': 'src/mypkg'},

package_data={'mypkg': ['data/*.dat']},

)

\end{verbatim}

\versionadded{2.4}

\section{Installing Additional Files}

The \option{data\_files} option can be used to specify additional

files needed by the module distribution: configuration files, message

catalogs, data files, anything which doesn't fit in the previous

categories.

\option{data\_files} specifies a sequence of (\var{directory},

\var{files}) pairs in the following way:

\begin{verbatim}

setup(...

data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),

('config', ['cfg/data.cfg']),

('/etc/init.d', ['init-script'])]

)

\end{verbatim}

Note that you can specify the directory names where the data files

will be installed, but you cannot rename the data files themselves.

Each (\var{directory}, \var{files}) pair in the sequence specifies the

installation directory and the files to install there. If

\var{directory} is a relative path, it is interpreted relative to the

installation prefix (Python's \code{sys.prefix} for pure-Python

packages, \code{sys.exec_prefix} for packages that contain extension

modules). Each file name in \var{files} is interpreted relative to

the \file{setup.py} script at the top of the package source

distribution. No directory information from \var{files} is used to

determine the final location of the installed file; only the name of

the file is used.

You can specify the \option{data\_files} options as a simple sequence

of files without specifying a target directory, but this is not recommended,

and the \command{install} command will print a warning in this case.

To install data files directly in the target directory, an empty

string should be given as the directory.

\section{Additional meta-data}

\label{meta-data}

The setup script may include additional meta-data beyond the name and

version. This information includes:

\begin{tableiv}{l|l|l|c}{code}%

{Meta-Data}{Description}{Value}{Notes}

\lineiv{name}{name of the package}

{short string}{(1)}

\lineiv{version}{version of this release}

{short string}{(1)(2)}

\lineiv{author}{package author's name}

{short string}{(3)}

\lineiv{author_email}{email address of the package author}

{email address}{(3)}

\lineiv{maintainer}{package maintainer's name}

{short string}{(3)}

\lineiv{maintainer_email}{email address of the package maintainer}

{email address}{(3)}

\lineiv{url}{home page for the package}

{URL}{(1)}

\lineiv{description}{short, summary description of the package}

{short string}{}

\lineiv{long_description}{longer description of the package}

{long string}{}

\lineiv{download_url}{location where the package may be downloaded}

{URL}{(4)}

\lineiv{classifiers}{a list of Trove classifiers}

{list of strings}{(4)}

\end{tableiv}

\noindent Notes:

\begin{description}

\item[(1)] These fields are required.

\item[(2)] It is recommended that versions take the form

\emph{major.minor\optional{.patch\optional{.sub}}}.

\item[(3)] Either the author or the maintainer must be identified.

\item[(4)] These fields should not be used if your package is to be

compatible with Python versions prior to 2.2.3 or 2.3. The list is

available from the \ulink{PyPI website}{http://www.python.org/pypi}.

\item['short string'] A single line of text, not more than 200 characters.

\item['long string'] Multiple lines of plain text in reStructuredText

format (see \url{http://docutils.sf.net/}).

\item['list of strings'] See below.

\end{description}

None of the string values may be Unicode.

Encoding the version information is an art in itself. Python packages

generally adhere to the version format

\emph{major.minor\optional{.patch}\optional{sub}}. The major number is

0 for

initial, experimental releases of software. It is incremented for

releases that represent major milestones in a package. The minor

number is incremented when important new features are added to the

package. The patch number increments when bug-fix releases are

made. Additional trailing version information is sometimes used to

indicate sub-releases. These are "a1,a2,...,aN" (for alpha releases,

where functionality and API may change), "b1,b2,...,bN" (for beta

releases, which only fix bugs) and "pr1,pr2,...,prN" (for final

pre-release release testing). Some examples:

\begin{description}

\item[0.1.0] the first, experimental release of a package

\item[1.0.1a2] the second alpha release of the first patch version of 1.0

\end{description}

\option{classifiers} are specified in a python list:

\begin{verbatim}

setup(...

classifiers=[

'Development Status :: 4 - Beta',

'Environment :: Console',

'Environment :: Web Environment',

'Intended Audience :: End Users/Desktop',

'Intended Audience :: Developers',

'Intended Audience :: System Administrators',

'License :: OSI Approved :: Python Software Foundation License',

'Operating System :: MacOS :: MacOS X',

'Operating System :: Microsoft :: Windows',

'Operating System :: POSIX',

'Programming Language :: Python',

'Topic :: Communications :: Email',

'Topic :: Office/Business',

'Topic :: Software Development :: Bug Tracking',

],

)

\end{verbatim}

If you wish to include classifiers in your \file{setup.py} file and also

wish to remain backwards-compatible with Python releases prior to 2.2.3,

then you can include the following code fragment in your \file{setup.py}

before the \function{setup()} call.

\begin{verbatim}

# patch distutils if it can't cope with the "classifiers" or

# "download_url" keywords

if sys.version < '2.2.3':

from distutils.dist import DistributionMetadata

DistributionMetadata.classifiers = None

DistributionMetadata.download_url = None

\end{verbatim}

\section{Debugging the setup script}

Sometimes things go wrong, and the setup script doesn't do what the

developer wants.

Distutils catches any exceptions when running the setup script, and

print a simple error message before the script is terminated. The

motivation for this behaviour is to not confuse administrators who

don't know much about Python and are trying to install a package. If

they get a big long traceback from deep inside the guts of Distutils,

they may think the package or the Python installation is broken

because they don't read all the way down to the bottom and see that

it's a permission problem.

On the other hand, this doesn't help the developer to find the cause

of the failure. For this purpose, the DISTUTILS_DEBUG environment

variable can be set to anything except an empty string, and distutils

will now print detailed information what it is doing, and prints the

full traceback in case an exception occurs.

\chapter{Writing the Setup Configuration File}

\label{setup-config}

Often, it's not possible to write down everything needed to build a

distribution \emph{a priori}: you may need to get some information from

the user, or from the user's system, in order to proceed. As long as

that information is fairly simple---a list of directories to search for

C header files or libraries, for example---then providing a

configuration file, \file{setup.cfg}, for users to edit is a cheap and

easy way to solicit it. Configuration files also let you provide

default values for any command option, which the installer can then

override either on the command-line or by editing the config file.

% (If you have more advanced needs, such as determining which extensions

% to build based on what capabilities are present on the target system,

% then you need the Distutils ``auto-configuration'' facility. This

% started to appear in Distutils 0.9 but, as of this writing, isn't mature

% or stable enough yet for real-world use.)

The setup configuration file is a useful middle-ground between the setup

script---which, ideally, would be opaque to installers\footnote{This

ideal probably won't be achieved until auto-configuration is fully

supported by the Distutils.}---and the command-line to the setup

script, which is outside of your control and entirely up to the

installer. In fact, \file{setup.cfg} (and any other Distutils

configuration files present on the target system) are processed after

the contents of the setup script, but before the command-line. This has

several useful consequences:

\begin{itemize}

\item installers can override some of what you put in \file{setup.py} by

editing \file{setup.cfg}

\item you can provide non-standard defaults for options that are not

easily set in \file{setup.py}

\item installers can override anything in \file{setup.cfg} using the

command-line options to \file{setup.py}

\end{itemize}

The basic syntax of the configuration file is simple:

\begin{verbatim}

[command]

option=value

...

\end{verbatim}

where \var{command} is one of the Distutils commands (e.g.

\command{build\_py}, \command{install}), and \var{option} is one of

the options that command supports. Any number of options can be

supplied for each command, and any number of command sections can be

included in the file. Blank lines are ignored, as are comments, which

run from a \character{\#} character until the end of the line. Long

option values can be split across multiple lines simply by indenting

the continuation lines.

You can find out the list of options supported by a particular command

with the universal \longprogramopt{help} option, e.g.

\begin{verbatim}

> python setup.py --help build_ext

[...]

Options for 'build_ext' command:

--build-lib (-b) directory for compiled extension modules

--build-temp (-t) directory for temporary files (build by-products)

--inplace (-i) ignore build-lib and put compiled extensions into the

source directory alongside your pure Python modules

--include-dirs (-I) list of directories to search for header files

--define (-D) C preprocessor macros to define

--undef (-U) C preprocessor macros to undefine

[...]

\end{verbatim}

Note that an option spelled \longprogramopt{foo-bar} on the command-line

is spelled \option{foo\_bar} in configuration files.

For example, say you want your extensions to be built

``in-place''---that is, you have an extension \module{pkg.ext}, and you

want the compiled extension file (\file{ext.so} on \UNIX, say) to be put

in the same source directory as your pure Python modules

\module{pkg.mod1} and \module{pkg.mod2}. You can always use the

\longprogramopt{inplace} option on the command-line to ensure this:

\begin{verbatim}

python setup.py build_ext --inplace

\end{verbatim}

But this requires that you always specify the \command{build\_ext}

command explicitly, and remember to provide \longprogramopt{inplace}.

An easier way is to ``set and forget'' this option, by encoding it in

\file{setup.cfg}, the configuration file for this distribution:

\begin{verbatim}

[build_ext]

inplace=1

\end{verbatim}

This will affect all builds of this module distribution, whether or not

you explcitly specify \command{build\_ext}. If you include

\file{setup.cfg} in your source distribution, it will also affect

end-user builds---which is probably a bad idea for this option, since

always building extensions in-place would break installation of the

module distribution. In certain peculiar cases, though, modules are

built right in their installation directory, so this is conceivably a

useful ability. (Distributing extensions that expect to be built in

their installation directory is almost always a bad idea, though.)

Another example: certain commands take a lot of options that don't

change from run to run; for example, \command{bdist\_rpm} needs to know

everything required to generate a ``spec'' file for creating an RPM

distribution. Some of this information comes from the setup script, and

some is automatically generated by the Distutils (such as the list of

files installed). But some of it has to be supplied as options to

\command{bdist\_rpm}, which would be very tedious to do on the

command-line for every run. Hence, here is a snippet from the

Distutils' own \file{setup.cfg}:

\begin{verbatim}

[bdist_rpm]

release = 1

packager = Greg Ward <[email protected]>

doc_files = CHANGES.txt

README.txt

USAGE.txt

doc/

examples/

\end{verbatim}

Note that the \option{doc\_files} option is simply a

whitespace-separated string split across multiple lines for readability.

\begin{seealso}

\seetitle[../inst/config-syntax.html]{Installing Python