\documentclass{howto}

\usepackage{distutils}

% TODO:

% Fill in XXX comments

\title{Installing Python Modules}

% The audience for this document includes people who don't know anything

% about Python and aren't about to learn the language just in order to

% install and maintain it for their users, i.e. system administrators.

% Thus, I have to be sure to explain the basics at some point:

% sys.path and PYTHONPATH at least. Should probably give pointers to

% other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc.

%

% Finally, it might be useful to include all the material from my "Care

% and Feeding of a Python Installation" talk in here somewhere. Yow!

\input{boilerplate}

\author{Greg Ward}

\authoraddress{

\strong{Python Software Foundation}\\

Email: \email{[email protected]}

}

\makeindex

\begin{document}

\maketitle

\begin{abstract}

\noindent

This document describes the Python Distribution Utilities

(``Distutils'') from the end-user's point-of-view, describing how to

extend the capabilities of a standard Python installation by building

and installing third-party Python modules and extensions.

\end{abstract}

%\begin{abstract}

%\noindent

%Abstract this!

%\end{abstract}

% The ugly "%begin{latexonly}" pseudo-environment suppresses the table

% of contents for HTML generation.

%

%begin{latexonly}

\tableofcontents

%end{latexonly}

\section{Introduction}

\label{intro}

Although Python's extensive standard library covers many programming

needs, there often comes a time when you need to add some new

functionality to your Python installation in the form of third-party

modules. This might be necessary to support your own programming, or to

support an application that you want to use and that happens to be

written in Python.

In the past, there has been little support for adding third-party

modules to an existing Python installation. With the introduction of

the Python Distribution Utilities (Distutils for short) in Python 2.0,

this changed.

This document is aimed primarily at the people who need to install

third-party Python modules: end-users and system administrators who just

need to get some Python application running, and existing Python

programmers who want to add some new goodies to their toolbox. You

don't need to know Python to read this document; there will be some

brief forays into using Python's interactive mode to explore your

installation, but that's it. If you're looking for information on how

to distribute your own Python modules so that others may use them, see

the \citetitle[../dist/dist.html]{Distributing Python Modules} manual.

\subsection{Best case: trivial installation}

\label{trivial-install}

In the best case, someone will have prepared a special version of the

module distribution you want to install that is targeted specifically at

your platform and is installed just like any other software on your

platform. For example, the module developer might make an executable

installer available for Windows users, an RPM package for users of

RPM-based Linux systems (Red Hat, SuSE, Mandrake, and many others), a

Debian package for users of Debian-based Linux systems, and so forth.

In that case, you would download the installer appropriate to your

platform and do the obvious thing with it: run it if it's an executable

installer, \code{rpm --install} it if it's an RPM, etc. You don't need

to run Python or a setup script, you don't need to compile

anything---you might not even need to read any instructions (although

it's always a good idea to do so anyways).

Of course, things will not always be that easy. You might be interested

in a module distribution that doesn't have an easy-to-use installer for

your platform. In that case, you'll have to start with the source

distribution released by the module's author/maintainer. Installing

from a source distribution is not too hard, as long as the modules are

packaged in the standard way. The bulk of this document is about

building and installing modules from standard source distributions.

\subsection{The new standard: Distutils}

\label{new-standard}

If you download a module source distribution, you can tell pretty

quickly if it was packaged and distributed in the standard way, i.e.

using the Distutils. First, the distribution's name and version number

will be featured prominently in the name of the downloaded archive, e.g.

\file{foo-1.0.tar.gz} or \file{widget-0.9.7.zip}. Next, the archive

will unpack into a similarly-named directory: \file{foo-1.0} or

\file{widget-0.9.7}. Additionally, the distribution will contain a

setup script \file{setup.py}, and a file named \file{README.txt} or possibly

just \file{README}, which should explain that building and installing the

module distribution is a simple matter of running

\begin{verbatim}

python setup.py install

\end{verbatim}

If all these things are true, then you already know how to build and

install the modules you've just downloaded: Run the command above.

Unless you need to install things in a non-standard way or customize the

build process, you don't really need this manual. Or rather, the above

command is everything you need to get out of this manual.

\section{Standard Build and Install}

\label{standard-install}

As described in section~\ref{new-standard}, building and installing

a module distribution using the Distutils is usually one simple command:

\begin{verbatim}

python setup.py install

\end{verbatim}

On \UNIX, you'd run this command from a shell prompt; on Windows, you

have to open a command prompt window (``DOS box'') and do it there; on

Mac OS X, you open a \command{Terminal} window to get a shell prompt.

\subsection{Platform variations}

\label{platform-variations}

You should always run the setup command from the distribution root

directory, i.e. the top-level subdirectory that the module source

distribution unpacks into. For example, if you've just downloaded a

module source distribution \file{foo-1.0.tar.gz} onto a

\UNIX{} system, the normal thing to do is:

\begin{verbatim}

gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0

cd foo-1.0

python setup.py install

\end{verbatim}

On Windows, you'd probably download \file{foo-1.0.zip}. If you

downloaded the archive file to \file{C:\textbackslash{}Temp}, then it

would unpack into \file{C:\textbackslash{}Temp\textbackslash{}foo-1.0};

you can use either a archive manipulator with a graphical user interface

(such as WinZip) or a command-line tool (such as \program{unzip} or

\program{pkunzip}) to unpack the archive. Then, open a command prompt

window (``DOS box''), and run:

\begin{verbatim}

cd c:\Temp\foo-1.0

python setup.py install

\end{verbatim}

\subsection{Splitting the job up}

\label{splitting-up}

Running \code{setup.py install} builds and installs all modules in one

run. If you prefer to work incrementally---especially useful if you

want to customize the build process, or if things are going wrong---you

can use the setup script to do one thing at a time. This is

particularly helpful when the build and install will be done by

different users---for example, you might want to build a module distribution

and hand it off to a system administrator for installation (or do it

yourself, with super-user privileges).

For example, you can build everything in one step, and then install

everything in a second step, by invoking the setup script twice:

\begin{verbatim}

python setup.py build

python setup.py install

\end{verbatim}

If you do this, you will notice that running the \command{install}

command first runs the \command{build} command, which---in this

case---quickly notices that it has nothing to do, since everything in

the \file{build} directory is up-to-date.

You may not need this ability to break things down often if all you do

is install modules downloaded off the 'net, but it's very handy for more

advanced tasks. If you get into distributing your own Python modules

and extensions, you'll run lots of individual Distutils commands on

their own.

\subsection{How building works}

\label{how-build-works}

As implied above, the \command{build} command is responsible for putting

the files to install into a \emph{build directory}. By default, this is

\file{build} under the distribution root; if you're excessively

concerned with speed, or want to keep the source tree pristine, you can

change the build directory with the \longprogramopt{build-base} option.

For example:

\begin{verbatim}

python setup.py build --build-base=/tmp/pybuild/foo-1.0

\end{verbatim}

(Or you could do this permanently with a directive in your system or

personal Distutils configuration file; see

section~\ref{config-files}.) Normally, this isn't necessary.

The default layout for the build tree is as follows:

\begin{verbatim}

--- build/ --- lib/

or

--- build/ --- lib.<plat>/

temp.<plat>/

\end{verbatim}

where \code{<plat>} expands to a brief description of the current

OS/hardware platform and Python version. The first form, with just a

\file{lib} directory, is used for ``pure module distributions''---that

is, module distributions that include only pure Python modules. If a

module distribution contains any extensions (modules written in C/\Cpp),

then the second form, with two \code{<plat>} directories, is used. In

that case, the \file{temp.\filevar{plat}} directory holds temporary

files generated by the compile/link process that don't actually get

installed. In either case, the \file{lib} (or

\file{lib.\filevar{plat}}) directory contains all Python modules (pure

Python and extensions) that will be installed.

In the future, more directories will be added to handle Python scripts,

documentation, binary executables, and whatever else is needed to handle

the job of installing Python modules and applications.

\subsection{How installation works}

\label{how-install-works}

After the \command{build} command runs (whether you run it explicitly,

or the \command{install} command does it for you), the work of the

\command{install} command is relatively simple: all it has to do is copy

everything under \file{build/lib} (or \file{build/lib.\filevar{plat}})

to your chosen installation directory.

If you don't choose an installation directory---i.e., if you just run

\code{setup.py install}---then the \command{install} command installs to

the standard location for third-party Python modules. This location

varies by platform and by how you built/installed Python itself. On

\UNIX{} (and Mac OS X, which is also Unix-based),

it also depends on whether the module distribution

being installed is pure Python or contains extensions (``non-pure''):

\begin{tableiv}{l|l|l|c}{textrm}%

{Platform}{Standard installation location}{Default value}{Notes}

\lineiv{\UNIX{} (pure)}

{\filenq{\filevar{prefix}/lib/python\shortversion/site-packages}}

{\filenq{/usr/local/lib/python\shortversion/site-packages}}

{(1)}

\lineiv{\UNIX{} (non-pure)}

{\filenq{\filevar{exec-prefix}/lib/python\shortversion/site-packages}}

{\filenq{/usr/local/lib/python\shortversion/site-packages}}

{(1)}

\lineiv{Windows}

{\filenq{\filevar{prefix}}}

{\filenq{C:\textbackslash{}Python}}

{(2)}

\end{tableiv}

\noindent Notes:

\begin{description}

\item[(1)] Most Linux distributions include Python as a standard part of

the system, so \filevar{prefix} and \filevar{exec-prefix} are usually

both \file{/usr} on Linux. If you build Python yourself on Linux (or

any \UNIX-like system), the default \filevar{prefix} and

\filevar{exec-prefix} are \file{/usr/local}.

\item[(2)] The default installation directory on Windows was

\file{C:\textbackslash{}Program Files\textbackslash{}Python} under

Python 1.6a1, 1.5.2, and earlier.

\end{description}

\filevar{prefix} and \filevar{exec-prefix} stand for the directories

that Python is installed to, and where it finds its libraries at

run-time. They are always the same under Windows, and very

often the same under \UNIX and Mac OS X. You can find out what your Python

installation uses for \filevar{prefix} and \filevar{exec-prefix} by

running Python in interactive mode and typing a few simple commands.

Under \UNIX, just type \code{python} at the shell prompt. Under

Windows, choose \menuselection{Start \sub Programs \sub Python

\shortversion \sub Python (command line)}.

Once the interpreter is started, you type Python code at the

prompt. For example, on my Linux system, I type the three Python

statements shown below, and get the output as shown, to find out my

\filevar{prefix} and \filevar{exec-prefix}:

\begin{verbatim}

Python 2.4 (#26, Aug 7 2004, 17:19:02)

Type "help", "copyright", "credits" or "license" for more information.

>>> import sys

>>> sys.prefix

'/usr'

>>> sys.exec_prefix

'/usr'

\end{verbatim}

If you don't want to install modules to the standard location, or if you

don't have permission to write there, then you need to read about

alternate installations in section~\ref{alt-install}. If you want to

customize your installation directories more heavily, see

section~\ref{custom-install} on custom installations.

% This rather nasty macro is used to generate the tables that describe

% each installation scheme. It's nasty because it takes two arguments

% for each "slot" in an installation scheme, there will soon be more

% than five of these slots, and TeX has a limit of 10 arguments to a

% macro. Uh-oh.

\newcommand{\installscheme}[8]

{\begin{tableiii}{l|l|l}{textrm}

{Type of file}

{Installation Directory}

{Override option}

\lineiii{pure module distribution}

{\filevar{#1}\filenq{#2}}

{\longprogramopt{install-purelib}}

\lineiii{non-pure module distribution}

{\filevar{#3}\filenq{#4}}

{\longprogramopt{install-platlib}}

\lineiii{scripts}

{\filevar{#5}\filenq{#6}}

{\longprogramopt{install-scripts}}

\lineiii{data}

{\filevar{#7}\filenq{#8}}

{\longprogramopt{install-data}}

\end{tableiii}}

\section{Alternate Installation}

\label{alt-install}

Often, it is necessary or desirable to install modules to a location

other than the standard location for third-party Python modules. For

example, on a \UNIX{} system you might not have permission to write to the

standard third-party module directory. Or you might wish to try out a

module before making it a standard part of your local Python

installation. This is especially true when upgrading a distribution

already present: you want to make sure your existing base of scripts

still works with the new version before actually upgrading.

The Distutils \command{install} command is designed to make installing

module distributions to an alternate location simple and painless. The

basic idea is that you supply a base directory for the installation, and

the \command{install} command picks a set of directories (called an

\emph{installation scheme}) under this base directory in which to

install files. The details differ across platforms, so read whichever

of the following sections applies to you.

\subsection{Alternate installation: the home scheme}

\label{alt-install-prefix}

The idea behind the ``home scheme'' is that you build and maintain a

personal stash of Python modules. This scheme's name is derived from

the idea of a ``home'' directory on \UNIX, since it's not unusual for

a \UNIX{} user to make their home directory have a layout similar to

\file{/usr/} or \file{/usr/local/}. This scheme can be used by

anyone, regardless of the operating system their installing for.

Installing a new module distribution is as simple as

\begin{verbatim}

python setup.py install --home=<dir>

\end{verbatim}

where you can supply any directory you like for the

\longprogramopt{home} option. On \UNIX, lazy typists can just type a

tilde (\code{\textasciitilde}); the \command{install} command will

expand this to your home directory:

\begin{verbatim}

python setup.py install --home=~

\end{verbatim}

The \longprogramopt{home} option defines the installation base

directory. Files are installed to the following directories under the

installation base as follows:

\installscheme{home}{/lib/python}

{home}{/lib/python}

{home}{/bin}

{home}{/share}

\versionchanged[The \longprogramopt{home} option used to be supported

only on \UNIX]{2.4}

\subsection{Alternate installation: \UNIX{} (the prefix scheme)}

\label{alt-install-home}

The ``prefix scheme'' is useful when you wish to use one Python

installation to perform the build/install (i.e., to run the setup

script), but install modules into the third-party module directory of a

different Python installation (or something that looks like a different

Python installation). If this sounds a trifle unusual, it is---that's

why the ``home scheme'' comes first. However, there are at least two

known cases where the prefix scheme will be useful.

First, consider that many Linux distributions put Python in \file{/usr},

rather than the more traditional \file{/usr/local}. This is entirely

appropriate, since in those cases Python is part of ``the system''

rather than a local add-on. However, if you are installing Python

modules from source, you probably want them to go in

\file{/usr/local/lib/python2.\filevar{X}} rather than

\file{/usr/lib/python2.\filevar{X}}. This can be done with

\begin{verbatim}

/usr/bin/python setup.py install --prefix=/usr/local

\end{verbatim}

Another possibility is a network filesystem where the name used to write

to a remote directory is different from the name used to read it: for

example, the Python interpreter accessed as \file{/usr/local/bin/python}

might search for modules in \file{/usr/local/lib/python2.\filevar{X}},

but those modules would have to be installed to, say,

\file{/mnt/\filevar{@server}/export/lib/python2.\filevar{X}}. This

could be done with

\begin{verbatim}

/usr/local/bin/python setup.py install --prefix=/mnt/@server/export

\end{verbatim}

In either case, the \longprogramopt{prefix} option defines the

installation base, and the \longprogramopt{exec-prefix} option defines

the platform-specific installation base, which is used for

platform-specific files. (Currently, this just means non-pure module

distributions, but could be expanded to C libraries, binary executables,

etc.) If \longprogramopt{exec-prefix} is not supplied, it defaults to

\longprogramopt{prefix}. Files are installed as follows:

\installscheme{prefix}{/lib/python2.\filevar{X}/site-packages}

{exec-prefix}{/lib/python2.\filevar{X}/site-packages}

{prefix}{/bin}

{prefix}{/share}

There is no requirement that \longprogramopt{prefix} or

\longprogramopt{exec-prefix} actually point to an alternate Python

installation; if the directories listed above do not already exist, they

are created at installation time.

Incidentally, the real reason the prefix scheme is important is simply

that a standard \UNIX{} installation uses the prefix scheme, but with

\longprogramopt{prefix} and \longprogramopt{exec-prefix} supplied by

Python itself as \code{sys.prefix} and \code{sys.exec\_prefix}. Thus,

you might think you'll never use the prefix scheme, but every time you

run \code{python setup.py install} without any other options, you're

using it.

Note that installing extensions to an alternate Python installation has

no effect on how those extensions are built: in particular, the Python

header files (\file{Python.h} and friends) installed with the Python

interpreter used to run the setup script will be used in compiling

extensions. It is your responsibility to ensure that the interpreter

used to run extensions installed in this way is compatible with the

interpreter used to build them. The best way to do this is to ensure

that the two interpreters are the same version of Python (possibly

different builds, or possibly copies of the same build). (Of course, if

your \longprogramopt{prefix} and \longprogramopt{exec-prefix} don't even

point to an alternate Python installation, this is immaterial.)

\subsection{Alternate installation: Windows (the prefix scheme)}

\label{alt-install-windows}

Windows has no concept of a user's home directory, and since the

standard Python installation under Windows is simpler than under

\UNIX, the \longprogramopt{prefix} option has traditionally been used

to install additional packages in separate locations on Windows.

\begin{verbatim}

python setup.py install --prefix="\Temp\Python"

\end{verbatim}

to install modules to the

\file{\textbackslash{}Temp\textbackslash{}Python} directory on the

current drive.

The installation base is defined by the \longprogramopt{prefix} option;

the \longprogramopt{exec-prefix} option is not supported under Windows.

Files are installed as follows:

\installscheme{prefix}{}

{prefix}{}

{prefix}{\textbackslash{}Scripts}

{prefix}{\textbackslash{}Data}

\section{Custom Installation}

\label{custom-install}

Sometimes, the alternate installation schemes described in

section~\ref{alt-install} just don't do what you want. You might

want to tweak just one or two directories while keeping everything under

the same base directory, or you might want to completely redefine the

installation scheme. In either case, you're creating a \emph{custom

installation scheme}.

You probably noticed the column of ``override options'' in the tables

describing the alternate installation schemes above. Those options are

how you define a custom installation scheme. These override options can

be relative, absolute, or explicitly defined in terms of one of the

installation base directories. (There are two installation base

directories, and they are normally the same---they only differ when you

use the \UNIX{} ``prefix scheme'' and supply different

\longprogramopt{prefix} and \longprogramopt{exec-prefix} options.)

For example, say you're installing a module distribution to your home

directory under \UNIX---but you want scripts to go in

\file{\textasciitilde/scripts} rather than \file{\textasciitilde/bin}.

As you might expect, you can override this directory with the

\longprogramopt{install-scripts} option; in this case, it makes most

sense to supply a relative path, which will be interpreted relative to

the installation base directory (your home directory, in this case):

\begin{verbatim}

python setup.py install --home=~ --install-scripts=scripts

\end{verbatim}

Another \UNIX{} example: suppose your Python installation was built and

installed with a prefix of \file{/usr/local/python}, so under a standard

installation scripts will wind up in \file{/usr/local/python/bin}. If

you want them in \file{/usr/local/bin} instead, you would supply this

absolute directory for the \longprogramopt{install-scripts} option:

\begin{verbatim}

python setup.py install --install-scripts=/usr/local/bin

\end{verbatim}

(This performs an installation using the ``prefix scheme,'' where the

prefix is whatever your Python interpreter was installed with---

\file{/usr/local/python} in this case.)

If you maintain Python on Windows, you might want third-party modules to

live in a subdirectory of \filevar{prefix}, rather than right in

\filevar{prefix} itself. This is almost as easy as customizing the

script installation directory---you just have to remember that there are

two types of modules to worry about, pure modules and non-pure modules

(i.e., modules from a non-pure distribution). For example:

\begin{verbatim}

python setup.py install --install-purelib=Site --install-platlib=Site

\end{verbatim}

The specified installation directories are relative to

\filevar{prefix}. Of course, you also have to ensure that these

directories are in Python's module search path, such as by putting a

\file{.pth} file in \filevar{prefix}. See section~\ref{search-path}

to find out how to modify Python's search path.

If you want to define an entire installation scheme, you just have to

supply all of the installation directory options. The recommended way

to do this is to supply relative paths; for example, if you want to

maintain all Python module-related files under \file{python} in your

home directory, and you want a separate directory for each platform that

you use your home directory from, you might define the following

installation scheme:

\begin{verbatim}

python setup.py install --home=~ \

--install-purelib=python/lib \

--install-platlib=python/lib.$PLAT \

--install-scripts=python/scripts

--install-data=python/data

\end{verbatim}

% $ % -- bow to font-lock

or, equivalently,

\begin{verbatim}

python setup.py install --home=~/python \

--install-purelib=lib \

--install-platlib='lib.$PLAT' \

--install-scripts=scripts

--install-data=data

\end{verbatim}

% $ % -- bow to font-lock

\code{\$PLAT} is not (necessarily) an environment variable---it will be

expanded by the Distutils as it parses your command line options, just

as it does when parsing your configuration file(s).

Obviously, specifying the entire installation scheme every time you

install a new module distribution would be very tedious. Thus, you can

put these options into your Distutils config file (see

section~\ref{config-files}):

\begin{verbatim}

[install]

install-base=$HOME

install-purelib=python/lib

install-platlib=python/lib.$PLAT

install-scripts=python/scripts

install-data=python/data

\end{verbatim}

or, equivalently,

\begin{verbatim}

[install]

install-base=$HOME/python

install-purelib=lib

install-platlib=lib.$PLAT

install-scripts=scripts

install-data=data

\end{verbatim}

Note that these two are \emph{not} equivalent if you supply a different

installation base directory when you run the setup script. For example,

\begin{verbatim}

python setup.py --install-base=/tmp

\end{verbatim}

would install pure modules to \filevar{/tmp/python/lib} in the first

case, and to \filevar{/tmp/lib} in the second case. (For the second

case, you probably want to supply an installation base of

\file{/tmp/python}.)

You probably noticed the use of \code{\$HOME} and \code{\$PLAT} in the

sample configuration file input. These are Distutils configuration

variables, which bear a strong resemblance to environment variables.

In fact, you can use environment variables in config files on

platforms that have such a notion but the Distutils additionally

define a few extra variables that may not be in your environment, such

as \code{\$PLAT}. (And of course, on systems that don't have

environment variables, such as Mac OS 9, the configuration

variables supplied by the Distutils are the only ones you can use.)

See section~\ref{config-files} for details.

% XXX need some Windows examples---when would custom

% installation schemes be needed on those platforms?

% XXX I'm not sure where this section should go.

\subsection{Modifying Python's Search Path}

\label{search-path}

When the Python interpreter executes an \keyword{import} statement, it

searches for both Python code and extension modules along a search

path. A default value for the path is configured into the Python

binary when the interpreter is built. You can determine the path by

importing the \module{sys} module and printing the value of

\code{sys.path}.

\begin{verbatim}

$ python

Python 2.2 (#11, Oct 3 2002, 13:31:27)

[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2

Type ``help'', ``copyright'', ``credits'' or ``license'' for more information.

>>> import sys

>>> sys.path

['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2',

'/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload',

'/usr/local/lib/python2.3/site-packages']

>>>

\end{verbatim} % $ <-- bow to font-lock

The null string in \code{sys.path} represents the current working

directory.

The expected convention for locally installed packages is to put them

in the \file{.../site-packages/} directory, but you may want to

install Python modules into some arbitrary directory. For example,

your site may have a convention of keeping all software related to the

web server under \file{/www}. Add-on Python modules might then belong

in \file{/www/python}, and in order to import them, this directory

must be added to \code{sys.path}. There are several different ways to

add the directory.

The most convenient way is to add a path configuration file to a

directory that's already on Python's path, usually to the

\file{.../site-packages/} directory. Path configuration files have an

extension of \file{.pth}, and each line must contain a single path

that will be appended to \code{sys.path}. (Because the new paths are

appended to \code{sys.path}, modules in the added directories will not

override standard modules. This means you can't use this mechanism

for installing fixed versions of standard modules.)

Paths can be absolute or relative, in which case they're relative to

the directory containing the \file{.pth} file. Any directories added

to the search path will be scanned in turn for \file{.pth} files. See

\citetitle[http://www.python.org/dev/doc/devel/lib/module-site.html]

{site module documentation} for more information.

A slightly less convenient way is to edit the \file{site.py} file in

Python's standard library, and modify \code{sys.path}. \file{site.py}

is automatically imported when the Python interpreter is executed,

unless the \programopt{-S} switch is supplied to suppress this

behaviour. So you could simply edit \file{site.py} and add two lines to it:

\begin{verbatim}

import sys

sys.path.append('/www/python/')

\end{verbatim}

However, if you reinstall the same major version of Python (perhaps

when upgrading from 2.2 to 2.2.2, for example) \file{site.py} will be

overwritten by the stock version. You'd have to remember that it was

modified and save a copy before doing the installation.

There are two environment variables that can modify \code{sys.path}.

\envvar{PYTHONHOME} sets an alternate value for the prefix of the

Python installation. For example, if \envvar{PYTHONHOME} is set to

\samp{/www/python}, the search path will be set to \code{['',

'/www/python/lib/python2.2/', '/www/python/lib/python2.3/plat-linux2',

...]}.

The \envvar{PYTHONPATH} variable can be set to a list of paths that

will be added to the beginning of \code{sys.path}. For example, if

\envvar{PYTHONPATH} is set to \samp{/www/python:/opt/py}, the search

path will begin with \code{['/www/python', '/opt/py']}. (Note that

directories must exist in order to be added to \code{sys.path}; the

\module{site} module removes paths that don't exist.)

Finally, \code{sys.path} is just a regular Python list, so any Python

application can modify it by adding or removing entries.

\section{Distutils Configuration Files}

\label{config-files}

As mentioned above, you can use Distutils configuration files to record

personal or site preferences for any Distutils options. That is, any

option to any command can be stored in one of two or three (depending on

your platform) configuration files, which will be consulted before the

command-line is parsed. This means that configuration files will

override default values, and the command-line will in turn override

configuration files. Furthermore, if multiple configuration files

apply, values from ``earlier'' files are overridden by ``later'' files.

\subsection{Location and names of config files}

\label{config-filenames}

The names and locations of the configuration files vary slightly across

platforms. On \UNIX{} and Mac OS X, the three configuration files (in

the order they are processed) are:

\begin{tableiii}{l|l|c}{textrm}

{Type of file}{Location and filename}{Notes}

\lineiii{system}{\filenq{\filevar{prefix}/lib/python\filevar{ver}/distutils/distutils.cfg}}{(1)}

\lineiii{personal}{\filenq{\$HOME/.pydistutils.cfg}}{(2)}

\lineiii{local}{\filenq{setup.cfg}}{(3)}

\end{tableiii}

And on Windows, the configuration files are:

\begin{tableiii}{l|l|c}{textrm}

{Type of file}{Location and filename}{Notes}

\lineiii{system}{\filenq{\filevar{prefix}\textbackslash{}Lib\textbackslash{}distutils\textbackslash{}distutils.cfg}}{(4)}

\lineiii{personal}{\filenq{\%HOME\%\textbackslash{}pydistutils.cfg}}{(5)}

\lineiii{local}{\filenq{setup.cfg}}{(3)}

\end{tableiii}

\noindent Notes:

\begin{description}

\item[(1)] Strictly speaking, the system-wide configuration file lives

in the directory where the Distutils are installed; under Python 1.6

and later on \UNIX, this is as shown. For Python 1.5.2, the Distutils

will normally be installed to

\file{\filevar{prefix}/lib/python1.5/site-packages/distutils},

so the system configuration file should be put there under Python

1.5.2.

\item[(2)] On \UNIX, if the \envvar{HOME} environment variable is not

defined, the user's home directory will be determined with the

\function{getpwuid()} function from the standard

\ulink{\module{pwd}}{../lib/module-pwd.html} module.

\item[(3)] I.e., in the current directory (usually the location of the

setup script).

\item[(4)] (See also note (1).) Under Python 1.6 and later, Python's

default ``installation prefix'' is \file{C:\textbackslash{}Python}, so

the system configuration file is normally

\file{C:\textbackslash{}Python\textbackslash{}Lib\textbackslash{}distutils\textbackslash{}distutils.cfg}.

Under Python 1.5.2, the default prefix was

\file{C:\textbackslash{}Program~Files\textbackslash{}Python}, and the

Distutils were not part of the standard library---so the system

configuration file would be

\file{C:\textbackslash{}Program~Files\textbackslash{}Python\textbackslash{}distutils\textbackslash{}distutils.cfg}

in a standard Python 1.5.2 installation under Windows.

\item[(5)] On Windows, if the \envvar{HOME} environment variable is not

defined, no personal configuration file will be found or used. (In

other words, the Distutils make no attempt to guess your home

directory on Windows.)

\end{description}

\subsection{Syntax of config files}

\label{config-syntax}

The Distutils configuration files all have the same syntax. The config

files are grouped into sections. There is one section for each Distutils

command, plus a \code{global} section for global options that affect

every command. Each section consists of one option per line, specified

as \code{option=value}.

For example, the following is a complete config file that just forces

all commands to run quietly by default:

\begin{verbatim}

[global]

verbose=0

\end{verbatim}

If this is installed as the system config file, it will affect all

processing of any Python module distribution by any user on the current

system. If it is installed as your personal config file (on systems

that support them), it will affect only module distributions processed

by you. And if it is used as the \file{setup.cfg} for a particular

module distribution, it affects only that distribution.

You could override the default ``build base'' directory and make the

\command{build*} commands always forcibly rebuild all files with the

following:

\begin{verbatim}

[build]

build-base=blib

force=1

\end{verbatim}

which corresponds to the command-line arguments

\begin{verbatim}

python setup.py build --build-base=blib --force

\end{verbatim}

except that including the \command{build} command on the command-line

means that command will be run. Including a particular command in

config files has no such implication; it only means that if the command

is run, the options in the config file will apply. (Or if other

commands that derive values from it are run, they will use the values in

the config file.)

You can find out the complete list of options for any command using the

\longprogramopt{help} option, e.g.:

\begin{verbatim}

python setup.py build --help

\end{verbatim}

and you can find out the complete list of global options by using

\longprogramopt{help} without a command:

\begin{verbatim}

python setup.py --help

\end{verbatim}

See also the ``Reference'' section of the ``Distributing Python

Modules'' manual.

\section{Building Extensions: Tips and Tricks}

\label{building-ext}

Whenever possible, the Distutils try to use the configuration

information made available by the Python interpreter used to run the

\file{setup.py} script. For example, the same compiler and linker

flags used to compile Python will also be used for compiling

extensions. Usually this will work well, but in complicated

situations this might be inappropriate. This section discusses how to

override the usual Distutils behaviour.

\subsection{Tweaking compiler/linker flags}

\label{tweak-flags}

Compiling a Python extension written in C or \Cpp{} will sometimes

require specifying custom flags for the compiler and linker in order

to use a particular library or produce a special kind of object code.

This is especially true if the extension hasn't been tested on your

platform, or if you're trying to cross-compile Python.

In the most general case, the extension author might have foreseen

that compiling the extensions would be complicated, and provided a

\file{Setup} file for you to edit. This will likely only be done if

the module distribution contains many separate extension modules, or

if they often require elaborate sets of compiler flags in order to work.

A \file{Setup} file, if present, is parsed in order to get a list of

extensions to build. Each line in a \file{Setup} describes a single

module. Lines have the following structure:

\begin{alltt}

\var{module} ... [\var{sourcefile} ...] [\var{cpparg} ...] [\var{library} ...]

\end{alltt}

Let's examine each of the fields in turn.

\begin{itemize}

\item \var{module} is the name of the extension module to be built,

and should be a valid Python identifier. You can't just change

this in order to rename a module (edits to the source code would

also be needed), so this should be left alone.

\item \var{sourcefile} is anything that's likely to be a source code

file, at least judging by the filename. Filenames ending in

\file{.c} are assumed to be written in C, filenames ending in

\file{.C}, \file{.cc}, and \file{.c++} are assumed to be

\Cpp, and filenames ending in \file{.m} or \file{.mm} are

assumed to be in Objective C.

\item \var{cpparg} is an argument for the C preprocessor,

and is anything starting with \programopt{-I}, \programopt{-D},

\programopt{-U} or \programopt{-C}.

\item \var{library} is anything ending in \file{.a} or beginning with

\programopt{-l} or \programopt{-L}.

\end{itemize}

If a particular platform requires a special library on your platform,

you can add it by editing the \file{Setup} file and running

\code{python setup.py build}. For example, if the module defined by the line

\begin{verbatim}

foo foomodule.c

\end{verbatim}

must be linked with the math library \file{libm.a} on your platform,

simply add \programopt{-lm} to the line:

\begin{verbatim}

foo foomodule.c -lm

\end{verbatim}

Arbitrary switches intended for the compiler or the linker can be

supplied with the \programopt{-Xcompiler} \var{arg} and

\programopt{-Xlinker} \var{arg} options:

\begin{verbatim}

foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm

\end{verbatim}

The next option after \programopt{-Xcompiler} and

\programopt{-Xlinker} will be appended to the proper command line, so

in the above example the compiler will be passed the \programopt{-o32}

option, and the linker will be passed \programopt{-shared}. If a

compiler option requires an argument, you'll have to supply multiple

\programopt{-Xcompiler} options; for example, to pass \code{-x c++} the

\file{Setup} file would have to contain

\code{-Xcompiler -x -Xcompiler c++}.

Compiler flags can also be supplied through setting the

\envvar{CFLAGS} environment variable. If set, the contents of

\envvar{CFLAGS} will be added to the compiler flags specified in the

\file{Setup} file.

\subsection{Using non-Microsoft compilers on Windows \label{non-ms-compilers}}

\sectionauthor{Rene Liebscher}{[email protected]}

\subsubsection{Borland \Cpp}

This subsection describes the necessary steps to use Distutils with the

Borland \Cpp{} compiler version 5.5.

%Should we mention that users have to create cfg-files for the compiler?

%see also http://community.borland.com/article/0,1410,21205,00.html

First you have to know that Borland's object file format (OMF) is

different from the format used by the Python version you can download

from the Python or ActiveState Web site. (Python is built with

Microsoft Visual \Cpp, which uses COFF as the object file format.)

For this reason you have to convert Python's library

\file{python24.lib} into the Borland format. You can do this as

follows:

\begin{verbatim}

coff2omf python24.lib python24_bcpp.lib

\end{verbatim}

The \file{coff2omf} program comes with the Borland compiler. The file

\file{python24.lib} is in the \file{Libs} directory of your Python

installation. If your extension uses other libraries (zlib,...) you

have to convert them too.

The converted files have to reside in the same directories as the

normal libraries.

How does Distutils manage to use these libraries with their changed

names? If the extension needs a library (eg. \file{foo}) Distutils