\documentclass{manual}

\usepackage[T1]{fontenc}

% Things to do:

% Should really move the Python startup file info to an appendix

\title{Python Tutorial}

\input{boilerplate}

\makeindex

\begin{document}

\maketitle

\ifhtml

\chapter*{Front Matter\label{front}}

\fi

\input{copyright}

\begin{abstract}

\noindent

Python is an easy to learn, powerful programming language. It has

efficient high-level data structures and a simple but effective

approach to object-oriented programming. Python's elegant syntax and

dynamic typing, together with its interpreted nature, make it an ideal

language for scripting and rapid application development in many areas

on most platforms.

The Python interpreter and the extensive standard library are freely

available in source or binary form for all major platforms from the

Python Web site, \url{http://www.python.org/}, and can be freely

distributed. The same site also contains distributions of and

pointers to many free third party Python modules, programs and tools,

and additional documentation.

The Python interpreter is easily extended with new functions and data

types implemented in C or \Cpp{} (or other languages callable from C).

Python is also suitable as an extension language for customizable

applications.

This tutorial introduces the reader informally to the basic concepts

and features of the Python language and system. It helps to have a

Python interpreter handy for hands-on experience, but all examples are

self-contained, so the tutorial can be read off-line as well.

For a description of standard objects and modules, see the

\citetitle[../lib/lib.html]{Python Library Reference} document. The

\citetitle[../ref/ref.html]{Python Reference Manual} gives a more

formal definition of the language. To write extensions in C or

\Cpp, read \citetitle[../ext/ext.html]{Extending and Embedding the

Python Interpreter} and \citetitle[../api/api.html]{Python/C API

Reference}. There are also several books covering Python in depth.

This tutorial does not attempt to be comprehensive and cover every

single feature, or even every commonly used feature. Instead, it

introduces many of Python's most noteworthy features, and will give

you a good idea of the language's flavor and style. After reading it,

you will be able to read and write Python modules and programs, and

you will be ready to learn more about the various Python library

modules described in the \citetitle[../lib/lib.html]{Python Library

Reference}.

\end{abstract}

\tableofcontents

\chapter{Whetting Your Appetite \label{intro}}

If you ever wrote a large shell script, you probably know this

feeling: you'd love to add yet another feature, but it's already so

slow, and so big, and so complicated; or the feature involves a system

call or other function that is only accessible from C \ldots Usually

the problem at hand isn't serious enough to warrant rewriting the

script in C; perhaps the problem requires variable-length strings or

other data types (like sorted lists of file names) that are easy in

the shell but lots of work to implement in C, or perhaps you're not

sufficiently familiar with C.

Another situation: perhaps you have to work with several C libraries,

and the usual C write/compile/test/re-compile cycle is too slow. You

need to develop software more quickly. Possibly perhaps you've

written a program that could use an extension language, and you don't

want to design a language, write and debug an interpreter for it, then

tie it into your application.

In such cases, Python may be just the language for you. Python is

simple to use, but it is a real programming language, offering much

more structure and support for large programs than the shell has. On

the other hand, it also offers much more error checking than C, and,

being a \emph{very-high-level language}, it has high-level data types

built in, such as flexible arrays and dictionaries that would cost you

days to implement efficiently in C. Because of its more general data

types Python is applicable to a much larger problem domain than

\emph{Awk} or even \emph{Perl}, yet many things are at least as easy

in Python as in those languages.

Python allows you to split up your program in modules that can be

reused in other Python programs. It comes with a large collection of

standard modules that you can use as the basis of your programs --- or

as examples to start learning to program in Python. There are also

built-in modules that provide things like file I/O, system calls,

sockets, and even interfaces to graphical user interface toolkits like Tk.

Python is an interpreted language, which can save you considerable time

during program development because no compilation and linking is

necessary. The interpreter can be used interactively, which makes it

easy to experiment with features of the language, to write throw-away

programs, or to test functions during bottom-up program development.

It is also a handy desk calculator.

Python allows writing very compact and readable programs. Programs

written in Python are typically much shorter than equivalent C or

\Cpp{} programs, for several reasons:

\begin{itemize}

\item

the high-level data types allow you to express complex operations in a

single statement;

\item

statement grouping is done by indentation instead of beginning and ending

brackets;

\item

no variable or argument declarations are necessary.

\end{itemize}

Python is \emph{extensible}: if you know how to program in C it is easy

to add a new built-in function or module to the interpreter, either to

perform critical operations at maximum speed, or to link Python

programs to libraries that may only be available in binary form (such

as a vendor-specific graphics library). Once you are really hooked,

you can link the Python interpreter into an application written in C

and use it as an extension or command language for that application.

By the way, the language is named after the BBC show ``Monty Python's

Flying Circus'' and has nothing to do with nasty reptiles. Making

references to Monty Python skits in documentation is not only allowed,

it is encouraged!

%\section{Where From Here \label{where}}

Now that you are all excited about Python, you'll want to examine it

in some more detail. Since the best way to learn a language is

using it, you are invited here to do so.

In the next chapter, the mechanics of using the interpreter are

explained. This is rather mundane information, but essential for

trying out the examples shown later.

The rest of the tutorial introduces various features of the Python

language and system through examples, beginning with simple

expressions, statements and data types, through functions and modules,

and finally touching upon advanced concepts like exceptions

and user-defined classes.

\chapter{Using the Python Interpreter \label{using}}

\section{Invoking the Interpreter \label{invoking}}

The Python interpreter is usually installed as

\file{/usr/local/bin/python} on those machines where it is available;

putting \file{/usr/local/bin} in your \UNIX{} shell's search path

makes it possible to start it by typing the command

\begin{verbatim}

python

\end{verbatim}

to the shell. Since the choice of the directory where the interpreter

lives is an installation option, other places are possible; check with

your local Python guru or system administrator. (E.g.,

\file{/usr/local/python} is a popular alternative location.)

Typing an end-of-file character (\kbd{Control-D} on \UNIX,

\kbd{Control-Z} on Windows) at the primary prompt causes the

interpreter to exit with a zero exit status. If that doesn't work,

you can exit the interpreter by typing the following commands:

\samp{import sys; sys.exit()}.

The interpreter's line-editing features usually aren't very

sophisticated. On \UNIX, whoever installed the interpreter may have

enabled support for the GNU readline library, which adds more

elaborate interactive editing and history features. Perhaps the

quickest check to see whether command line editing is supported is

typing Control-P to the first Python prompt you get. If it beeps, you

have command line editing; see Appendix \ref{interacting} for an

introduction to the keys. If nothing appears to happen, or if

\code{\^P} is echoed, command line editing isn't available; you'll

only be able to use backspace to remove characters from the current

line.

The interpreter operates somewhat like the \UNIX{} shell: when called

with standard input connected to a tty device, it reads and executes

commands interactively; when called with a file name argument or with

a file as standard input, it reads and executes a \emph{script} from

that file.

A second way of starting the interpreter is

\samp{\program{python} \programopt{-c} \var{command} [arg] ...}, which

executes the statement(s) in \var{command}, analogous to the shell's

\programopt{-c} option. Since Python statements often contain spaces

or other characters that are special to the shell, it is best to quote

\var{command} in its entirety with double quotes.

Note that there is a difference between \samp{python file} and

\samp{python <file}. In the latter case, input requests from the

program, such as calls to \function{input()} and \function{raw_input()}, are

satisfied from \emph{file}. Since this file has already been read

until the end by the parser before the program starts executing, the

program will encounter end-of-file immediately. In the former case

(which is usually what you want) they are satisfied from whatever file

or device is connected to standard input of the Python interpreter.

When a script file is used, it is sometimes useful to be able to run

the script and enter interactive mode afterwards. This can be done by

passing \programopt{-i} before the script. (This does not work if the

script is read from standard input, for the same reason as explained

in the previous paragraph.)

\subsection{Argument Passing \label{argPassing}}

When known to the interpreter, the script name and additional

arguments thereafter are passed to the script in the variable

\code{sys.argv}, which is a list of strings. Its length is at least

one; when no script and no arguments are given, \code{sys.argv[0]} is

an empty string. When the script name is given as \code{'-'} (meaning

standard input), \code{sys.argv[0]} is set to \code{'-'}. When

\programopt{-c} \var{command} is used, \code{sys.argv[0]} is set to

\code{'-c'}. Options found after \programopt{-c} \var{command} are

not consumed by the Python interpreter's option processing but left in

\code{sys.argv} for the command to handle.

\subsection{Interactive Mode \label{interactive}}

When commands are read from a tty, the interpreter is said to be in

\emph{interactive mode}. In this mode it prompts for the next command

with the \emph{primary prompt}, usually three greater-than signs

(\samp{>\code{>}>~}); for continuation lines it prompts with the

\emph{secondary prompt}, by default three dots (\samp{...~}).

The interpreter prints a welcome message stating its version number

and a copyright notice before printing the first prompt:

\begin{verbatim}

python

Python 1.5.2b2 (#1, Feb 28 1999, 00:02:06) [GCC 2.8.1] on sunos5

Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam

>>>

\end{verbatim}

Continuation lines are needed when entering a multi-line construct.

As an example, take a look at this \keyword{if} statement:

\begin{verbatim}

>>> the_world_is_flat = 1

>>> if the_world_is_flat:

... print "Be careful not to fall off!"

...

Be careful not to fall off!

\end{verbatim}

\section{The Interpreter and Its Environment \label{interp}}

\subsection{Error Handling \label{error}}

When an error occurs, the interpreter prints an error

message and a stack trace. In interactive mode, it then returns to

the primary prompt; when input came from a file, it exits with a

nonzero exit status after printing

the stack trace. (Exceptions handled by an \keyword{except} clause in a

\keyword{try} statement are not errors in this context.) Some errors are

unconditionally fatal and cause an exit with a nonzero exit; this

applies to internal inconsistencies and some cases of running out of

memory. All error messages are written to the standard error stream;

normal output from the executed commands is written to standard

output.

Typing the interrupt character (usually Control-C or DEL) to the

primary or secondary prompt cancels the input and returns to the

primary prompt.\footnote{

A problem with the GNU Readline package may prevent this.

}

Typing an interrupt while a command is executing raises the

\exception{KeyboardInterrupt} exception, which may be handled by a

\keyword{try} statement.

\subsection{Executable Python Scripts \label{scripts}}

On BSD'ish \UNIX{} systems, Python scripts can be made directly

executable, like shell scripts, by putting the line

\begin{verbatim}

#! /usr/bin/env python

\end{verbatim}

(assuming that the interpreter is on the user's \envvar{PATH}) at the

beginning of the script and giving the file an executable mode. The

\samp{\#!} must be the first two characters of the file. On some

platforms, this first line must end with a \UNIX-style line ending

(\character{\e n}), not a Mac OS (\character{\e r}) or Windows

(\character{\e r\e n}) line ending. Note that

the hash, or pound, character, \character{\#}, is used to start a

comment in Python.

The script can be given a executable mode, or permission, using the

\program{chmod} command:

\begin{verbatim}

$ chmod +x myscript.py

\end{verbatim} % $ <-- bow to font-lock

\subsection{Source Code Encoding}

It is possible to use encodings different than \ASCII{} in Python source

files. The best way to do it is to put one more special comment line

right after the \code{\#!} line to define the source file encoding:

\begin{verbatim}

# -*- coding: iso-8859-1 -*-

\end{verbatim}

With that declaration, all characters in the source file will be treated as

{}\code{iso-8859-1}, and it will be

possible to directly write Unicode string literals in the selected

encoding. The list of possible encodings can be found in the

\citetitle[../lib/lib.html]{Python Library Reference}, in the section

on \ulink{\module{codecs}}{../lib/module-codecs.html}.

If your editor supports saving files as \code{UTF-8} with a UTF-8

\emph{byte order mark} (aka BOM), you can use that instead of an

encoding declaration. IDLE supports this capability if

\code{Options/General/Default Source Encoding/UTF-8} is set. Notice

that this signature is not understood in older Python releases (2.2

and earlier), and also not understood by the operating system for

\code{\#!} files.

By using UTF-8 (either through the signature or an encoding

declaration), characters of most languages in the world can be used

simultaneously in string literals and comments. Using non-\ASCII

characters in identifiers is not supported. To display all these

characters properly, your editor must recognize that the file is

UTF-8, and it must use a font that supports all the characters in the

file.

\subsection{The Interactive Startup File \label{startup}}

% XXX This should probably be dumped in an appendix, since most people

% don't use Python interactively in non-trivial ways.

When you use Python interactively, it is frequently handy to have some

standard commands executed every time the interpreter is started. You

can do this by setting an environment variable named

\envvar{PYTHONSTARTUP} to the name of a file containing your start-up

commands. This is similar to the \file{.profile} feature of the

\UNIX{} shells.

This file is only read in interactive sessions, not when Python reads

commands from a script, and not when \file{/dev/tty} is given as the

explicit source of commands (which otherwise behaves like an

interactive session). It is executed in the same namespace where

interactive commands are executed, so that objects that it defines or

imports can be used without qualification in the interactive session.

You can also change the prompts \code{sys.ps1} and \code{sys.ps2} in

this file.

If you want to read an additional start-up file from the current

directory, you can program this in the global start-up file using code

like \samp{if os.path.isfile('.pythonrc.py'):

execfile('.pythonrc.py')}. If you want to use the startup file in a

script, you must do this explicitly in the script:

\begin{verbatim}

import os

filename = os.environ.get('PYTHONSTARTUP')

if filename and os.path.isfile(filename):

execfile(filename)

\end{verbatim}

\chapter{An Informal Introduction to Python \label{informal}}

In the following examples, input and output are distinguished by the

presence or absence of prompts (\samp{>\code{>}>~} and \samp{...~}): to repeat

the example, you must type everything after the prompt, when the

prompt appears; lines that do not begin with a prompt are output from

the interpreter. %

%\footnote{

% I'd prefer to use different fonts to distinguish input

% from output, but the amount of LaTeX hacking that would require

% is currently beyond my ability.

%}

Note that a secondary prompt on a line by itself in an example means

you must type a blank line; this is used to end a multi-line command.

Many of the examples in this manual, even those entered at the

interactive prompt, include comments. Comments in Python start with

the hash character, \character{\#}, and extend to the end of the

physical line. A comment may appear at the start of a line or

following whitespace or code, but not within a string literal. A hash

character within a string literal is just a hash character.

Some examples:

\begin{verbatim}

# this is the first comment

SPAM = 1 # and this is the second comment

# ... and now a third!

STRING = "# This is not a comment."

\end{verbatim}

\section{Using Python as a Calculator \label{calculator}}

Let's try some simple Python commands. Start the interpreter and wait

for the primary prompt, \samp{>\code{>}>~}. (It shouldn't take long.)

\subsection{Numbers \label{numbers}}

The interpreter acts as a simple calculator: you can type an

expression at it and it will write the value. Expression syntax is

straightforward: the operators \code{+}, \code{-}, \code{*} and

\code{/} work just like in most other languages (for example, Pascal

or C); parentheses can be used for grouping. For example:

\begin{verbatim}

>>> 2+2

4

>>> # This is a comment

... 2+2

4

>>> 2+2 # and a comment on the same line as code

4

>>> (50-5*6)/4

5

>>> # Integer division returns the floor:

... 7/3

2

>>> 7/-3

-3

\end{verbatim}

Like in C, the equal sign (\character{=}) is used to assign a value to a

variable. The value of an assignment is not written:

\begin{verbatim}

>>> width = 20

>>> height = 5*9

>>> width * height

900

\end{verbatim}

A value can be assigned to several variables simultaneously:

\begin{verbatim}

>>> x = y = z = 0 # Zero x, y and z

>>> x

0

>>> y

0

>>> z

0

\end{verbatim}

There is full support for floating point; operators with mixed type

operands convert the integer operand to floating point:

\begin{verbatim}

>>> 3 * 3.75 / 1.5

7.5

>>> 7.0 / 2

3.5

\end{verbatim}

Complex numbers are also supported; imaginary numbers are written with

a suffix of \samp{j} or \samp{J}. Complex numbers with a nonzero

real component are written as \samp{(\var{real}+\var{imag}j)}, or can

be created with the \samp{complex(\var{real}, \var{imag})} function.

\begin{verbatim}

>>> 1j * 1J

(-1+0j)

>>> 1j * complex(0,1)

(-1+0j)

>>> 3+1j*3

(3+3j)

>>> (3+1j)*3

(9+3j)

>>> (1+2j)/(1+1j)

(1.5+0.5j)

\end{verbatim}

Complex numbers are always represented as two floating point numbers,

the real and imaginary part. To extract these parts from a complex

number \var{z}, use \code{\var{z}.real} and \code{\var{z}.imag}.

\begin{verbatim}

>>> a=1.5+0.5j

>>> a.real

1.5

>>> a.imag

0.5

\end{verbatim}

The conversion functions to floating point and integer

(\function{float()}, \function{int()} and \function{long()}) don't

work for complex numbers --- there is no one correct way to convert a

complex number to a real number. Use \code{abs(\var{z})} to get its

magnitude (as a float) or \code{z.real} to get its real part.

\begin{verbatim}

>>> a=3.0+4.0j

>>> float(a)

Traceback (most recent call last):

File "<stdin>", line 1, in ?

TypeError: can't convert complex to float; use abs(z)

>>> a.real

3.0

>>> a.imag

4.0

>>> abs(a) # sqrt(a.real**2 + a.imag**2)

5.0

>>>

\end{verbatim}

In interactive mode, the last printed expression is assigned to the

variable \code{_}. This means that when you are using Python as a

desk calculator, it is somewhat easier to continue calculations, for

example:

\begin{verbatim}

>>> tax = 12.5 / 100

>>> price = 100.50

>>> price * tax

12.5625

>>> price + _

113.0625

>>> round(_, 2)

113.06

>>>

\end{verbatim}

This variable should be treated as read-only by the user. Don't

explicitly assign a value to it --- you would create an independent

local variable with the same name masking the built-in variable with

its magic behavior.

\subsection{Strings \label{strings}}

Besides numbers, Python can also manipulate strings, which can be

expressed in several ways. They can be enclosed in single quotes or

double quotes:

\begin{verbatim}

>>> 'spam eggs'

'spam eggs'

>>> 'doesn\'t'

"doesn't"

>>> "doesn't"

"doesn't"

>>> '"Yes," he said.'

'"Yes," he said.'

>>> "\"Yes,\" he said."

'"Yes," he said.'

>>> '"Isn\'t," she said.'

'"Isn\'t," she said.'

\end{verbatim}

String literals can span multiple lines in several ways. Continuation

lines can be used, with a backslash as the last character on the line

indicating that the next line is a logical continuation of the line:

\begin{verbatim}

hello = "This is a rather long string containing\n\

several lines of text just as you would do in C.\n\

Note that whitespace at the beginning of the line is\

significant."

print hello

\end{verbatim}

Note that newlines would still need to be embedded in the string using

\code{\e n}; the newline following the trailing backslash is

discarded. This example would print the following:

\begin{verbatim}

This is a rather long string containing

several lines of text just as you would do in C.

Note that whitespace at the beginning of the line is significant.

\end{verbatim}

If we make the string literal a ``raw'' string, however, the

\code{\e n} sequences are not converted to newlines, but the backslash

at the end of the line, and the newline character in the source, are

both included in the string as data. Thus, the example:

\begin{verbatim}

hello = r"This is a rather long string containing\n\

several lines of text much as you would do in C."

print hello

\end{verbatim}

would print:

\begin{verbatim}

This is a rather long string containing\n\

several lines of text much as you would do in C.

\end{verbatim}

Or, strings can be surrounded in a pair of matching triple-quotes:

\code{"""} or \code{'\code{'}'}. End of lines do not need to be escaped

when using triple-quotes, but they will be included in the string.

\begin{verbatim}

print """

Usage: thingy [OPTIONS]

-h Display this usage message

-H hostname Hostname to connect to

"""

\end{verbatim}

produces the following output:

\begin{verbatim}

Usage: thingy [OPTIONS]

-h Display this usage message

-H hostname Hostname to connect to

\end{verbatim}

The interpreter prints the result of string operations in the same way

as they are typed for input: inside quotes, and with quotes and other

funny characters escaped by backslashes, to show the precise

value. The string is enclosed in double quotes if the string contains

a single quote and no double quotes, else it's enclosed in single

quotes. (The \keyword{print} statement, described later, can be used

to write strings without quotes or escapes.)

Strings can be concatenated (glued together) with the

\code{+} operator, and repeated with \code{*}:

\begin{verbatim}

>>> word = 'Help' + 'A'

>>> word

'HelpA'

>>> '<' + word*5 + '>'

'<HelpAHelpAHelpAHelpAHelpA>'

\end{verbatim}

Two string literals next to each other are automatically concatenated;

the first line above could also have been written \samp{word = 'Help'

'A'}; this only works with two literals, not with arbitrary string

expressions:

\begin{verbatim}

>>> 'str' 'ing' # <- This is ok

'string'

>>> 'str'.strip() + 'ing' # <- This is ok

'string'

>>> 'str'.strip() 'ing' # <- This is invalid

File "<stdin>", line 1, in ?

'str'.strip() 'ing'

^

SyntaxError: invalid syntax

\end{verbatim}

Strings can be subscripted (indexed); like in C, the first character

of a string has subscript (index) 0. There is no separate character

type; a character is simply a string of size one. Like in Icon,

substrings can be specified with the \emph{slice notation}: two indices

separated by a colon.

\begin{verbatim}

>>> word[4]

'A'

>>> word[0:2]

'He'

>>> word[2:4]

'lp'

\end{verbatim}

Slice indices have useful defaults; an omitted first index defaults to

zero, an omitted second index defaults to the size of the string being

sliced.

\begin{verbatim}

>>> word[:2] # The first two characters

'He'

>>> word[2:] # Everything except the first two characters

'lpA'

\end{verbatim}

Unlike a C string, Python strings cannot be changed. Assigning to an

indexed position in the string results in an error:

\begin{verbatim}

>>> word[0] = 'x'

Traceback (most recent call last):

File "<stdin>", line 1, in ?

TypeError: object doesn't support item assignment

>>> word[:1] = 'Splat'

Traceback (most recent call last):

File "<stdin>", line 1, in ?

TypeError: object doesn't support slice assignment

\end{verbatim}

However, creating a new string with the combined content is easy and

efficient:

\begin{verbatim}

>>> 'x' + word[1:]

'xelpA'

>>> 'Splat' + word[4]

'SplatA'

\end{verbatim}

Here's a useful invariant of slice operations:

\code{s[:i] + s[i:]} equals \code{s}.

\begin{verbatim}

>>> word[:2] + word[2:]

'HelpA'

>>> word[:3] + word[3:]

'HelpA'

\end{verbatim}

Degenerate slice indices are handled gracefully: an index that is too

large is replaced by the string size, an upper bound smaller than the

lower bound returns an empty string.

\begin{verbatim}

>>> word[1:100]

'elpA'

>>> word[10:]

''

>>> word[2:1]

''

\end{verbatim}

Indices may be negative numbers, to start counting from the right.

For example:

\begin{verbatim}

>>> word[-1] # The last character

'A'

>>> word[-2] # The last-but-one character

'p'

>>> word[-2:] # The last two characters

'pA'

>>> word[:-2] # Everything except the last two characters

'Hel'

\end{verbatim}

But note that -0 is really the same as 0, so it does not count from

the right!

\begin{verbatim}

>>> word[-0] # (since -0 equals 0)

'H'

\end{verbatim}

Out-of-range negative slice indices are truncated, but don't try this

for single-element (non-slice) indices:

\begin{verbatim}

>>> word[-100:]

'HelpA'

>>> word[-10] # error

Traceback (most recent call last):

File "<stdin>", line 1, in ?

IndexError: string index out of range

\end{verbatim}

The best way to remember how slices work is to think of the indices as

pointing \emph{between} characters, with the left edge of the first

character numbered 0. Then the right edge of the last character of a

string of \var{n} characters has index \var{n}, for example:

\begin{verbatim}

+---+---+---+---+---+

| H | e | l | p | A |

+---+---+---+---+---+

0 1 2 3 4 5

-5 -4 -3 -2 -1

\end{verbatim}

The first row of numbers gives the position of the indices 0...5 in

the string; the second row gives the corresponding negative indices.

The slice from \var{i} to \var{j} consists of all characters between

the edges labeled \var{i} and \var{j}, respectively.

For non-negative indices, the length of a slice is the difference of

the indices, if both are within bounds. For example, the length of

\code{word[1:3]} is 2.

The built-in function \function{len()} returns the length of a string:

\begin{verbatim}

>>> s = 'supercalifragilisticexpialidocious'

>>> len(s)

34

\end{verbatim}

\begin{seealso}

\seetitle[../lib/typesseq.html]{Sequence Types}%

{Strings, and the Unicode strings described in the next

section, are examples of \emph{sequence types}, and

support the common operations supported by such types.}

\seetitle[../lib/string-methods.html]{String Methods}%

{Both strings and Unicode strings support a large number of

methods for basic transformations and searching.}

\seetitle[../lib/typesseq-strings.html]{String Formatting Operations}%

{The formatting operations invoked when strings and Unicode

strings are the left operand of the \code{\%} operator are

described in more detail here.}

\end{seealso}

\subsection{Unicode Strings \label{unicodeStrings}}

\sectionauthor{Marc-Andre Lemburg}{[email protected]}

Starting with Python 2.0 a new data type for storing text data is

available to the programmer: the Unicode object. It can be used to

store and manipulate Unicode data (see \url{http://www.unicode.org/})

and integrates well with the existing string objects providing

auto-conversions where necessary.

Unicode has the advantage of providing one ordinal for every character

in every script used in modern and ancient texts. Previously, there

were only 256 possible ordinals for script characters and texts were

typically bound to a code page which mapped the ordinals to script

characters. This lead to very much confusion especially with respect

to internationalization (usually written as \samp{i18n} ---

\character{i} + 18 characters + \character{n}) of software. Unicode

solves these problems by defining one code page for all scripts.

Creating Unicode strings in Python is just as simple as creating

normal strings:

\begin{verbatim}

>>> u'Hello World !'

u'Hello World !'

\end{verbatim}

The small \character{u} in front of the quote indicates that an

Unicode string is supposed to be created. If you want to include

special characters in the string, you can do so by using the Python

\emph{Unicode-Escape} encoding. The following example shows how:

\begin{verbatim}

>>> u'Hello\u0020World !'

u'Hello World !'

\end{verbatim}

The escape sequence \code{\e u0020} indicates to insert the Unicode

character with the ordinal value 0x0020 (the space character) at the

given position.

Other characters are interpreted by using their respective ordinal

values directly as Unicode ordinals. If you have literal strings

in the standard Latin-1 encoding that is used in many Western countries,

you will find it convenient that the lower 256 characters

of Unicode are the same as the 256 characters of Latin-1.

For experts, there is also a raw mode just like the one for normal

strings. You have to prefix the opening quote with 'ur' to have

Python use the \emph{Raw-Unicode-Escape} encoding. It will only apply

the above \code{\e uXXXX} conversion if there is an uneven number of

backslashes in front of the small 'u'.

\begin{verbatim}

>>> ur'Hello\u0020World !'

u'Hello World !'

>>> ur'Hello\\u0020World !'

u'Hello\\\\u0020World !'

\end{verbatim}

The raw mode is most useful when you have to enter lots of

backslashes, as can be necessary in regular expressions.

Apart from these standard encodings, Python provides a whole set of

other ways of creating Unicode strings on the basis of a known

encoding.

The built-in function \function{unicode()}\bifuncindex{unicode} provides

access to all registered Unicode codecs (COders and DECoders). Some of

the more well known encodings which these codecs can convert are

\emph{Latin-1}, \emph{ASCII}, \emph{UTF-8}, and \emph{UTF-16}.

The latter two are variable-length encodings that store each Unicode

character in one or more bytes. The default encoding is

normally set to \ASCII, which passes through characters in the range

0 to 127 and rejects any other characters with an error.

When a Unicode string is printed, written to a file, or converted

with \function{str()}, conversion takes place using this default encoding.

\begin{verbatim}

>>> u"abc"

u'abc'

>>> str(u"abc")

'abc'

>>> u"äöü"

u'\xe4\xf6\xfc'

>>> str(u"äöü")

Traceback (most recent call last):

File "<stdin>", line 1, in ?

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)

\end{verbatim}

To convert a Unicode string into an 8-bit string using a specific

encoding, Unicode objects provide an \function{encode()} method

that takes one argument, the name of the encoding. Lowercase names

for encodings are preferred.

\begin{verbatim}

>>> u"äöü".encode('utf-8')

'\xc3\xa4\xc3\xb6\xc3\xbc'

\end{verbatim}

If you have data in a specific encoding and want to produce a

corresponding Unicode string from it, you can use the

\function{unicode()} function with the encoding name as the second

argument.

\begin{verbatim}

>>> unicode('\xc3\xa4\xc3\xb6\xc3\xbc', 'utf-8')

u'\xe4\xf6\xfc'

\end{verbatim}

\subsection{Lists \label{lists}}

Python knows a number of \emph{compound} data types, used to group

together other values. The most versatile is the \emph{list}, which

can be written as a list of comma-separated values (items) between

square brackets. List items need not all have the same type.

\begin{verbatim}

>>> a = ['spam', 'eggs', 100, 1234]

>>> a

['spam', 'eggs', 100, 1234]

\end{verbatim}

Like string indices, list indices start at 0, and lists can be sliced,

concatenated and so on:

\begin{verbatim}

>>> a[0]

'spam'

>>> a[3]

1234

>>> a[-2]

100

>>> a[1:-1]

['eggs', 100]

>>> a[:2] + ['bacon', 2*2]

['spam', 'eggs', 'bacon', 4]

>>> 3*a[:3] + ['Boe!']

['spam', 'eggs', 100, 'spam', 'eggs', 100, 'spam', 'eggs', 100, 'Boe!']

\end{verbatim}

Unlike strings, which are \emph{immutable}, it is possible to change

individual elements of a list:

\begin{verbatim}

>>> a

['spam', 'eggs', 100, 1234]

>>> a[2] = a[2] + 23

>>> a

['spam', 'eggs', 123, 1234]

\end{verbatim}

Assignment to slices is also possible, and this can even change the size

of the list:

\begin{verbatim}

>>> # Replace some items:

... a[0:2] = [1, 12]

>>> a

[1, 12, 123, 1234]

>>> # Remove some:

... a[0:2] = []

>>> a

[123, 1234]

>>> # Insert some:

... a[1:1] = ['bletch', 'xyzzy']

>>> a

[123, 'bletch', 'xyzzy', 1234]

>>> a[:0] = a # Insert (a copy of) itself at the beginning

>>> a

[123, 'bletch', 'xyzzy', 1234, 123, 'bletch', 'xyzzy', 1234]

\end{verbatim}

The built-in function \function{len()} also applies to lists:

\begin{verbatim}

>>> len(a)

8