Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 066f392

Browse files
brettcannonbrettcannon
committed
Library documentation for the 'test' package. Still needs to be checked by Fred.
1 parent aed0a4a commit 066f392

2 files changed

Lines changed: 276 additions & 0 deletions

File tree

‎Doc/lib/lib.tex‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,7 @@ \chapter*{Front Matter\label{front}}
118118
\input{libpydoc}
119119
\input{libdoctest}
120120
\input{libunittest}
121+
\input{libtest}
121122
\input{libmath}
122123
\input{libcmath}
123124
\input{librandom}

‎Doc/lib/libtest.tex‎

Lines changed: 275 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,275 @@
1+
\section{\module{test} ---
2+
Regression tests package for Python}
3+
4+
\declaremodule{standard}{test}
5+
6+
\sectionauthor{Brett Cannon}{[email protected]}
7+
8+
9+
\modulesynopsis{Regression tests package containing the testing suite for
10+
Python.}
11+
12+
13+
The \module{test} package contains all regression tests for Python as well as
14+
the modules \module{test_support} and \module{regrtest.py}.
15+
\module{test_support} is used to enhance your tests while \module{regrtest.py}
16+
drives the testing suite.
17+
18+
Each module in the \module{test} package whose name starts with
19+
\code{'test_'} is a testing suite for a specific module or feature.
20+
All new tests should be written using the \module{unittest} module; using
21+
\module{unittest} is not required but makes the tests more flexible and
22+
maintenance of the tests easier.
23+
Some older tests are written to use \module{doctest} and a ``traditional''
24+
testing style; these styles of tests will not be covered.
25+
26+
\begin{seealso}
27+
\seemodule{unittest}{Writing PyUnit regression tests.}
28+
\seemodule{doctest}{Tests embedded in documentation strings.}
29+
\end{seealso}
30+
31+
32+
\subsection{test_support \label{test_support-docs}}
33+
34+
The \module{test_support} module contains functions for assisting with writing
35+
regression tests.
36+
37+
38+
The \module{test_support} module defines the following exceptions:
39+
40+
\begin{excdesc}{TestFailed}
41+
Exception to be raised when a test fails.
42+
\end{excdesc}
43+
44+
\begin{excdesc}{TestSkipped}
45+
Subclass of \exception{TestFailed}.
46+
Raised when a test is skipped.
47+
This occurs when a needed resource (such as a network connection) is not
48+
available at the time of testing.
49+
\end{excdesc}
50+
51+
\begin{excdesc}{ResourceDenied}
52+
Subclass of \exception{TestSkipped}.
53+
Raised when a resource (such as a network connection) is not available.
54+
Raised by the \function{requires} function.
55+
\end{excdesc}
56+
57+
58+
The \module{test_support} module defines the following constants:
59+
60+
\begin{datadesc}{verbose}
61+
\constant{True} when verbose output is enabled.
62+
Should be checked when more detailed information is desired about a running
63+
test.
64+
\var{verbose} is set by \module{regrtest.py}.
65+
\end{datadesc}
66+
67+
\begin{datadesc}{have_unicode}
68+
\constant{True} when Unicode support is available.
69+
\end{datadesc}
70+
71+
\begin{datadesc}{is_jython}
72+
\constant{True} if the running interpreter is Jython.
73+
\end{datadesc}
74+
75+
\begin{datadesc}{TESTFN}
76+
Set to the path that a temporary file may be created at.
77+
Any temporary that is created should be closed and unlinked (removed).
78+
\end{datadesc}
79+
80+
81+
The \module{test_support} module defines the following functions:
82+
83+
\begin{funcdesc}{forget}{module_name}
84+
Removes the module named \var{module_name} from \module{sys.modules} and deletes
85+
any byte-compiled files of the module.
86+
\end{funcdesc}
87+
88+
\begin{funcdesc}{is_resource_enabled}{resource}
89+
Returns \constant{True} if \var{resource} is enabled and available.
90+
The list of available resources is only set when \module{regrtest.py} is
91+
executing the tests.
92+
\end{funcdest}
93+
94+
\begin{funcdesc}{requires}{resource\optional{, msg}}
95+
Raises \exception{ResourceDenied} if \var{resource} is not available.
96+
\var{msg} is the argument to \exception{ResourceDenied} if it is raised.
97+
Always returns true if called by a function whose \var{__name__} is
98+
\code{"__main__"}.
99+
Used when tests are executed by \module{regrtest.py}.
100+
\end{funcdesc}
101+
102+
\begin{funcdesc}{findfile}{filename}
103+
Return the path to the file named \var{filename}.
104+
If no match is found \var{filename} is returned.
105+
This does not equal a failure since it could be the path to the file.
106+
\end{funcdesc}
107+
108+
\begin{funcdesc}{run_unittest}{*classes}
109+
Execute \class{unittest.TestCase} subclasses passed to the function.
110+
The function scans the classes for methods starting with the name
111+
\code{"test_"} and executes the tests individually.
112+
This is the preferred way to execute tests.
113+
\end{datadesc}
114+
115+
\begin{funcdesc}{run_suite}{suite\optional{, testclass=None}}
116+
Execute the \class{unittest.TestSuite} instance, \var{suite}.
117+
The optional argument \var{testclass} accepts one of the test classes in the
118+
suite so as to print out more detailed information on where the testing suite
119+
originated from.
120+
\end{funcdesc}
121+
122+
123+
124+
\subsection{Writing Unit Tests for the \module{test} package \label{writing-tests}}
125+
126+
It is preferred that tests for the \module{test} package use the
127+
\module{unittest} module and follow a few guidelines.
128+
One is to have the name of all the test methods start with \code{"test_"} as
129+
well as the module's name.
130+
This is needed so that the methods are recognized by the test driver as
131+
test methods.
132+
Also, no documentation string for the method should be included.
133+
A comment (such as
134+
\var{# Tests function returns only True or False}) should be used to provide
135+
documentation for test methods.
136+
This is done because documentation strings get printed out if they exist and
137+
thus what test is being run is not stated.
138+
139+
A basic boilerplate is often used:
140+
141+
\begin{verbatim}
142+
import unittest
143+
from test import test_support
144+
145+
class MyTestCase1(unittest.TestCase):
146+
147+
# Only use setUp() and tearDown() if necessary
148+
149+
def setUp(self):
150+
... code to execute in preparation for tests ...
151+
152+
def tearDown(self):
153+
... code to execute to clean up after tests ...
154+
155+
def test_feature_one(self):
156+
# Test feature one.
157+
... testing code ...
158+
159+
def test_feature_two(self):
160+
# Test feature two.
161+
... testing code ...
162+
163+
... more test methods ...
164+
165+
class MyTestCase2(unittest.TestCase):
166+
... same structure as MyTestCase1 ...
167+
168+
... more test classes ...
169+
170+
def test_main():
171+
test_support.run_unittest(MyTestCase1,
172+
MyTestCase2,
173+
... list other tests ...
174+
)
175+
176+
if __name__ == '__main__':
177+
test_main()
178+
\end{verbatim}
179+
180+
This boilerplate code allows the testing suite to be run by \module{regrtest.py}
181+
as well as on its own as a script.
182+
183+
The goal for regression testing is to try to break code.
184+
This leads to a few guidelines to be followed:
185+
186+
\begin{itemize}
187+
\item The testing suite should exercise all classes, functions, and
188+
constants.
189+
This includes not just the external API that is to be presented to the
190+
outside world but also "private" code.
191+
\item Whitebox testing (examining the code being tested when the tests are
192+
being written) is preferred.
193+
Blackbox testing (testing only the published user interface) is not
194+
complete enough to make sure all boundary and edge cases are tested.
195+
\item Make sure all possible values are tested including invalid ones.
196+
This makes sure that not only all valid values are acceptable but also
197+
that improper values are handled correctly.
198+
\item Exhaust as many code paths as possible.
199+
Test where branching occurs and thus tailor input to make sure as many
200+
different paths through the code are taken.
201+
\item Add an explicit test for any bugs discovered for the tested code.
202+
This will make sure that the error does not crop up again if the code is
203+
changed in the future.
204+
\item Make sure to clean up after your tests (such as close and remove all
205+
temporary files).
206+
\item Import as few modules as possible and do it as soon as possible.
207+
This minimizes external dependencies of tests and also minimizes possible
208+
anomalous behavior from side-effects of importing a module.
209+
\item Try to maximize code reuse.
210+
On occasion tests will vary by something as small as what type of input
211+
they take.
212+
Minimize code duplication by subclassing a basic test class with a class
213+
that specifies the input:
214+
\begin{verbatim}
215+
class TestFuncAcceptsSequences(unittest.TestCase):
216+
217+
func = mySuperWhammyFunction
218+
219+
def test_func(self):
220+
self.func(self.arg)
221+
222+
class AcceptLists(TestFuncAcceptsSequences):
223+
arg = [1,2,3]
224+
225+
class AcceptStrings(TestFuncAcceptsSequences):
226+
arg = 'abc'
227+
228+
class AcceptTuples(TestFuncAcceptsSequences):
229+
arg = (1,2,3)
230+
\end{verbatim}
231+
\end{itemize}
232+
233+
\begin{seealso}
234+
\seetitle{Test Driven Development}{A book by Kent Beck on writing tests before
235+
code}
236+
\end{seealso}
237+
238+
239+
240+
\subsection{Running tests Using \module{regrtest.py} \label{regrtest}}
241+
242+
\module{regrtest.py} is the script used to drive Python's regression test
243+
suite.
244+
Running the script by itself automatically starts running all
245+
regression tests in the \module{test} package.
246+
It does this by finding all modules in the package whose name starts with
247+
\code{test_}, importing them, and executing the function \function{test_main}
248+
if present.
249+
The names of tests to execute may also be passed to the script.
250+
Specifying a single regression test (\code{python regrtest.py test_spam.py})
251+
will minimize output and only print whether the test passed or failed and thus
252+
minimize output.
253+
254+
Running \module{regrtest.py} directly allows what resources are
255+
available for tests to use to be set.
256+
You do this by using the \code{-u} command-line option.
257+
Run \code{python regrtest.py -uall} to turn on all resources;
258+
specifying \code{all} as an option for \code{-u} enables all possible
259+
resources.
260+
If all but one resource is desired (a more common case), a
261+
comma-separated list of resources that are not desired may be listed after
262+
\code{all}.
263+
The command \code{python regrtest.py -uall,-audio,-largefile} will run
264+
\module{regrtest.py} with all resources except the audio and largefile
265+
resources.
266+
For a list of all resources and more command-line options, run
267+
\code{python regrtest.py -h}.
268+
269+
Some other ways to execute the regression tests depend on what platform the
270+
tests are being executed on.
271+
On \UNIX{}, you can run \code{make test} at the top-level directory
272+
where Python was built.
273+
On Windows, executing \code{rt.bat} from your PCBuild directory will run all
274+
regression tests.
275+

0 commit comments

Comments
 (0)