Doctest has new traceback gimmicks in 2.4. While trying to document

tim-one · tim-one · commit 41a65ea7fec6 · 2004-08-13T03:55:05.000Z
them (which they are now), I had to rewrite the code to understand
it.  This has got to be the most DWIM part of doctest -- but in context
is really necessary.
diff --git a/Doc/lib/libdoctest.tex b/Doc/lib/libdoctest.tex
@@ -108,7 +108,8 @@ \section{\module{doctest} ---
 ok
 Trying: [factorial(long(n)) for n in range(6)]
 Expecting: [1, 1, 2, 6, 24, 120]
-ok\end{verbatim}
+ok
+\end{verbatim}
 
 And so on, eventually ending with:
 
@@ -129,12 +130,14 @@ \section{\module{doctest} ---
 \end{verbatim}
 
 That's all you need to know to start making productive use of
-\module{doctest}!  Jump in.
+\module{doctest}!  Jump in.  The following sections provide full
+details.  Note that there are many examples of doctests in
+the standard Python test suite and libraries.
 
 \subsection{Simple Usage}
 
-The simplest (not necessarily the best) way to start using doctest is to
-end each module \module{M} with:
+The simplest way to start using doctest (but not necessarily the way
+you'll continue to do it) is to end each module \module{M} with:
 
 \begin{verbatim}
 def _test():
@@ -146,8 +149,7 @@ \subsection{Simple Usage}
 \end{verbatim}
 
 \module{doctest} then examines docstrings in the module calling
-\function{testmod()}.  If you want to test a different module, you can
-pass that module object to \function{testmod()}.
+\function{testmod()}.
 
 Running the module as a script causes the examples in the docstrings
 to get executed and verified:
@@ -292,35 +294,95 @@ \subsection{Which Docstrings Are Examined?}
 
 \subsection{What's the Execution Context?}
 
-By default, each time \function{testmod()} finds a docstring to test, it uses
-a \emph{copy} of \module{M}'s globals, so that running tests on a module
+By default, each time \function{testmod()} finds a docstring to test, it
+uses a \emph{shallow copy} of \module{M}'s globals, so that running tests
 doesn't change the module's real globals, and so that one test in
 \module{M} can't leave behind crumbs that accidentally allow another test
 to work.  This means examples can freely use any names defined at top-level
 in \module{M}, and names defined earlier in the docstring being run.
+Examples cannot see names defined in other docstrings.
 
 You can force use of your own dict as the execution context by passing
-\code{globs=your_dict} to \function{testmod()} instead.  Presumably this
-would be a copy of \code{M.__dict__} merged with the globals from other
-imported modules.
+\code{globs=your_dict} to \function{testmod()} instead.
 
 \subsection{What About Exceptions?}
 
-No problem, as long as the only output generated by the example is the
-traceback itself.  For example:
+No problem:  just paste in the expected traceback.  Since
+tracebacks contain details that are likely to change
+rapidly (for example, exact file paths and line numbers), this is one
+case where doctest works hard to be flexible in what it accepts.
+This makes the full story involved, but you really don't have
+to remember much.  Simple example:
 
 \begin{verbatim}
 >>> [1, 2, 3].remove(42)
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 ValueError: list.remove(x): x not in list
->>>
 \end{verbatim}
 
-Note that only the exception type and value are compared (specifically,
-only the last line in the traceback).  The various ``File'' lines in
-between can be left out (unless they add significantly to the documentation
-value of the example).
+That doctest succeeds if, and only if, \exception{ValueError} is raised,
+with the \samp{list.remove(x): x not in list} detail as shown.
+
+The expected output for an exception is divided into four parts.
+First, an example may produce some normal output before an exception
+is raised, although that's unusual.  The "normal output" is taken to
+be everything until the first "Traceback" line, and is usually an
+empty string.  Next, the traceback line must be one of these two, and
+indented the same as the first line in the example:
+
+\begin{verbatim}
+Traceback (most recent call last):
+Traceback (innermost last):
+\end{verbatim}
+
+The most interesting part is the last part:  the line(s) starting with the
+exception type and detail.  This is usually the last line of a traceback,
+but can extend across any number of lines.  After the "Traceback" line,
+doctest simply ignores everything until the first line indented the same as
+the first line of the example, \emph{and} starting with an alphanumeric
+character.  This example illustrates the complexities that are possible:
+
+\begin{verbatim}
+>>> print 1, 2; raise ValueError('printed 1\nand 2\n  but not 3')
+1 2
+Traceback (most recent call last):
+... indented the same, but doesn't start with an alphanumeric
+  not indented the same, so ignored too
+  File "/Python23/lib/doctest.py", line 442, in _run_examples_inner
+    compileflags, 1) in globs
+  File "<string>", line 1, in ?   # and all these are ignored
+ValueError: printed 1
+and 2
+  but not 3
+\end{verbatim}
+
+The first (\samp{1 2}) and last three (starting with
+\exception{ValueError}) lines are compared, and the rest are ignored.
+
+Best practice is to omit the ``File'' lines, unless they add
+significant documentation value to the example.  So the example above
+is probably better as:
+
+\begin{verbatim}
+>>> print 1, 2; raise ValueError('printed 1\nand 2\n  but not 3')
+1 2
+Traceback (most recent call last):
+   ...
+ValueError: printed 1
+and 2
+  but not 3
+\end{verbatim}
+
+Note the tracebacks are treated very specially.  In particular, in the
+rewritten example, the use of \samp{...} is independent of doctest's
+\constant{ELLIPSIS} option.  The ellipsis in that example could
+be left out, or could just as well be three (or three hundred) commas.
+
+\versionchanged[The abilities to check both normal output and an
+                exception in a single example, and to have a multi-line
+                exception detail, were added]{2.4}
+
 
 \subsection{Option Flags and Directive Names\label{doctest-options}}
 
diff --git a/Lib/doctest.py b/Lib/doctest.py
@@ -1191,16 +1191,27 @@ def _failure_header(self, test, example):
     #/////////////////////////////////////////////////////////////////
 
     # A regular expression for handling `want` strings that contain
-    # expected exceptions.  It divides `want` into two pieces: the
-    # pre-exception output (`out`) and the exception message (`exc`),
-    # as generated by traceback.format_exception_only().  (I assume
-    # that the exception_only message is the first non-indented line
-    # starting with word characters after the "Traceback ...".)
-    _EXCEPTION_RE = re.compile(('^(?P<out>.*)'
-                                '^(?P<hdr>Traceback \((?:%s|%s)\):)\s*$.*?'
-                                '^(?P<exc>\w+.*)') %
-                               ('most recent call last', 'innermost last'),
-                               re.MULTILINE | re.DOTALL)
+    # expected exceptions.  It divides `want` into three pieces:
+    #    - the pre-exception output (`want`)
+    #    - the traceback header line (`hdr`)
+    #    - the exception message (`msg`), as generated by
+    #      traceback.format_exception_only()
+    # `msg` may have multiple lines.  We assume/require that the
+    # exception message is the first non-indented line starting with a word
+    # character following the traceback header line.
+    _EXCEPTION_RE = re.compile(r"""
+        (?P<want> .*?)   # suck up everything until traceback header
+        # Grab the traceback header.  Different versions of Python have
+        # said different things on the first traceback line.
+        ^(?P<hdr> Traceback\ \(
+            (?: most\ recent\ call\ last
+            |   innermost\ last
+            ) \) :
+        )
+        \s* $  # toss trailing whitespace on traceback header
+        .*?    # don't blink:  absorb stuff until a line *starts* with \w
+        ^ (?P<msg> \w+ .*)
+        """, re.VERBOSE | re.MULTILINE | re.DOTALL)
 
     def __run(self, test, compileflags, out):
         """
@@ -1274,20 +1285,19 @@ def __run(self, test, compileflags, out):
                                                      exc_info)
                     failures += 1
                 else:
-                    exc_hdr = m.group('hdr')+'\n' # Exception header
+                    e_want, e_msg = m.group('want', 'msg')
                     # The test passes iff the pre-exception output and
                     # the exception description match the values given
                     # in `want`.
-                    if (self._checker.check_output(m.group('out'), got,
+                    if (self._checker.check_output(e_want, got,
                                                    self.optionflags) and
-                        self._checker.check_output(m.group('exc'), exc_msg,
+                        self._checker.check_output(e_msg, exc_msg,
                                                    self.optionflags)):
-                        # Is +exc_msg the right thing here??
                         self.report_success(out, test, example,
-                                       got+_exception_traceback(exc_info))
+                                       got + _exception_traceback(exc_info))
                     else:
                         self.report_failure(out, test, example,
-                                       got+_exception_traceback(exc_info))
+                                       got + _exception_traceback(exc_info))
                         failures += 1
 
         # Restore the option flags (in case they were modified)
