Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit d67ddbb

Browse files
committed
Eric Raymond:
(1) Added and documented the capability for shlex to handle lexical-level inclusion and a stack of input sources. Also, the input stream member is now documented, and the constructor takes an optional source-filename. The class provides facilities to generate error messages that track file and line number. (2) Add a convenience function to generate C-compiler style error leaders.
1 parent 4b83ecb commit d67ddbb

1 file changed

Lines changed: 64 additions & 2 deletions

File tree

Doc/lib/libshlex.tex

Lines changed: 64 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,15 @@ \section{\module{shlex} ---
1313
be useful for writing minilanguages, e.g.\ in run control files for
1414
Python applications.
1515

16-
\begin{classdesc}{shlex}{\optional{stream}}
16+
\begin{classdesc}{shlex}{\optional{stream}, \optional{file}}
1717
A \class{shlex} instance or subclass instance is a lexical analyzer
1818
object. The initialization argument, if present, specifies where to
1919
read characters from. It must be a file- or stream-like object with
2020
\method{read()} and \method{readline()} methods. If no argument is given,
21-
input will be taken from \code{sys.stdin}.
21+
input will be taken from \code{sys.stdin}. The second optional
22+
argument is a filename string, which sets the initial value of the
23+
\member{infile} member. If the stream argument is omitted or
24+
equal to \code{sys.stdin}, this second argument defauilts to ``stdin''.
2225
\end{classdesc}
2326

2427

@@ -43,6 +46,38 @@ \subsection{shlex Objects \label{shlex-objects}}
4346
Push the argument onto the token stack.
4447
\end{methoddesc}
4548

49+
\begin{methoddesc}{read_token}{}
50+
Read a raw token. Ignore the pushback stack, and do not interpret source
51+
requests. (This is not ordinarily a useful entry point, and is
52+
documented here only for the sake of completeness.)
53+
\end{methoddesc}
54+
55+
\begin{methoddesc}{openhook}{filename}
56+
When shlex detects a source request (see \member{source} below)
57+
this method is given the following token as argument, and expected to
58+
return a tuple consisting of a filename and an opened stream object.
59+
60+
Normally, this method just strips any quotes off the argument and
61+
treats it as a filename, calling \code{open()} on it. It is exposed so that
62+
you can use it to implement directory search paths, addition of
63+
file extensions, and other namespace hacks.
64+
65+
There is no corresponding `close' hook, but a shlex instance will call
66+
the \code{close()} method of the sourced input stream when it returns EOF.
67+
\end{methoddesc}
68+
69+
\begin{methoddesc}{error_leader}{\optional{file}, \optional{line}}
70+
This method generates an error message leader in the format of a
71+
Unix C compiler error label; the format is '"\%s", line \%d: ',
72+
where the \%s is replaced with the name of the current source file and
73+
the \%d with the current input line number (the optional arguments
74+
can be used to override these).
75+
76+
This convenience is provided to encourage shlex users to generate
77+
error messages in the standard, parseable format understood by Emacs
78+
and other Unix tools.
79+
\end{methoddesc}
80+
4681
Instances of \class{shlex} subclasses have some public instance
4782
variables which either control lexical analysis or can be used
4883
for debugging:
@@ -72,6 +107,33 @@ \subsection{shlex Objects \label{shlex-objects}}
72107
\ASCII{} single and double quotes.
73108
\end{memberdesc}
74109

110+
\begin{memberdesc}{infile}
111+
The name of the current input file, as initially set at class
112+
instantiation time or stacked by later source requests. It may
113+
be useful to examine this when constructing error messages.
114+
\end{memberdesc}
115+
116+
\begin{memberdesc}{instream}
117+
The input stream from which this shlex instance is reading characters.
118+
\end{memberdesc}
119+
120+
\begin{memberdesc}{source}
121+
This member is None by default. If you assign a string to it, that
122+
string will be recognized as a lexical-level inclusion request similar
123+
to the `source' keyword in various shells. That is, the immediately
124+
following token will opened as a filename and input taken from that
125+
stream until EOF, at which point the \code{close()} method of that
126+
stream will be called and the input source will again become the
127+
original input stream. Source requests may be stacked any number of
128+
levels deep.
129+
\end{memberdesc}
130+
131+
\begin{memberdesc}{debug}
132+
If this member is numeric and 1 or more, a shlex instance will print
133+
verbose progress output on its behavior. If you need to use this,
134+
you can read the module source code to learn the details.
135+
\end{memberdesc}
136+
75137
Note that any character not declared to be a word character,
76138
whitespace, or a quote will be returned as a single-character token.
77139

0 commit comments

Comments
 (0)