File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -131,6 +131,24 @@ function it uses to do this is available:
131131
132132 .. versionadded :: 3.2
133133
134+ .. exception :: TokenError
135+
136+ Raised when either a docstring or expression that may be split over several
137+ lines is not completed anywhere in the file, for example::
138+
139+ """Beginning of
140+ docstring
141+
142+ or::
143+
144+ [1,
145+ 2,
146+ 3
147+
148+ Note that unclosed single-quoted strings do not cause an error to be
149+ raised. They are tokenized as ``ERRORTOKEN ``, followed by the tokenization of
150+ their contents.
151+
134152
135153.. _tokenize-cli :
136154
Original file line number Diff line number Diff line change @@ -755,6 +755,7 @@ Julia Lawall
755755Chris Lawrence
756756Brian Leair
757757Mathieu Leduc-Hamel
758+ Amandine Lee
758759Antony Lee
759760Christopher Lee
760761Inyeol Lee
You can’t perform that action at this time.
0 commit comments