Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 2a7f384

Browse files
tim-onetim-one
committed
SF bug 430991: wrong co_lnotab
Armin Rigo pointed out that the way the line-# table got built didn't work for lines generating more than 255 bytes of bytecode. Fixed as he suggested, plus corresponding changes to pyassem.py, plus added some long overdue docs about this subtle table to compile.c. Bugfix candidate (line numbers may be off in tracebacks under -O).
1 parent eefa964 commit 2a7f384

4 files changed

Lines changed: 83 additions & 44 deletions

File tree

‎Lib/compiler/pyassem.py‎

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -613,16 +613,16 @@ def twobyte(val):
613613
class LineAddrTable:
614614
"""lnotab
615615
616-
This class builds the lnotab, which is undocumented but described
617-
by com_set_lineno in compile.c. Here's an attempt at explanation:
616+
This class builds the lnotab, which is documented in compile.c.
617+
Here's a brief recap:
618618
619619
For each SET_LINENO instruction after the first one, two bytes are
620620
added to lnotab. (In some cases, multiple two-byte entries are
621621
added.) The first byte is the distance in bytes between the
622622
instruction for the last SET_LINENO and the current SET_LINENO.
623623
The second byte is offset in line numbers. If either offset is
624-
greater than 255, multiple two-byte entries are added -- one entry
625-
for each factor of 255.
624+
greater than 255, multiple two-byte entries are added -- see
625+
compile.c for the delicate details.
626626
"""
627627

628628
def __init__(self):
@@ -657,19 +657,16 @@ def nextLine(self, lineno):
657657
# compiler because it only generates a SET_LINENO instruction
658658
# for the assignment.
659659
if line > 0:
660-
while addr > 0 or line > 0:
661-
# write the values in 1-byte chunks that sum
662-
# to desired value
663-
trunc_addr = addr
664-
trunc_line = line
665-
if trunc_addr > 255:
666-
trunc_addr = 255
667-
if trunc_line > 255:
668-
trunc_line = 255
669-
self.lnotab.append(trunc_addr)
670-
self.lnotab.append(trunc_line)
671-
addr = addr - trunc_addr
672-
line = line - trunc_line
660+
push = self.lnotab.append
661+
while addr > 255:
662+
push(255); push(0)
663+
addr -= 255
664+
while line > 255:
665+
push(addr); push(255)
666+
line -= 255
667+
addr = 0
668+
if addr > 0 or line > 0:
669+
push(addr); push(line)
673670
self.lastline = lineno
674671
self.lastoff = self.codeOffset
675672

‎Misc/ACKS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -319,6 +319,7 @@ Michael P. Reilly
319319
Bernhard Reiter
320320
Steven Reiz
321321
Jan Pieter Riegel
322+
Armin Rigo
322323
Nicholas Riley
323324
Jean-Claude Rimbault
324325
Andy Robinson

‎Python/compile.c‎

Lines changed: 54 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -336,6 +336,50 @@ PyCode_New(int argcount, int nlocals, int stacksize, int flags,
336336
c_argcount, c_globals, and c_flags.
337337
*/
338338

339+
/* All about c_lnotab.
340+
341+
c_lnotab is an array of unsigned bytes disguised as a Python string. In -O
342+
mode, SET_LINENO opcodes aren't generated, and bytecode offsets are mapped
343+
to source code line #s (when needed for tracebacks) via c_lnotab instead.
344+
The array is conceptually a list of
345+
(bytecode offset increment, line number increment)
346+
pairs. The details are important and delicate, best illustrated by example:
347+
348+
byte code offset source code line number
349+
0 1
350+
6 2
351+
50 7
352+
350 307
353+
361 308
354+
355+
The first trick is that these numbers aren't stored, only the increments
356+
from one row to the next (this doesn't really work, but it's a start):
357+
358+
0, 1, 6, 1, 44, 5, 300, 300, 11, 1
359+
360+
The second trick is that an unsigned byte can't hold negative values, or
361+
values larger than 255, so (a) there's a deep assumption that byte code
362+
offsets and their corresponding line #s both increase monotonically, and (b)
363+
if at least one column jumps by more than 255 from one row to the next, more
364+
than one pair is written to the table. In case #b, there's no way to know
365+
from looking at the table later how many were written. That's the delicate
366+
part. A user of c_lnotab desiring to find the source line number
367+
corresponding to a bytecode address A should do something like this
368+
369+
lineno = addr = 0
370+
for addr_incr, line_incr in c_lnotab:
371+
addr += addr_incr
372+
if addr > A:
373+
return lineno
374+
lineno += line_incr
375+
376+
In order for this to work, when the addr field increments by more than 255,
377+
the line # increment in each pair generated must be 0 until the remaining addr
378+
increment is < 256. So, in the example above, com_set_lineno should not (as
379+
was actually done until 2.2) expand 300, 300 to 255, 255, 45, 45, but to
380+
255, 0, 45, 255, 0, 45.
381+
*/
382+
339383
struct compiling {
340384
PyObject *c_code; /* string */
341385
PyObject *c_consts; /* list of objects */
@@ -692,17 +736,17 @@ com_set_lineno(struct compiling *c, int lineno)
692736
else {
693737
int incr_addr = c->c_nexti - c->c_last_addr;
694738
int incr_line = lineno - c->c_last_line;
695-
while (incr_addr > 0 || incr_line > 0) {
696-
int trunc_addr = incr_addr;
697-
int trunc_line = incr_line;
698-
if (trunc_addr > 255)
699-
trunc_addr = 255;
700-
if (trunc_line > 255)
701-
trunc_line = 255;
702-
com_add_lnotab(c, trunc_addr, trunc_line);
703-
incr_addr -= trunc_addr;
704-
incr_line -= trunc_line;
739+
while (incr_addr > 255) {
740+
com_add_lnotab(c, 255, 0);
741+
incr_addr -= 255;
742+
}
743+
while (incr_line > 255) {
744+
com_add_lnotab(c, incr_addr, 255);
745+
incr_line -=255;
746+
incr_addr = 0;
705747
}
748+
if (incr_addr > 0 || incr_line > 0)
749+
com_add_lnotab(c, incr_addr, incr_line);
706750
c->c_last_addr = c->c_nexti;
707751
c->c_last_line = lineno;
708752
}

‎Tools/compiler/compiler/pyassem.py‎

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -613,16 +613,16 @@ def twobyte(val):
613613
class LineAddrTable:
614614
"""lnotab
615615
616-
This class builds the lnotab, which is undocumented but described
617-
by com_set_lineno in compile.c. Here's an attempt at explanation:
616+
This class builds the lnotab, which is documented in compile.c.
617+
Here's a brief recap:
618618
619619
For each SET_LINENO instruction after the first one, two bytes are
620620
added to lnotab. (In some cases, multiple two-byte entries are
621621
added.) The first byte is the distance in bytes between the
622622
instruction for the last SET_LINENO and the current SET_LINENO.
623623
The second byte is offset in line numbers. If either offset is
624-
greater than 255, multiple two-byte entries are added -- one entry
625-
for each factor of 255.
624+
greater than 255, multiple two-byte entries are added -- see
625+
compile.c for the delicate details.
626626
"""
627627

628628
def __init__(self):
@@ -657,19 +657,16 @@ def nextLine(self, lineno):
657657
# compiler because it only generates a SET_LINENO instruction
658658
# for the assignment.
659659
if line > 0:
660-
while addr > 0 or line > 0:
661-
# write the values in 1-byte chunks that sum
662-
# to desired value
663-
trunc_addr = addr
664-
trunc_line = line
665-
if trunc_addr > 255:
666-
trunc_addr = 255
667-
if trunc_line > 255:
668-
trunc_line = 255
669-
self.lnotab.append(trunc_addr)
670-
self.lnotab.append(trunc_line)
671-
addr = addr - trunc_addr
672-
line = line - trunc_line
660+
push = self.lnotab.append
661+
while addr > 255:
662+
push(255); push(0)
663+
addr -= 255
664+
while line > 255:
665+
push(addr); push(255)
666+
line -= 255
667+
addr = 0
668+
if addr > 0 or line > 0:
669+
push(addr); push(line)
673670
self.lastline = lineno
674671
self.lastoff = self.codeOffset
675672

0 commit comments

Comments
 (0)