From andy@hmc.edu

This is a bug report for perl from andy@hmc.edu,
generated with the help of perlbug 1.39 running under perl 5.18.2.

Off-by-one when truncating *shared* array by: $#ary -= $chop_this_many

What should happen: Shared array max index should drop to a new, lower value.

What does happen: Actual new max index is *too large by one* (Too few elements chopped off)
See the 4th line in the sample output; "...index is: 5"; It should say "...index is: 4".

See additional notes and environment details after example code and example output below.

$ ./perl-shared-array-truncate.pl
Before truncation, ordinary array highest index is: 9
After removing five, ordinary array highest index is: 4

Before truncation, shared array highest index is: 9
After removing five, shared array highest index is: 5

Shared array now contains: 0 | 1 | 2 | 3 | 4 | 5

$ cat perl-shared-array-truncate.pl
#!/usr/bin/perl -w
use strict;

use threads;
use threads::shared;

### First with an ordinary array
my @ordinary_array;
@ordinary_array = (0,1,2,3,4,5,6,7,8,9);

printf "Before truncation, ordinary array highest index is: %s\n", $#ordinary_array;
$#ordinary_array = $#ordinary_array - 5;
printf "After removing five, ordinary array highest index is: %s\n\n", $#ordinary_array;

### Then try it with a shared array
my @shared_array :shared;
lock @shared_array; # Same with or without this line
@shared_array = (0,1,2,3,4,5,6,7,8,9);

printf "Before truncation, shared array highest index is: %s\n", $#shared_array;
$#shared_array = $#shared_array - 5;
printf "After removing five, shared array highest index is: %s\n\n", $#shared_array;
printf "Shared array now contains: %s\n", join(' | ', @shared_array);

----- environment and notes -----

$ uname -a
Linux andy.xxxxxxx 3.15.10-201.fc20.x86_64 #1 SMP Wed Aug 27 21:10:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ /usr/bin/perl -v
This is perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-linux-thread-multi
(with 19 registered patches, see perl -V for more detail)

Notes:

Although I am using the "threads" module in this example script, I am not spawning
any threads here (although I do in the real code in which this problem surfaced).
This is not a concurrency problem. I guess that was obvious. :-)

I am aware that for removing a small number of elements I can "pop" a few times however
I am using this to implement an LRU queue of keys to track the order of elements added
to a hash. I want to occasionally remove ~1000 elements from the end of a ~10000 element
array and I fear that pop on that scale will be slow and inefficient.
Yes, I can compensate for the off-by-one in the meantime.

I am aware that splice is not allowed with shared arrays. This truncation does not appear
to violate (my reading of) the rules for shared arrays, and one of the test scripts at:

cpansearch.perl.org/src/RJBS/perl-5.17.6/dist/threads-shared/t/av_simple.t

uses this method to truncate an array.

I am aware of Thread::Queue.

I have also repeated this on these environments with same results:

Linux xxxxxxxx 2.6.32-358.2.1.el6.centos.plus.x86_64 #1 SMP Wed Mar 13 02:09:07 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
This is perl, v5.10.1 (*) built for x86_64-linux-thread-multi

Linux andy2 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi

Thank you very much.

Andy

Flags:
category=library
severity=medium
module=threads::shared

Site configuration information for perl 5.18.2:

Configured by Red Hat, Inc. at Tue Jan 7 14:45:19 UTC 2014.

Summary of my perl5 (revision 5 version 18 subversion 2) configuration:

Platform:
osname=linux, osvers=3.11.9-200.fc19.x86_64, archname=x86_64-linux-thread-multi
uname='linux buildvm-12.phx2.fedoraproject.org 3.11.9-200.fc19.x86_64 #1 smp wed nov 20 21:22:24 utc 2013 x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro -Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.18.2 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.8.2 20131212 (Red Hat 4.8.2-7)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.18'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches:
Fedora Patch1: Removes date check, Fedora/RHEL specific
Fedora Patch3: support for libdir64
Fedora Patch4: use libresolv instead of libbind
Fedora Patch5: USE_MM_LD_RUN_PATH
Fedora Patch6: Skip hostname tests, due to builders not being network capable
Fedora Patch7: Dont run one io test due to random builder failures
Fedora Patch9: Fix find2perl to translate ? glob properly (RT#113054)
Fedora Patch10: Update h2ph(1) documentation (RT#117647)
Fedora Patch11: Update pod2html(1) documentation (RT#117623)
Fedora Patch12: Disable ornaments on perl5db AutoTrace tests (RT#118817)
Fedora Patch14: Do not use system Term::ReadLine::Gnu in tests (RT#118821)
Fedora Patch15: Define SONAME for libperl.so
Fedora Patch16: Install libperl.so to -Dshrpdir value
Fedora Patch18: Fix crash with \\&$glob_copy (RT#119051)
Fedora Patch19: Fix coreamp.t rand test (RT#118237)
Fedora Patch20: Reap child in case where exception has been thrown (RT#114722)
Fedora Patch21: Fix using regular expressions containing multiple code blocks (RT#117917)
Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on Linux
Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux

@INC for perl 5.18.2:
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5
.

Environment for perl 5.18.2:
HOME=/home/andy
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/andy/.local/bin:/home/andy/bin
PERL_BADLANG (unset)
SHELL=/bin/bash

Off-by-one when truncating *shared* array by: $#ary -= $chop_this_many #14151

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Off-by-one when truncating shared array by: $#ary -= $chop_this_many #14151