locale.c: Fix compilation on platforms with only a C locale #22569

the-horo · 2024-09-04T12:02:45Z

Reported downstream in: https://bugs.gentoo.org/939014. The bug is about being impossible to bootstrap a Gentoo prefix with perl-5.40.0.

A Gentoo prefix is a Linux installation under some custom directory, modifiable by a non-root user. Bootstrapping such an installation starts with a host compiler and a shell and builds packages until the full installation is complete. Because of the nature of perl, it ends up being compiled relatively early during the whole process, before the installation is complete and the user has had the chance to select which locales to generate.

In this situation perl ends up being compiled on a bare system with only the C locale. The code in Configure

perl5/Configure

Lines 17672 to 17675 in 67e8521

    
           if (distincts_count < C_ARRAY_LENGTH(categories)) { 
        
               fprintf(stderr, "Couldn't find a locale distinguishable from C\n"); 
        
               return 1; 
        
           }

then exits early failing to prove the format of LC_ALL which then takes the second case branch

perl5/Configure

Lines 17812 to 17834 in 67e8521

    
              output=`$run ./try 2>/dev/null` 
        
              separator=`echo "$output" | $sed 1q` 
        
              case $separator in 
        
           "\"=;\"") 
        
                    d_perl_lc_all_uses_name_value_pairs="$define" 
        
                    d_perl_lc_all_separator="$undef" 
        
                      perl_lc_all_separator= 
        
             d_perl_lc_all_category_positions_init="$undef" 
        
               perl_lc_all_category_positions_init= 
        
               ;; 
        
           "") d_perl_lc_all_uses_name_value_pairs="$undef" 
        
                      d_perl_lc_all_separator="$undef" 
        
                        perl_lc_all_separator= 
        
               d_perl_lc_all_category_positions_init="$undef" 
        
                 perl_lc_all_category_positions_init= 
        
               ;; 
        
                  *)  d_perl_lc_all_uses_name_value_pairs="$undef" 
        
                      d_perl_lc_all_separator="$define" 
        
                        perl_lc_all_separator="$separator" 
        
               d_perl_lc_all_category_positions_init="$define" 
        
                 perl_lc_all_category_positions_init=`echo "$output" | sed -n 2p` 
        
               ;; 
        
              esac

which makes all three of PERL_LC_ALL_USES_NAME_VALUE_PAIRS, PERL_LC_ALL_SEPARATOR, and PERL_LC_ALL_CATEGORY_POSITIONS_INIT end up being undefined. The locale.c code, however, assumes that either PERL_LC_ALL_USES_NAME_VALUE_PAIRS or the other two macros are defined which leads to a compilation failure because of the usage of PERL_LC_ALL_CATEGORY_POSITIONS_INIT:

x86_64-pc-linux-gnu-gcc -c -DPERL_CORE -O2 -pipe -O2 -pipe -fno-strict-aliasing -DNO_PERL_RAND_SEED -fwrapv -I/home/gabi/ghentu/usr/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -std=c99 -O2 -pipe -O2 -pipe -fno-strict-aliasing -Wall -Werror=pointer-arith -Werror=vla -Wextra -Wno-long-long -Wno-declaration-after-statement -Wc++-compat -Wwrite-strings -fPIC locale.c
locale.c: In function 'Perl_init_i18nl10n':
locale.c:8812:43: error: 'PERL_LC_ALL_CATEGORY_POSITIONS_INIT' undeclared (first use in this function)
 8812 |         int lc_all_category_positions[] = PERL_LC_ALL_CATEGORY_POSITIONS_INIT;
      |                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
locale.c:8812:43: note: each undeclared identifier is reported only once for each function it appears in

There is already code:

perl5/locale.c

Lines 649 to 658 in 67e8521

    
           #  else     /* name=value */ 
        
               /* In the, hopefully never occurring, event that the platform doesn't use 
        
                * either mechanism for disparate LC_ALL's, assume the name=value pairs 
        
                * form, rather than taking the extreme step of refusing to compile.  Many 
        
                * programs won't have disparate locales, so will generally work */ 
        
           #    define PERL_LC_ALL_SEPARATOR  ";" 
        
           #    define is_disparate_LC_ALL(name)  cBOOL(   strchr(name, ';')   \ 
        
                                                        && strchr(name, '=')) 
        
           #  endif

that prevents compilation failures with missing PERL_LC_ALL_SEPARATOR. This is a similar situation in purpose but, because PERL_LC_ALL_CATEGORY_POSITIONS_INIT is suppose to be an array initializer and, if generated inside the code, it would require a #include "locale_table.h" I think it's simpler to replace its single usage point. The map_LC_ALL_position_to_index array should still be initialized, even with logically wrong values, to prevent possible out-of-bounds errors in other parts of the code.

Signed-off-by: Andrei Horodniceanu <[email protected]>

thesamesam · 2024-09-09T13:01:01Z

cc @khwilliamson

khwilliamson · 2024-09-09T15:42:46Z

Thank you for this. I will also look into changing Configure to handle things so that this case doesn't arise. But it is a good fallback for Configure failures

Changelog: NAME perldelta - what is new for perl v5.40.1 DESCRIPTION This document describes differences between the 5.40.0 release and the 5.40.1 release. If you are upgrading from an earlier release such as 5.39.0, first read perl5400delta, which describes differences between 5.39.0 and 5.40.0. Incompatible Changes There are no changes intentionally incompatible with 5.40.0. If any exist, they are bugs, and we request that you submit a report. See "Reporting Bugs" below. Modules and Pragmata Updated Modules and Pragmata * File::Spec has been upgraded from version 3.90 to 3.91. * Module::CoreList has been upgraded from version 5.20240609 to 5.20250118_40. * warnings has been upgraded from version 1.69 to 1.70. Documentation Changes to Existing Documentation We have attempted to update the documentation to reflect the changes listed in this document. If you find any we have missed, open an issue at <https://github.com/Perl/perl5/issues>. Configuration and Compilation * Fixed compilation on platforms (e.g. "Gentoo Prefix") with only a C locale. [GH #22569 <Perl/perl5#22569>] Bug first reported downstream at bugs.gentoo.org/939014 <https://bugs.gentoo.org/939014>. * Fixed compilation error on some systems due to a typo in a printf() format. [GH #22793 <Perl/perl5#22793>] Testing Tests were added and changed to reflect the other additions and changes in this release. Selected Bug Fixes * Starting in Perl 5.39.8, ""strftime"" in POSIX would crash or produce odd errors (such as "Out of memory in perl:util:safesysmalloc") when given a format string that wasn't actually a string, but a number, "undef", or an object (even one with overloaded string conversion). Now "strftime" stringifies its first argument, as before. [GH #22498 <Perl/perl5#22498>] * Builds with "-msse" and quadmath on 32-bit x86 systems would crash with a misaligned access early in the build. [GH #22577 <Perl/perl5#22577>] * Using "goto" to tail call, or using the call_sv() and related APIs to call, any of trim(), refaddr(), reftype(), ceil(), floor() or stringify() in the "builtin::" package would crash or assert due to a "TARG" handling bug. [GH #22542 <Perl/perl5#22542>] * Fixed an issue where utf8n_to_uvchr() failed to correctly identify certain invalid UTF-8 sequences as invalid. Specifically, sequences that start with continuation bytes or unassigned bytes could cause unexpected behaviour or a panic. This fix ensures that such invalid sequences are now properly detected and handled. This correction also resolves related issues in modules that handle UTF-8 processing, such as Encode. * Fixed a double free error or possible memory leak when failing to compile certain regexps. [GH #21661 <Perl/perl5#21661>]

locale.c: Fix compilation on platforms with only a C locale

bd0ab50

Signed-off-by: Andrei Horodniceanu <[email protected]>

the-horo force-pushed the locale-fix branch from bcd554f to bd0ab50 Compare September 4, 2024 18:16

khwilliamson merged commit f18202d into Perl:blead Sep 9, 2024

thibaultduponchelle mentioned this pull request Sep 13, 2024

perldelta: Add notes about fix related to broken compilation on environments with very limited number of locales (e.g. Gentoo Prefix) #22589

Merged

3 tasks

stigtsp mentioned this pull request Sep 22, 2024

[staging] perl540: fix build in stage1, add C locale patch NixOS/nixpkgs#343761

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

locale.c: Fix compilation on platforms with only a C locale #22569

locale.c: Fix compilation on platforms with only a C locale #22569

Uh oh!

the-horo commented Sep 4, 2024

Uh oh!

thesamesam commented Sep 9, 2024

Uh oh!

khwilliamson commented Sep 9, 2024

Uh oh!

Uh oh!

	if (distincts_count < C_ARRAY_LENGTH(categories)) {
	fprintf(stderr, "Couldn't find a locale distinguishable from C\n");
	return 1;
	}

	output=`$run ./try 2>/dev/null`
	separator=`echo "$output" \| $sed 1q`
	case $separator in
	"\"=;\"")
	d_perl_lc_all_uses_name_value_pairs="$define"
	d_perl_lc_all_separator="$undef"
	perl_lc_all_separator=
	d_perl_lc_all_category_positions_init="$undef"
	perl_lc_all_category_positions_init=
	;;
	"") d_perl_lc_all_uses_name_value_pairs="$undef"
	d_perl_lc_all_separator="$undef"
	perl_lc_all_separator=
	d_perl_lc_all_category_positions_init="$undef"
	perl_lc_all_category_positions_init=
	;;
	*) d_perl_lc_all_uses_name_value_pairs="$undef"
	d_perl_lc_all_separator="$define"
	perl_lc_all_separator="$separator"
	d_perl_lc_all_category_positions_init="$define"
	perl_lc_all_category_positions_init=`echo "$output" \| sed -n 2p`
	;;
	esac

	# else /* name=value */

	/* In the, hopefully never occurring, event that the platform doesn't use
	* either mechanism for disparate LC_ALL's, assume the name=value pairs
	* form, rather than taking the extreme step of refusing to compile. Many
	* programs won't have disparate locales, so will generally work */
	# define PERL_LC_ALL_SEPARATOR ";"
	# define is_disparate_LC_ALL(name) cBOOL( strchr(name, ';') \
	&& strchr(name, '='))
	# endif

locale.c: Fix compilation on platforms with only a C locale #22569

locale.c: Fix compilation on platforms with only a C locale #22569

Uh oh!

Conversation

the-horo commented Sep 4, 2024

Uh oh!

thesamesam commented Sep 9, 2024

Uh oh!

khwilliamson commented Sep 9, 2024

Uh oh!

Uh oh!