This patch syncs the regex implementation with gnulib (commit 0ee5212).
Only two changes in GLIBC regex testing are required:
1. posix/bug-regex28.c: as previously discussed [1] the change of
expected results on the pattern should be safe.
2. posix/PCRE.tests: the ERE (a)|\1 is malformed (in the sense that
the \1 doesn't mean anything) and although current GLIBC accepts
it has undefined behavior. This patch removes the specific test.
This sync contains some patches from thread 'Regex: Make libc regex
more usable outside GLIBC.' [2] which have been pushed upstream in
gnulib. This patches also fixes some regex issues (BZ #23233,
BZ #21163, BZ #18986, BZ #13762) and I did not add testcases for
both #23233 and #13762 because I couldn't think a simple way to
trigger the expected failure path to trigger them.
Checked on x86_64-linux-gnu and i686-linux-gnu.
[BZ #23233]
[BZ #21163]
[BZ #18986]
[BZ #13762]
* posix/Makefile (tests): Add bug-regex37 and bug-regex38.
* posix/PCRE.tests: Remove invalid test.
* posix/bug-regex28.c: Fix expected values for used syntax.
* posix/bug-regex37.c: New file.
* posix/bug-regex38.c: Likewise.
* posix/regcomp.c: Sync with gnulib.
* posix/regex.c: Likewise.
* posix/regex.h: Likewise.
* posix/regex_internal.c: Likewise.
* posix/regex_internal.h: Likewise.
* posix/regexec.c: Likewise.
[1] https://sourceware.org/ml/libc-alpha/2017-12/msg00807.html
[2] https://sourceware.org/ml/libc-alpha/2017-12/msg00237.html
This completes the deprecation and removal of this inclusion, which
was begun in the 2.25 release.
* posix/sys/types.h: Don't include sys/sysmacros.h.
* misc/sys/sysmacros.h: Remove the conditional deprecation
warnings for the macros defined by this header.
The tst-glob_lstat_compat test needs to run tests on the previous
version of glob. On alpha, there are three versions of glob, GLIBC_2.0,
GLIBC_2.1 and GLIBC_2.27, while on other architectures there are only
the GLIBC_2.0 and GLIBC_2.27 version. Therefore on alpha the previous
version is GLIBC_2.1 and not GLIBC_2.0.
Changelog:
[BZ #22818]
* posix/tst-glob_lstat_compat.c [__alpha__] (glob): Access
the GLIBC_2.1 version.
(cherry picked from commit f8d7958289)
This patch increases timeouts on three tests I observed timing out on
slow systems.
* malloc/tst-malloc-tcache-leak.c (TIMEOUT): Define to 50.
* posix/tst-glob-tilde.c (TIMEOUT): Define to 200.
* resolv/tst-resolv-res_ninit.c (TIMEOUT): Define to 50.
Sync with gnulib 0e14f025d2.
Checked on x86_64-linux-gnu.
* lib/glob.c (glob): Use a 'char *', not a 'void *', in pointer
arithmetic.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Problem reported by Tim Rühsen [1]. Sync with gnulib 0e14f025d2.
[1] https://lists.gnu.org/archive/html/bug-gnulib/2017-10/msg00054.html
Checked on x86_64-linux-gnu.
* lib/glob.c (glob): Do not pass NULL to mempcpy.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The test posix/tst-glob_symlinks.c fails to build with GCC mainline:
tst-glob_symlinks.c: In function 'do_test':
tst-glob_symlinks.c:124:30: error: 'snprintf' output may be truncated before the last format character [-Werror=format-truncation=]
snprintf (buf, sizeof buf, "%s?", dangling_link);
^~~~~
tst-glob_symlinks.c:124:3: note: 'snprintf' output between 2 and 4097 bytes into a destination of size 4096
snprintf (buf, sizeof buf, "%s?", dangling_link);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tst-glob_symlinks.c:128:30: error: 'snprintf' output may be truncated before the last format character [-Werror=format-truncation=]
snprintf (buf, sizeof buf, "%s*", dangling_link);
^~~~~
tst-glob_symlinks.c:128:3: note: 'snprintf' output between 2 and 4097 bytes into a destination of size 4096
snprintf (buf, sizeof buf, "%s*", dangling_link);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This patch fixes the test to avoid such truncation warnings by
increasing the buffer in question by one byte, to ensure it can hold
any possible result of %s? or %s* formats where %s comes from a buffer
of size PATH_MAX.
Tested compilation with build-many-glibcs.py for aarch64-linux-gnu.
* posix/tst-glob_symlinks.c (do_test): Increase size of buf.
Three tests fail with a read-only source directory because they try to
write into the source directory. None of these write into it in a way
that should actually be problematic for concurrent builds sharing the
same writable source directory, but avoiding any writing into the
source directory (from testing, or from building glibc if the source
timestamps are properly ordered) is still a good idea, as being able
to build with read-only sources helps make sure there isn't anything
that could cause problems for concurrent builds.
This patch changes the tests in question to use either /tmp or the
build directory to write their temporary files (or to test O_TMPFILE,
as applicable).
Tested for x86_64.
* io/Makefile (tst-open-tmpfile-ARGS): New variable.
* posix/tst-mmap-offset.c (fname): Use /tmp.
* stdlib/tst-setcontext3.sh (tempfile): Use ${objpfx}.
These tests need a working Internet connection with DNS.
We have additional coverage of getaddrinfo through the resolv tests, so
the loss of default test coverage seems acceptable.
Hide internal __sched_setparam function to allow direct access within
libc.so and libc.a without using GOT nor PLT.
__GI___sched_setparam is defined when sysdeps/unix/syscalls.list is
used to generate sched_setparam. Otherwise libc_hidden_def is needed
explicitly.
[BZ #18822]
* include/sched.h (__sched_setparam): Add libc_hidden_proto.
* posix/sched_setp.c (__sched_setparam): Add libc_hidden_def.
Hide internal __posix_spawn_file_actions_realloc and /__spawni functions
to allow direct access within libc.so and libc.a without using GOT nor
PLT.
[BZ #18822]
* posix/spawn_int.h (__posix_spawn_file_actions_realloc): Add
attribute_hidden.
(__spawni): Likewise.
This patch follows commit 5554304f0 (posix: Allow glob to match dangling
symlinks [BZ #866]) by adding a compat symbol that follow previous
semantic of not following dangling symlinks and thus avoiding call
gl_lstat with GLOB_ALTDIRFUNC.
It avoids failure with old binaries that not set the alternate function
pointer for lstat (GNUmake for instance). The following scenario, for
instance, fails with current GNUmake because glibc will access unitialized
memory when calling gl_lstat:
$ cat src/t/t.c
int main ()
{
return 0;
}
$ cat Makefile
SRC = $(wildcard src/*/t.c)
OBJ = $(patsubst src/%.c, obj/%.o, $(SRC))
prog: $(OBJ)
$(CC) $(CFLAGS) $(LDFLAGS) $(LIBS) $(OBJ) -o prog
obj/%.o: src/%.c
$(CC) $(CFLAGS) -c $< -o $@
$ make
This works as expected with the patch applied. Since it is for generic
ABI, default compat symbols are added with override for Linux due LFS.
Now we have two compat symbols for glob on Linux:
1. sysdeps/unix/sysv/linux/oldglob.c which implements glob64 with
the old dirent layout. For this implementation I also set it to
not follow dangling symlinks (which is the safest path).
2. sysdeps/unix/sysv/linux/glob{64}-lstat-compat.c which implements
the compat symbol for dangling symlinks. As for generic glob,
the implementation uses XSTAT_IS_XSTAT64 to define whether
both __glob_lstat_compat and __glob64_lstat_compat should be
different implementations. For archictures that define
XSTAT_IS_XSTAT64, __glob_lstat_compat is aliased to
__glob64_lstat_compat.
3. sysdeps/unix/sysv/linux/alpha/oldglob.c with a different glob_t
layout. As for 1. this patch changes it to not follow dangling
symlinks.
The patch also bumps _GNU_GLOB_INTERFACE_VERSION to 2 to advertise the
new semantic. On GNUmake, for instance, it will force to it use its
internal glob implementation instead and avoiding triggering the same
failure on builds against newer GLIBCs.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also checked
with a build against the major ABIs required to check for the abilist.
The changes should also work on gnulib (I run gnulib-tool.py check glob
and it shown no regressions).
[BZ #22183]
* include/gnu-versions.h (_GNU_GLOB_INTERFACE_VERSION): Increase
version to 2.
* posix/Makefile (routines): Add glob-lstat-compat and
glob64-lstat-compat.
* posix/Versions (GLIBC_2.27, glob, glob64): Add symbol version.
* posix/glob-lstat-compat.c: New file.
* posix/glob64-lstat-compat.c: Likewise.
* posix/tst-glob_lstat_compat.c: Likewise.
* sysdeps/unix/sysv/linux/glob-lstat-compat.c: Likewise.
* sysdeps/unix/sysv/linux/alpha/glob-lstat-compat.c: Likewise.
* sysdeps/unix/sysv/linux/glob64-lstat-compat.c: Likewise.
* sysdeps/unix/sysv/linux/alpha/glob.c: Remove file.
* posix/glob.c (glob_lstat): New function.
(glob): Rename to __glob and add versioned symbol to 2.27.
(glob_in_dir): Use glob_lstat.
* posix/glob64.c (glob64): Add GLOB_ATTRIBUTE.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/glob.c (glob): Add versioned symbol for
2.27.
* sysdeps/unix/sysv/linux/glob64.c (glob64): Likewise.
* sysdeps/unix/sysv/linux/oldglob.c (GLOB_NO_LSTAT): Define.
* sysdeps/unix/sysv/linux/alpha/oldglob.c (__old_glob): Do not use
gl_lstat on glob call.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Add GLIBC_2.27 glob
and glob64 symbols.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/linux/powerpc/powerpc32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
This patch replaces the internal integer constant from
__glob_pattern_type return with a proper enum.
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
* posix/glob_internal.h (glob_pattern_type_t): New enumeration.
(__glob_pattern_type): Use __glob_pat_types.
* posix/glob_pattern_p.c (__glob_pattern_p): Likewise.
* posix/glob.c (glob): Likewise.
(glob_in_dir): Likewise.
Current glob implementation allows unlimited user name for home
directory construction on GLOB_TILDE case. To accomplish it glob
either construct a name on stack if size are small enough (based
on current alloca_used) or in heap otherwise.
This patch simplifies storage allocation by using the same scratch
buffer for both get_rlogin_r and getpwnam_r.
This also syncs with gnulib commit 064df0b (glob: fix bugs with long
login names).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
* posix/glob.c (GET_LOGIN_NAME_MAX): Remove.
(glob): Use the same scratch buffer for both getlogin_r and
getpwnam_r. Don’t require preallocation of the login name. This
simplifies storage allocation, and corrects the handling of
long login names.
This patch fixes longstanding misuse of errno after getpwnam_r,
which returns an error number rather than setting errno. This is
sync with gnulib commit 5db9301.
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #1062]
* posix/glob.c (glob): Port recent patches to platforms
lacking getpwnam_r.
(glob): Fix longstanding misuse of errno after getpwnam_r, which
returns an error number rather than setting errno.
This patch makes glob match dangling symlinks. Compared to other glob
implementation (*BSD, bash, musl, and other shells as well), GLIBC seems
the be the only one that does not match dangling symlinks. As for
comment #5 in BZ #866, POSIX does not have any strict specification for
dangling symlinks match and it is reasonable that trying to glob everything
in a path should return all types of files (such as for a 'rm *'). Also,
comment #7 shows even more example where GLIBC current behavior is
unexepected.
I avoided adding another GNU specific flag to set this behavior and
instead make it the default. Although this change the semanthic from
previous implementation, I think adding another compat symbol to be
really unecessary as from aforementioned reasons (current behavior not
defined in any standard, general idea of different implementation is
to list dangling symbols).
This also sync glob with gnulib commit fd1daf4 (glob: match dangling
symlinks).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #866]
[BZ #1062]
* posix/Makefile (tests): Remove bug-glob1 and tst-glob_symlinks.
* posix/bug-glob1.c: Remove file.
* posix/tst-glob_symlinks.c: New file.
* posix/glob.c (__lstat64): New macro.
(is_dir): New function.
(glob, glob_in_dir): Match symlinks even if they are dangling.
(link_stat, link_exists_p): Remove. All uses removed.
According to this kernel commit 2adc376c55194, d_ino 0 is a regular inode
number on Linux (which also matches POSIX, as it does not treat the value
as special). This patch makes glob accept is a valid inode number.
This is also a sync with gnulib commit c8e57c1.
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #1062]
[BZ #19971]
* posix/glob.c (struct readdir_result): Remove skip_entry member.
(readdir_result_skip_entry, D_INO_TO_RESULT): Remove.
All uses removed.
This patch syncs posix/glob.c implementation with gnulib version
b5ec983 (glob: simplify symlink detection). The only difference
to gnulib code is
* DT_UNKNOWN, DT_DIR, and DT_LNK definition in the case there
were not already defined. Gnulib code which uses
HAVE_STRUCT_DIRENT_D_TYPE will redefine them wrongly because
GLIBC does not define HAVE_STRUCT_DIRENT_D_TYPE. Instead
the patch check for each definition instead.
Also, the patch requires additional globfree and globfree64 files
for compatibility version on some architectures. Also the code
simplification leads to not macro simplification (not need for
NO_GLOB_PATTERN_P anymore).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #1062]
* posix/Makefile (routines): Add globfree, globfree64, and
glob_pattern_p.
* posix/flexmember.h: New file.
* posix/glob_internal.h: Likewise.
* posix/glob_pattern_p.c: Likewise.
* posix/globfree.c: Likewise.
* posix/globfree64.c: Likewise.
* sysdeps/gnu/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/alpha/globfree.c: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/oldglob.c: Likewise.
* sysdeps/unix/sysv/linux/wordsize-64/globfree64.c: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/globfree.c: Likewise.
* sysdeps/wordsize-64/globfree.c: Likewise.
* sysdeps/wordsize-64/globfree64.c: Likewise.
* posix/glob.c (HAVE_CONFIG_H): Use !_LIBC instead.
[NDEBUG): Remove comments.
(GLOB_ONLY_P, _AMIGA, VMS): Remove define.
(dirent_type): New type. Use uint_fast8_t not
uint8_t, as C99 does not require uint8_t.
(DT_UNKNOWN, DT_DIR, DT_LNK): New macros.
(struct readdir_result): Use dirent_type. Do not define skip_entry
unless it is needed; this saves a byte on platforms lacking d_ino.
(readdir_result_type, readdir_result_skip_entry):
New functions, replacing ...
(readdir_result_might_be_symlink, readdir_result_might_be_dir):
these functions, which were removed. This makes the callers
easier to read. All callers changed.
(D_INO_TO_RESULT): Now empty if there is no d_ino.
(size_add_wrapv, glob_use_alloca): New static functions.
(glob, glob_in_dir): Check for size_t overflow in several places,
and fix some size_t checks that were not quite right.
Remove old code using SHELL since Bash no longer
uses this.
(glob, prefix_array): Separate MS code better.
(glob_in_dir): Remove old Amiga and VMS code.
(globfree, __glob_pattern_type, __glob_pattern_p): Move to
separate files.
(glob_in_dir): Do not rely on undefined behavior in accessing
struct members beyond their bounds. Use a flexible array member
instead
(link_stat): Rename from link_exists2_p and return -1/0 instead of
0/1. Caller changed.
(glob): Fix memory leaks.
* posix/glob64 (globfree64): Move to separate file.
* sysdeps/gnu/glob64.c (NO_GLOB_PATTERN_P): Remove define.
(globfree64): Remove hidden alias.
* sysdeps/unix/sysv/linux/Makefile (sysdeps_routines): Add
oldglob.
* sysdeps/unix/sysv/linux/alpha/glob.c (__new_globfree): Move to
separate file.
* sysdeps/unix/sysv/linux/i386/glob64.c (NO_GLOB_PATTERN_P): Remove
define.
Move compat code to separate file.
* sysdeps/wordsize-64/glob.c (globfree): Move definitions to
separate file.
Mark internal nss symbols with attribute_hidden to allow direct access
within libc.so and libc.a without using GOT nor PLT.
Tested on x86-64 with and without --disable-nscd.
[BZ #18822]
* grp/initgroups.c (__nss_group_database): Removed.
(__nss_initgroups_database): Likewise.
* nscd/gai.c (__nss_hosts_database): Likewise.
* nss/XXX-lookup.c (DATABASE_NAME_SYMBOL): Likewise.
* posix/tst-rfc3484-2.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484-3.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484.c (__nss_hosts_database): Likewise.
* sysdeps/posix/getaddrinfo.c (__nss_hosts_database): Likewise.
* nss/getXXent.c (INTERNAL (REENTRANT_GETNAME)): Add
attribute_hidden.
* nss/nsswitch.c (__nss_database_custom): Define only if
USE_NSCD is defined.
(__nss_configure_lookup): Use __nss_database_custom only if
USE_NSCD is defined.
* nss/nsswitch.h (__nss_database_custom): Declare only if
USE_NSCD is defined. Add attribute_hidden.
(__nss_setent): Add attribute_hidden.
(__nss_endent): Likewise.
(__nss_getent_r): Likewise.
(__nss_getent): Likewise.
(DEFINE_DATABASE): Declare __nss_##arg##_database.
bits/types.h defines a type __qaddr_t that is not used anywhere in
glibc. It doesn't appear to be widely used outside glibc either
(judging by codesearch.debian.net, where hits are generally copies of
definitions of this type, not uses), so it seems appropriate to remove
this type, which this patch does.
Tested for x86_64.
* posix/bits/types.h (__qaddr_t): Remove.
