This is the first of a 2-part patch set that fixes slow DSO sorting behavior in
the dynamic loader, as reported in BZ #17645. In order to facilitate such a
large modification to the dynamic loader, this first patch implements a testing
framework for validating shared object sorting behavior, to enable comparison
between old/new sorting algorithms, and any later enhancements.
This testing infrastructure consists of a Python script
scripts/dso-ordering-test.py' which takes in a description language, consisting
of strings that describe a set of link dependency relations between DSOs, and
generates testcase programs and Makefile fragments to automatically test the
described situation, for example:
a->b->c->d # four objects linked one after another
a->[bc]->d;b->c # a depends on b and c, which both depend on d,
# b depends on c (b,c linked to object a in fixed order)
a->b->c;{+a;%a;-a} # a, b, c serially dependent, main program uses
# dlopen/dlsym/dlclose on object a
a->b->c;{}!->[abc] # a, b, c serially dependent; multiple tests generated
# to test all permutations of a, b, c ordering linked
# to main program
(Above is just a short description of what the script can do, more
documentation is in the script comments.)
Two files containing several new tests, elf/dso-sort-tests-[12].def are added,
including test scenarios for BZ #15311 and Redhat issue #1162810 [1].
Due to the nature of dynamic loader tests, where the sorting behavior and test
output occurs before/after main(), generating testcases to use
support/test-driver.c does not suffice to control meaningful timeout for ld.so.
Therefore a new utility program 'support/test-run-command', based on
test-driver.c/support_test_main.c has been added. This does the same testcase
control, but for a program specified through a command-line rather than at the
source code level. This utility is used to run the dynamic loader testcases
generated by dso-ordering-test.py.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1162810
Signed-off-by: Chung-Lin Tang <cltang@codesourcery.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
I'd like to be able to test narrow and wide string interfaces, with
the narrow string tests using TEST_COMPARE_STRING and the wide string
tests using something analogous (possibly generated using macros from
a common test template for both the narrow and wide string tests where
appropriate).
Add such a TEST_COMPARE_STRING_WIDE, along with functions
support_quote_blob_wide and support_test_compare_string_wide that it
builds on. Those functions are built using macros from common
templates shared by the narrow and wide string implementations, though
I didn't do that for the tests of test functions. In
support_quote_blob_wide, I chose to use the \x{} delimited escape
sequence syntax proposed for C2X in N2785, rather than e.g. trying to
generate the end of a string and the start of a new string when
ambiguity would result from undelimited \x (when the next character
after such an escape sequence is valid hex) or forcing an escape
sequence to be used for the next character in the case of such
ambiguity.
Tested for x86_64.
It returns a range of file descriptor referring to the '/dev/null'
pathname. The function takes care of restarting the open range
if a file descriptor is found within the specified range and
also increases RLIMIT_NOFILE if required.
Checked on x86_64-linux-gnu.
The code to allocate a stack from xsigstack is refactored so it can
be more generic. The new support_stack_alloc() also set PROT_EXEC
if DEFAULT_STACK_PERMS has PF_X. This is required on some
architectures (hppa for instance) and trying to access the rtld
global from testsuite will require more intrusive refactoring
in the ldsodefs.h header.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also ran
tst-xsigstack on both hppa and ia64.
It is a wrapper for Linux clone syscall, to simplify the call to the
use only the most common arguments and remove architecture specific
handling (such as ia64 different name and signature).
It is a simple wrapper over timer_create, timer_settime, and
sigaction. It will be used to check for large timeout to trigger an
EINTR and to avoid use a large timeout (as for alarm()).
Reviewed-by: Lukasz Majewski <lukma@denx.de>
The patch adds redirections for xstat, xlstat, and xfstat when
_TIME_BITS=64 is defined.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
The patch adds redirections for xclock_gettime, xclock_settime,
timespec_add, timespec_sub, test_timespec_before_impl,
test_timespec_equal_or_after_impl, support_timespec_ns,
support_timespec_normalize, and support_timespec_check_in_range when
_TIME_BITS=64 is defined.
Co-authored-by: Lukasz Majewski <lukma@denx.de>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Both new tests io/tst-stat and io/tst-stat-lfs (_FILE_OFFSET_BITS=64)
are comparing the nanosecond fields with the statx result. Unfortunately
on s390(31bit) those fields are always zero if old KABI with non-LFS
support is used. With _FILE_OFFSET_BITS=64 stat is using statx internally.
As suggested by Adhemerval this patch disables the nanosecond check for
s390(31bit).
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Some Linux filesystems might not fully support 64 bit timestamps [1],
which make some Linux specific tests to fail when they check for the
functionality.
This patch adds a new libsupport function, support_path_support_time64,
that returns whether the target file supports or not 64 bit timestamps.
The support is checked by issuing a utimensat and verifying both the
last access and last modification time against a statx call.
The tests that might fail are also adjusted to check the file support
as well:
$ dd if=/dev/zero of=loopbackfile.img bs=100M count=1
1+0 records in
1+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0,0589568 s, 1,8 GB/s
$ sudo losetup -fP loopbackfile.img
$ mkfs.xfs loopbackfile.img
meta-data=loopbackfile.img isize=512 agcount=4, agsize=6400 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1
data = bsize=4096 blocks=25600, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=1368, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
$ mkdir loopfs
$ sudo mount -o loop /dev/loop0 loopfs/
$ sudo chown -R azanella:azanella loopfs
$ TMPDIR=loopfs/ ./testrun.sh misc/tst-utimes
error: ../sysdeps/unix/sysv/linux/tst-utimes.c:55: File loopfs//utimesfECsK1 does not support 64-bit timestamps
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1795576
The xclock_settime is a wrapper function on the clock_settime syscall
to be used in the test code.
It checks if the GLIBC_TEST_ALLOW_TIME_SETTING env variable is defined
in the environment in which test is executed. If it is not - the test
ends as unsupported. Otherwise, the clock-settime is executed and return
value is assessed.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
POSIX states that system returned code for failure to execute the shell
shall be as if the shell had terminated using _exit(127). This
behaviour was removed with 5fb7fc9635.
Checked on x86_64-linux-gnu.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
commit 04deeaa9ea
Author: Lucas A. M. Magalhaes <lamm@linux.ibm.com>
Date: Fri Jul 10 19:41:06 2020 -0300
Fix time/tst-cpuclock1 intermitent failures
has 2 issues:
1. It assumes time_t == long which is false on x32.
2. tst-timespec.c is compiled without -fexcess-precision=standard which
generates incorrect results on i686 in support_timespec_check_in_range:
double ratio = (double)observed_norm / expected_norm;
return (lower_bound <= ratio && ratio <= upper_bound);
This patch does
1. Compile tst-timespec.c with -fexcess-precision=standard.
2. Replace long with time_t.
3. Replace LONG_MIN and LONG_MAX with TYPE_MINIMUM (time_t) and
TYPE_MAXIMUM (time_t).
This test fails intermittently in systems with heavy load as
CLOCK_PROCESS_CPUTIME_ID is subject to scheduler pressure. Thus the
test boundaries were relaxed to keep it from failing on such systems.
A refactor of the spent time checking was made with some support
functions. With the advantage to representing time jitter in percent
of the target.
The values used by the test boundaries are all empirical.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Checked on x86-64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu,
and s390x-linux-gnu.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
During testing of localedef running in a minimal container
there were several error cases which were hard to diagnose
since they appeared as strerror (errno) values printed by the
higher level functions. This change adds three new verbose
messages for potential failure paths. The new messages give
the user the opportunity to use -v and display additional
information about why localedef might be failing. I found
these messages useful myself while writing a localedef
container test for --no-hard-links.
Since the changes cleanup the code that handle codeset
normalization we add tst-localedef-path-norm which contains
many sub-tests to verify the correct expected normalization of
codeset strings both when installing to default paths (the
only time normalization is enabled) and installing to absolute
paths. During the refactoring I created at least one
buffer-overflow which valgrind caught, but these tests did not
catch because the exec in the container had a very clean heap
with zero-initialized memory. However, between valgrind and
the tests the results are clean.
The new tst-localedef-path-norm passes without regression on
x86_64.
Change-Id: I28b9f680711ff00252a2cb15625b774cc58ecb9d
It allows parent process to wait for child state using a polling
strategy over procfs on Linux. The polling is used over ptrace to
avoid the need to handle signals on the target pid and to handle some
system specific limitation (such as YAMA).
The polling has some limitations, such as resource consumption due
the procfs read in a loop and the lack of synchronization after the
state is obtained.
The interface idea is to simplify some sleep synchronization waitid
tests and is to reduce timeouts by replacing arbitrary waits.
The testcase forks a child process and runs pldd with PID of
this child. On systems where /proc/sys/kernel/yama/ptrace_scope
differs from zero, pldd will fail with
/usr/bin/pldd: cannot attach to process 3: Operation not permitted
This patch checks if ptrace_scope exists, is zero "classic ptrace permissions"
or one "restricted ptrace". If ptrace_scope exists and has a higher
restriction, then the test is marked as UNSUPPORTED.
The case "restricted ptrace" is handled by rearranging the processes involved
during the test. Now we have the following process tree:
-parent: do_test (performs output checks)
--subprocess 1: pldd_process (becomes pldd via execve)
---subprocess 2: target_process (ptraced via pldd)
ChangeLog:
* elf/tst-pldd.c (do_test): Add UNSUPPORTED check.
Rearrange subprocesses.
(pldd_process): New function.
* support/Makefile (libsupport-routines): Add support_ptrace.
* support/xptrace.h: New file.
* support/support_ptrace.c: Likewise.
nss_db allows for getpwent et al to be called without a set*ent,
but it only works once. After the last get*ent a set*ent is
required to restart, because the end*ent did not properly reset
the module. Resetting it to NULL allows for a proper restart.
If the database doesn't exist, however, end*ent erroniously called
munmap which set errno.
The test case runs "makedb" inside the testroot, so needs selinux
DSOs installed.
It adds useful functions for tests that use struct timespec.
Checked on x86_64-linux-gnu and i686-linux-gnu.
* support/timespec.h: New file. Provide timespec helper functions
along with macros in the style of those in check.h.
* support/timespec.c: New file. Implement check functions declared
in support/timespec.h.
* support/timespec-add.c: New file from gnulib containing
timespec_add implementation that handles overflow.
* support/timespec-sub.c: New file from gnulib containing
timespec_sub implementation that handles overflow.
* support/README: Mention timespec.h.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* support/xclock_gettime.c (xclock_gettime): New file. Provide
clock_gettime wrapper for use in tests that fails the test rather
than returning failure.
* support/xtime.h: New file to declare xclock_gettime.
* support/Makefile: Add xclock_gettime.c.
* support/README: Mention xtime.h.
Its API is similar to support_capture_subprocess, but rather creates a
new process based on the input path and arguments. Under the hoods it
uses posix_spawn to create the new process.
It also allows the use of other support_capture_* functions to check
for expected results and free the resources.
Checked on x86_64-linux-gnu.
* support/Makefile (libsupport-routines): Add support_subprocess,
xposix_spawn, xposix_spawn_file_actions_addclose, and
xposix_spawn_file_actions_adddup2.
(tst-support_capture_subprocess-ARGS): New rule.
* support/capture_subprocess.h (support_capture_subprogram): New
prototype.
* support/support_capture_subprocess.c (support_capture_subprocess):
Refactor to use support_subprocess and support_capture_poll.
(support_capture_subprogram): New function.
* support/tst-support_capture_subprocess.c (write_mode_to_str,
str_to_write_mode, test_common, parse_int, handle_restart,
do_subprocess, do_subprogram, do_multiple_tests): New functions.
(do_test): Add support_capture_subprogram tests.
* support/subprocess.h: New file.
* support/support_subprocess.c: Likewise.
* support/xposix_spawn.c: Likewise.
* support/xposix_spawn_file_actions_addclose.c: Likewise.
* support/xposix_spawn_file_actions_adddup2.c: Likewise.
* support/xspawn.h: Likewise.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The alignment of TLS variables is wrong if accessed from within a thread
for architectures with tls variant TLS_TCB_AT_TP.
For the main thread the static tls data is properly aligned.
For other threads the alignment depends on the alignment of the thread
pointer as the static tls data is located relative to this pointer.
This patch adds this alignment for TLS_TCB_AT_TP variants in the same way
as it is already done for TLS_DTV_AT_TP. The thread pointer is also already
properly aligned if the user provides its own stack for the new thread.
This patch extends the testcase nptl/tst-tls1.c in order to check the
alignment of the tls variables and it adds a pthread_create invocation
with a user provided stack.
The test itself is migrated from test-skeleton.c to test-driver.c
and the missing support functions xpthread_attr_setstack and xposix_memalign
are added.
ChangeLog:
[BZ #23403]
* nptl/allocatestack.c (allocate_stack): Align pointer pd for
TLS_TCB_AT_TP tls variant.
* nptl/tst-tls1.c: Migrate to support/test-driver.c.
Add alignment checks.
* support/Makefile (libsupport-routines): Add xposix_memalign and
xpthread_setstack.
* support/support.h: Add xposix_memalign.
* support/xthread.h: Add xpthread_attr_setstack.
* support/xposix_memalign.c: New File.
* support/xpthread_attr_setstack.c: Likewise.
For a full analysis of both the pthread_rwlock_tryrdlock() stall
and the pthread_rwlock_trywrlock() stall see:
https://sourceware.org/bugzilla/show_bug.cgi?id=23844#c14
In the pthread_rwlock_trydlock() function we fail to inspect for
PTHREAD_RWLOCK_FUTEX_USED in __wrphase_futex and wake the waiting
readers.
In the pthread_rwlock_trywrlock() function we write 1 to
__wrphase_futex and loose the setting of the PTHREAD_RWLOCK_FUTEX_USED
bit, again failing to wake waiting readers during unlock.
The fix in the case of pthread_rwlock_trydlock() is to check for
PTHREAD_RWLOCK_FUTEX_USED and wake the readers.
The fix in the case of pthread_rwlock_trywrlock() is to only write
1 to __wrphase_futex if we installed the write phase, since all other
readers would be spinning waiting for this step.
We add two new tests, one exercises the stall for
pthread_rwlock_trywrlock() which is easy to exercise, and one exercises
the stall for pthread_rwlock_trydlock() which is harder to exercise.
The pthread_rwlock_trywrlock() test fails consistently without the fix,
and passes after. The pthread_rwlock_tryrdlock() test fails roughly
5-10% of the time without the fix, and passes all the time after.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
Signed-off-by: Torvald Riegel <triegel@redhat.com>
Signed-off-by: Rik Prohaska <prohaska7@gmail.com>
Co-authored-by: Torvald Riegel <triegel@redhat.com>
Co-authored-by: Rik Prohaska <prohaska7@gmail.com>
There is general agreement that the very short list of things that ISO
C says you can do in an async signal handler should all work when the
handler is running on an alternate signal stack with only MINSIGSTKSZ
space. This patch adds tests to make sure those things do work.
To facilitate this, there is a new set of test support routines for
setting up alternate signal stacks; see support/xsignal.h for the API.
* support/xsignal.h (xalloc_sigstack, xfree_sigstack)
(xget_sigstack_location): New test support functions.
* support/xsigstack.c: New file, implementing them.
* support/tst-xsigstack.c: New test for them.
* support/Makefile: Update.
* signal/tst-minsigstksz-1.c
* signal/tst-minsigstksz-2.c
* signal/tst-minsigstksz-3.c
* signal/tst-minsigstksz-3a.c
* signal/tst-minsigstksz-4.c: New tests.
* signal/Makefile: Run them.
