This is the first of a 2-part patch set that fixes slow DSO sorting behavior in
the dynamic loader, as reported in BZ #17645. In order to facilitate such a
large modification to the dynamic loader, this first patch implements a testing
framework for validating shared object sorting behavior, to enable comparison
between old/new sorting algorithms, and any later enhancements.
This testing infrastructure consists of a Python script
scripts/dso-ordering-test.py' which takes in a description language, consisting
of strings that describe a set of link dependency relations between DSOs, and
generates testcase programs and Makefile fragments to automatically test the
described situation, for example:
a->b->c->d # four objects linked one after another
a->[bc]->d;b->c # a depends on b and c, which both depend on d,
# b depends on c (b,c linked to object a in fixed order)
a->b->c;{+a;%a;-a} # a, b, c serially dependent, main program uses
# dlopen/dlsym/dlclose on object a
a->b->c;{}!->[abc] # a, b, c serially dependent; multiple tests generated
# to test all permutations of a, b, c ordering linked
# to main program
(Above is just a short description of what the script can do, more
documentation is in the script comments.)
Two files containing several new tests, elf/dso-sort-tests-[12].def are added,
including test scenarios for BZ #15311 and Redhat issue #1162810 [1].
Due to the nature of dynamic loader tests, where the sorting behavior and test
output occurs before/after main(), generating testcases to use
support/test-driver.c does not suffice to control meaningful timeout for ld.so.
Therefore a new utility program 'support/test-run-command', based on
test-driver.c/support_test_main.c has been added. This does the same testcase
control, but for a program specified through a command-line rather than at the
source code level. This utility is used to run the dynamic loader testcases
generated by dso-ordering-test.py.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1162810
Signed-off-by: Chung-Lin Tang <cltang@codesourcery.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Unlike GCC, Clang parses asm statements and verifies they are valid
instructions/directives. Place the magic @@@ into a comment to avoid
a parse error.
The renaming of installed shared libraries to use the SONAME directly
rather than linking to a versioned name stopped build-many-glibcs.py
--strip (used to facilitate comparing binaries before and after
changes that aren't meant to change any generated code in installed
glibc shared libraries) from stripping most of the installed shared
libraries, because it stripped only the *.so names. Fix it to strip
*.so* names instead and to detect the case of linker scripts using
grep instead of hardcoding particular files that are linker scripts.
Tested with build-many-glibcs.py --strip.
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Since the shared code now has special status with respect to
copyrights, port them into a more structured format in the source tree
and add a python function that parses and returns a dictionary with
the information.
I need this to exclude these files from the Contributed-by changes and
I reckon it would be useful to know these files for future tooling.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The previous approach uses readelf -DWs, which does not produce
a stable output format (older binutils versions do not include
symbol version information). This commit re-uses scripts/abilist.awk
with a tweak to include GLIBC_PRIVATE symbols. This awk script
is based on objdump -T output, which appears to be stable over time.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Some targets have a GLIBC_2.0 baseline for libdl, while using
GLIBC_2.2 for libc. This means that the generated libc.map file
does not have any version nodes for GLIBC_2.0 or GLIBC_2.1. However,
moving symbols from libdl into libc needs such version nodes.
(Future symbol moves from librt will need this as well.)
This kludge is only necessary for symbols predating GLIBC_2.2 because
the affected targets use GLIBC_2.2 as the baseline for libc. Given
the small number and fixed set of affected architectures, no generic
mechanism is implemented, and instead the map file fragment is
hard-coded in scripts/versions.mk.
The compat_symbol macro already emits the appropriate version strings,
so no adjustments are needed there.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This requires that all exported symbol versions are listed in
Versions files. It results in more consistent behavior across
architectures because previously, symbols could be exported
via explicit versioned_symbol and compat_symbol macros if the
version node existed in some Versions file (without listing the
symbol), and it was not the base version for the library (which
already had the local: * directive).
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This generates new macros of this from:
They are useful for symbol lookups using _dl_lookup_direct.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This patch makes build-many-glibcs.py use Linux 5.12 and GCC 11
branch.
Tested with build-many-glibcs.py (host-libraries, compilers and glibcs
builds).
With gdb 10, the pretty printer tests are UNSUPPORTED::
The gdb version string (gdb -v) is incorrectly formatted.
This is observable in:
nptl/test-cond-printers, nptl/test-condattr-printers,
nptl/test-mutex-printers, nptl/test-mutexattr-printers,
nptl/test-rwlock-printers, nptl/test-rwlockattr-printers
After updating the regexp and building with debug-info,
all those tests are passing.
This code adds new flag - '--allow-time-setting' to cross-test-ssh.sh
script to indicate if it is allowed to alter the date on the system
on which tests are executed. This change is supposed to be used with
test systems, which use virtual machines for testing.
The GLIBC_TEST_ALLOW_TIME_SETTING env variable is exported to the
remote environment on which the eligible test is run and brings no
functional change when it is not.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The -O option avoids interleaving (e.g.) compiler error messages
with other build messages, and makes attribution of messages to
commands easier.
According to my tests, the impact on build time is within the noise.
With static pie linking pointers in the tunables list need
RELATIVE relocs since the absolute address is not known at link
time. We want to avoid relocations so the static pie self
relocation can be done after tunables are initialized.
This is a simple fix that embeds the tunable strings into the
tunable list instead of using pointers. It is possible to have
a more compact representation of tunables with some additional
complexity in the generator and tunable parser logic. Such
optimization will be useful if the list of tunables grows.
There is still an issue that tunables_strdup allocates and the
failure handling code path is sufficiently complex that it can
easily have RELATIVE relocations. It is possible to avoid the
early allocation and only change environment variables in a
setuid exe after relocations are processed. But that is a
bigger change and early failure is fatal anyway so it is not
as critical to fix right away. This is bug 27181.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The representation of the tunables including type information and
the tunable list structure are only used in the implementation not
in the tunables api that is exposed to usage within glibc.
This patch moves the representation related definitions into the
existing dl-tunable-types.h and uses that only for implementation.
The tunable callback and related types are moved to dl-tunables.h
because they are part of the tunables api.
This reduces the details exposed in the tunables api so the internals
are easier to change.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
GNU ld and gold have supported --print-output-format since 2011. glibc
requires binutils>=2.25 (2015), so if LD is GNU ld or gold, we can
assume the option is supported.
lld is by default a cross linker supporting multiple targets. It auto
detects the file format and does not need OUTPUT_FORMAT. It does not
support --print-output-format.
By parsing objdump -f, we can support all the three linkers.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
I've updated copyright dates in glibc for 2021. This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files. As well as the usual annual
updates, mainly dates in --version output (minus csu/version.c which
previously had to be handled manually but is now successfully updated
by update-copyrights), there is a small change to the copyright notice
in NEWS which should let NEWS get updated automatically next year.
Please remember to include 2021 in the dates for any new files added
in future (which means updating any existing uncommitted patches you
have that add new files to use the new copyright dates in them).
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
This patch makes build-many-glibcs.py use the recent GMP 6.2.1
release.
Tested with build-many-glibcs.py (host-libraries, compilers and glibcs
builds).
The inner loop is called thousands of times per "make check" even
if there's otherwise nothing to do. Avoid calling /bin/head all
those times when a builtin will do.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Support building three variants of 32-bit RISC-V glibc as follows:
- riscv32-linux-gnu-rv32imac-ilp32
- riscv32-linux-gnu-rv32imafdc-ilp32
- riscv32-linux-gnu-rv32imafdc-ilp32d
Reviewed-by: Maciej W. Rozycki <macro@wdc.com>
ProjectQuirks moved into its own file in gnulib because one cannot
import modules with hyphens in them. Adjust the quirks file to
reflect this reality.
On s390x, gcc inlines more aggresive compared to other architectures.
This occaisionally leads to build warnings / errors.
Therefore this patch adds a s390x glibc variant with optimization.
There is the ccopts field which contain ABI options which are passed
to configure as CC / CXX. Now there is also the cflags field which
contains non-ABI options like -g or -O. Those are passed to configure
as CFLAGS / CXXFLAGS.
Currently CC is passed to conformtest.py or linknamespace.py but not
the CFLAGS.
Since binutils has obsoleted ia64 support, use --enable-obsolete for
now when configuring binutils for ia64 in build-many-glibcs.py (which
requires adding support for architecture-specific binutils configure
options there). As with other obsoletions, the removal of support for
ia64 in any of (binutils, GCC, Linux kernel) should imply its removal
from glibc.
Tested with build-many-glibcs.py for ia64-linux-gnu (compilers and
glibcs build).
Sun RPC was removed from glibc. This includes rpcgen program, librpcsvc,
and Sun RPC headers. Also test for bug #20790 was removed
(test for rpcgen).
Backward compatibility for old programs is kept only for architectures
and ABIs that have been added in or before version 2.28.
libtirpc is mature enough, librpcsvc and rpcgen are provided in
rpcsvc-proto project.
NOTE: libnsl code depends on Sun RPC (installed libnsl headers use
installed Sun RPC headers), thus --enable-obsolete-rpc was a dependency
for --enable-obsolete-nsl (removed in a previous commit).
The arc ABI list file has to be updated because the port was added
with the sunrpc symbols
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
this means that *always* libnsl is only built as shared library for
backward compatibility and the NSS modules libnss_nis and libnss_nisplus
are not built at all, libnsl's headers aren't installed.
This compatibility is kept only for architectures and ABIs that have
been added in or before version 2.28.
Replacement implementations based on TIRPC, which additionally support
IPv6, are available from <https://github.com/thkukuk/>.
This change does not affect libnss_compat which does not depended
on libnsl since 2.27 and thus can be used without NIS.
libnsl code depends on Sun RPC, e.g. on --enable-obsolete-rpc (installed
libnsl headers use installed Sun RPC headers), which will be removed in
the following commit.
The symbol is deprecated by strerror since its usage imposes some issues
such as copy relocations.
Its internal name is also changed to _sys_errlist_internal to avoid
static linking usage. The compat code is also refactored by removing
the over enginered errlist-compat.c generation from manual entried and
extra comment token in linker script file. It disantangle the code
generation from manual and simplify both Linux and Hurd compat code.
The definitions from errlist.c are moved to errlist.h and a new test
is added to avoid a new errno entry without an associated one in manual.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also run a check-abi
on all affected platforms.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
