The conventional configure triplet for ARM GNU/Linux with hard-float
ABI is arm-*-linux-gnueabihf. However, GCC does not automatically use
the hard-float ABI based on that triplet. This patch fixes
build-many-glibcs.py to pass --with-float=hard so that the
arm-linux-gnueabihf configurations actually build with the intended
ABI.
Tested building the affected configurations with build-many-glibcs.py.
* scripts/build-many-glibcs.py (Context.add_all_configs): Use
--with-float=hard for arm-linux-gnueabihf configurations.
There is a configure option --without-fp that specifies that nofpu
sysdeps directories should be used instead of fpu directories.
For most glibc configurations, this option is of no use: either there
is no valid nofpu variant of that configuration, or there are no fpu
or nofpu sysdeps directories for that processor and so the option does
nothing. For a few configurations, if you are using a soft-float
compiler this option is required, and failing to use it generally
results in compilation errors from inline asm using unavailable
floating-point instructions.
We're moving away from --with-cpu to configuring glibc based on how
the compiler generates code, and it is natural to do so for
--without-fp as well; in most cases the soft-float and hard-float ABIs
are incompatible so you have no hope of building a working glibc with
an inappropriately configured compiler or libgcc.
This patch eliminates --without-fp, replacing it entirely by automatic
configuration based on the compiler. Configurations for which this is
relevant (coldfire / mips / powerpc32 / sh) define a variable
with_fp_cond in their preconfigure fragments (under the same
conditions under which those fragments do anything); this is a
preprocessor conditional which the toplevel configure script then uses
in a test to determine which sysdeps directories to use.
The config.make with-fp variable remains. It's used only by powerpc
(sysdeps/powerpc/powerpc32/Makefile) to add -mhard-float to various
flags variables. For powerpc, -mcpu= options can imply use of
soft-float. That could be an issue if you want to build for
e.g. 476fp, but are using --with-cpu=476 because there isn't a 476fp
sysdeps directory. If in future we eliminate --with-cpu and replace
it entirely by testing the compiler, it would be natural at that point
to eliminate that code as well (as the user should then just use a
compiler defaulting to 476fp and the 476 sysdeps directory would be
used automatically).
Tested for x86_64, and tested with build-many-glibcs.py that installed
shared libraries are unchanged by this patch.
* configure.ac (--with-fp): Remove configure option.
(with_fp_cond): New variable.
(libc_cv_with_fp): New configure test. Use this variable instead
of with_fp.
* configure: Regenerated.
* config.make.in (with-fp): Use @libc_cv_with_fp@.
* manual/install.texi (Configuring and compiling): Remove
--without-fp.
* INSTALL: Regenerated.
* sysdeps/m68k/preconfigure (with_fp_cond): Define for ColdFire.
* sysdeps/mips/preconfigure (with_fp_cond): Define.
* sysdeps/powerpc/preconfigure (with_fp_cond): Define for 32-bit.
* sysdeps/sh/preconfigure (with_fp_cond): Define.
* scripts/build-many-glibcs.py (Context.add_all_configs): Do not
use --without-fp to configure glibc.
Since this file is no longer checked in, update-copyrights no longer
needs to avoid changing it.
* scripts/update-copyrights: Do not handle intl/plural.c
specially.
While working on SPARC changes to use libm_alias_* I noticed that the
non-multi-arch sparc32/sparcv9/fpu/s_fabs.S was missing compat symbol
support for fabsl. This clearly shows inadequate test coverage, so
this patch adds SPARC --disable-multi-arch builds to
build-many-glibcs.py (the 32-bit one fails testing until that bug is
fixed, the 64-bit one passes testing).
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
SPARC --disable-multi-arch glibc variants.
This patch adds two extra configuration for arm-linux-gnueabihf to
cover for multiarch support:
1. arm-linux-gnueabihf-v7a: enables multiarch support by using
-march=armv7a.
2. Same as 1. but with --disable-multiarch.
Check with build-many-glibcs.py for both options.
* scripts/build-many-glibcs.py (Context.add_all_configs):
Add arm-linux-gnueabihf multiarch extra_glibcs.
glibc has an add-ons mechanism to allow additional software to be
integrated into the glibc build. Such add-ons may be within the glibc
source tree, or outside it at a path passed to the --enable-add-ons
configure option.
localedata and crypt were once add-ons, distributed in separate
release tarballs, but long since stopped using that mechanism.
Linuxthreads was always an add-on. Ports spent some time as an add-on
with separate release tarballs, then was first moved into the glibc
source tree, then had its sysdeps files moved into the main sysdeps
hierarchy so the add-ons mechanism was no longer used. NPTL spent
some time as an add-on in the main glibc tree before stopping using
the add-on mechanism. libidn used to have separate release tarballs
but no longer does so, but still uses the add-ons mechanism within the
glibc source tree. Various other software has supported building with
the add-ons mechanism at times in the past, but I don't think any is
still widely used.
Add-ons involve significant, little-used complexity in the glibc build
system, and make it hard to understand what the space of possible
glibc configurations is. This patch removes the add-ons mechanism.
libidn is now built via the Subdirs mechanism to cause any
configuration using sysdeps/unix/inet to build libidn; HAVE_LIBIDN
(which effectively means shared libraries are available) is now
defined via sysdeps/unix/inet/configure. Various references to
add-ons around the source tree are removed (in the case of maint.texi,
the example list of sysdeps directories is still very out of date).
Externally maintained ports should now put their files in the normal
sysdeps directory structure rather than being arranged as add-ons;
they probably need to change e.g. elf.h anyway, rather than actually
being able to work just as a drop-in subtree. Hurd libpthread should
be arranged similarly to NPTL, so some files might go in a
hurd-pthreads (or similar) top-level directory in glibc, while sysdeps
files should go in the normal sysdeps directory structure (possibly in
hurd or hurd-pthreads subdirectories, just as there are nptl
subdirectories in the sysdeps tree).
Tested for x86_64, and with build-many-glibcs.py.
* configure.ac (--enable-add-ons): Remove option.
(machine): Do not mention add-ons in comment.
(LIBC_PRECONFIGURE): Likewise.
(add_ons): Remove variable and sanity checks and logic to locate
add-ons.
(add_ons_automatic): Remove variable.
(configured_add_ons): Likewise.
(add_ons_sfx): Likewise.
(add_ons_pfx): Likewise.
(add_on_subdirs): Likewise.
(sysnames_add_ons): Likewise. Remove loop over add-ons and
consideration of add-ons in Implies handling.
(sysdeps_add_ons): Likewise.
* configure: Regenerated.
* libidn/configure.ac: Remove.
* libidn/configure: Likewise.
* sysdeps/unix/inet/configure.ac: New file.
* sysdeps/unix/inet/configure: New generated file.
* sysdeps/unix/inet/Subdirs: Add libidn.
* Makeconfig (sysdeps-srcdirs): Remove variable.
(+sysdep_dirs): Do not include $(sysdeps-srcdirs).
($(common-objpfx)config.status): Do not depend on add-on files.
($(common-objpfx)shlib-versions.v.i): Do not mention add-ons in
comment.
(all-subdirs): Do not include $(add-on-subdirs).
* Makefile (dist-prepare): Do not use $(sysdeps-add-ons).
* config.make.in (add-ons): Remove variable.
(add-on-subdirs): Likewise.
(sysdeps-add-ons): Likewise.
* manual/Makefile (add-chapters): Remove.
($(objpfx)texis): Do not depend on $(add-chapters).
(nonexamples): Do not handle $(add-chapters).
(examples): Do not handle $(add-ons).
(chapters.% top-menu.%): Do not pass '$(add-chapters)' to
libc-texinfo.sh.
* manual/install.texi (Installation): Do not mention add-ons.
(--enable-add-ons): Do not document configure option.
* INSTALL: Regenerated.
* manual/libc-texinfo.sh: Do not handle $2 add-ons argument.
* manual/maint.texi (Hierarchy Conventions): Do not mention
add-ons.
* scripts/build-many-glibcs.py (Glibc.build_glibc): Do not use
--enable-add-ons.
* scripts/gen-sorted.awk: Do not handle Subdirs files from
add-ons.
* scripts/test-installation.pl: Do not handle glibc-compat add-on.
* sysdeps/nptl/Makeconfig: Do not mention add-ons in comment.
When configuring and building GNU libc using the Mozilla NSS library
for cryptography (--enable-nss-crypt option), also include the
NSPR header files along with the Mozilla NSS library header files.
Finally, when running the check-local-headers test, ignore the
Mozilla NSPR library header files (used by the Mozilla NSS library).
Current implementation of tunables does not set arena_max and arena_test
values. Any value provided by glibc.malloc.arena_max and
glibc.malloc.arena_test parameters is ignored.
These tunables have minval value set to 1 (see elf/dl-tunables.list file)
and undefined maxval value. In that case default value (which is 0. see
scripts/gen-tunables.awk) is being used to set maxval.
For instance, generated tunable_list[] entry for arena_max is:
(gdb) p *cur
$1 = {name = 0x7ffff7df6217 "glibc.malloc.arena_max",
type = {type_code = TUNABLE_TYPE_SIZE_T, min = 1, max = 0},
val = {numval = 0, strval = 0x0}, initialized = false,
security_level = TUNABLE_SECLEVEL_SXID_IGNORE,
env_alias = 0x7ffff7df622e "MALLOC_ARENA_MAX"}
As a result, any value of glibc.malloc.arena_max is ignored by
TUNABLE_SET_VAL_IF_VALID_RANGE macro
__type min = (__cur)->type.min; <- initialized to 1
__type max = (__cur)->type.max; <- initialized to 0!
if (min == max) <- false
{
min = __default_min;
max = __default_max;
}
if ((__type) (__val) >= min && (__type) (val) <= max) <- false
{
(__cur)->val.numval = val;
(__cur)->initialized = true;
}
Assigning correct min/max values at a build time fixes a problem.
Plus, a bit of optimization: Setting of default min/max values for the
given type at a run time might be eliminated.
* elf/dl-tunables.c (do_tunable_update_val): Range checking fix.
* scripts/gen-tunables.awk: Set unspecified minval and/or maxval
values to correct default value for given type.
This patch updates build-many-glibcs.py to use the current release
branch of binutils and current releases of GMP and the Linux kernel.
* scripts/build-many-glibcs.py (Context.checkout): Default
binutils version to 2.29 branch, GMP version to 6.1.2 and Linux
kernel version to 4.12.
Awk 3.1.x does not support multi-dimensional arrays, so fix up to make
sure that gen-tunables.awk works on it.
* scripts/gen-tunables.awk: Avoid multi-dimensional arrays.
Add LD_HWCAP_MASK to tunables in preparation of it being removed from
rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it
can influence IFUNC resolution in aarch64.
This patch does not actually do anything other than read the
LD_HWCAP_MASK variable and add the tunables way to set the
LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a
follow-up patch, the _dl_hwcap_mask will be replaced with
glibc.tune.hwcap_mask to complete the transition.
* elf/dl-tunables.list: Add glibc.tune.hwcap_mask.
* scripts/gen-tunables.awk: Include dl-procinfo.h.
* manual/tunables.texi: Document glibc.tune.hwcap_mask.
This patch makes the glibc build generate an additional header
ldbl-compat-choose.h that defines LONG_DOUBLE_COMPAT_CHOOSE_* macros
for each libc and libm symbol, which select one or the other of their
arguments based on whether the symbol was added before a change to
long double != double.
The effect of this is that it is then possible to define a macro
maybe_long_double_symbol that automatically acts as either
long_double_symbol or weak_alias depending on when the symbol being
defined was added. This can be used when building long double
functions from type-generic templates. Thus, with this patch ldbl-opt
no longer needs special long double implementations of each new libm
function added using such a template, and the existing such
implementations are removed.
This is a step towards being able more generally to use common macros
to create all the aliases needed for a libm function, so reducing the
amount of special-case code needed in ldbl-opt and ldbl-64-128, and
facilitating subsequently adding *f32 / *f64 / *f128 / *f32x / *f64x
aliases to existing functions (where the set of aliases that a
function should have may depend on the architecture in various ways).
Tested with build-many-glibcs.py. Except for on
powerpc64le-linux-gnu, installed stripped shared libraries are
unchanged by the patch. powerpc64le-linux-gnu is the unique
configuration which used ldbl-opt from the start rather than adding a
new long double choice after originally only having had long double =
double. The effect of the patch there is that various cases that
previously used long_double_symbol unconditionally now use weak_alias
instead, so .os files contain e.g. a symbol cabsl instead of
cabsl@@GLIBC_2.17. The final dynamic symbols and versions in the
resulting shared libraries are unchanged (ABI tests pass), as is the
disassembly of the shared libraries, but the differences in the .os
files still result in different .gnu_hash contents in libm.so; the
differences are of no significance and logically using weak_alias is
what's most appropriate in those cases.
* scripts/versions.awk: Generate ldbl-compat-choose.h.
* sysdeps/ieee754/ldbl-opt/math-type-macros-ldouble.h: Include
<ldbl-compat-choose.h>.
(maybe_long_double_symbol): New macro.
[!declare_mgen_alias] (declare_mgen_alias): Use
maybe_long_double_symbol.
* sysdeps/ieee754/ldbl-opt/s_canonicalizel.c: Remove.
* sysdeps/ieee754/ldbl-opt/s_fmaxmagl.c: Likewise.
* sysdeps/ieee754/ldbl-opt/s_fminmagl.c: Likewise.
* sysdeps/ieee754/ldbl-opt/s_nextdownl.c: Likewise.
* sysdeps/ieee754/ldbl-opt/w_llogbl.c: Likewise.
* Makerules [$(build-shared) = yes && !avoid-generated]
(before-compile): Add $(common-objpfx)ldbl-compat-choose.h.
[$(build-shared) = yes && !avoid-generated]
($(common-objpfx)ldbl-compat-choose.h): New target.
This patch arranges for the glibc build to generate a header
first-versions.h that defines macros for the earliest symbol version
in which each public symbol (GLIBC_[0-9]* symbol version, name only
uses C identifier characters) is available.
This is used in sysdeps/ieee754/ldbl-opt/math-type-macros-double.h to
replace the manually defined LDOUBLE_*_libm_version macros for various
functions defined using type-generic templates, the purpose of which
is to use in LONG_DOUBLE_COMPAT tests "was this function originally
added before glibc supported long double != double on this platform?".
As discussed in
<https://sourceware.org/ml/libc-alpha/2016-12/msg00246.html>, I expect
this to be useful more generally in reducing the amount of
special-case code needed in ldbl-opt and ldbl-64-128.
Tested with build-many-glibcs.py that installed stripped shared
libraries are unchanged by this patch.
* scripts/versions.awk: Generate first-versions.h.
* sysdeps/ieee754/ldbl-opt/math-type-macros-double.h: Include
<first-versions.h>.
(LDOUBLE_cabsl_libm_version): Remove macro.
(LDOUBLE_cargl_libm_version): Likewise.
(LDOUBLE_cimagl_libm_version): Likewise.
(LDOUBLE_conjl_libm_version): Likewise.
(LDOUBLE_creall_libm_version): Likewise.
(LDOUBLE_cacosl_libm_version): Likewise.
(LDOUBLE_cacoshl_libm_version): Likewise.
(LDOUBLE_ccosl_libm_version): Likewise.
(LDOUBLE_ccoshl_libm_version): Likewise.
(LDOUBLE_casinl_libm_version): Likewise.
(LDOUBLE_csinl_libm_version): Likewise.
(LDOUBLE_casinhl_libm_version): Likewise.
(LDOUBLE_csinhl_libm_version): Likewise.
(LDOUBLE_catanl_libm_version): Likewise.
(LDOUBLE_catanhl_libm_version): Likewise.
(LDOUBLE_ctanl_libm_version): Likewise.
(LDOUBLE_ctanhl_libm_version): Likewise.
(LDOUBLE_cexpl_libm_version): Likewise.
(LDOUBLE_clogl_libm_version): Likewise.
(LDOUBLE_cprojl_libm_version): Likewise.
(LDOUBLE_csqrtl_libm_version): Likewise.
(LDOUBLE_cpowl_libm_version): Likewise.
(LDOUBLE_clog10l_libm_version): Likewise.
(LDOUBLE___clog10l_libm_version): Likewise.
(LDOUBLE_fdiml_libm_version): Likewise.
(LDOUBLE_fmaxl_libm_version): Likewise.
(LDOUBLE_fminl_libm_version): Likewise.
(LDOUBLE_ilogbl_libm_version): Likewise.
(LDOUBLE_nanl_libm_version): Likewise.
[!M_LIBM_NEED_COMPAT] (M_LIBM_NEED_COMPAT): Use
FIRST_VERSION_libm_* macros.
[!declare_mgen_libm_compat] (declare_mgen_libm_compat): Likewise.
* Makerules [$(build-shared) = yes && !avoid-generated]
(before-compile): Add $(common-objpfx)first-versions.h.
[$(build-shared) = yes && !avoid-generated]
($(common-objpfx)first-versions.h): New target.
($(common-objpfx)sysd-versions): Depend on and change to rule for
building $(common-objpfx)versions.stmp.
Enhance dl-tunables.list to allow specifying a default value for a
tunable that it would be initialized to.
* scripts/gen-tunables.awk: Recognize 'default' keyword in
dl-tunables.list.
* README.tunables: Document it.
As of now we don't need tunables to be writable after they have been
set initially, so there is no point in having them writable. Put
tunable_list in .data.rel.ro so that it is set as read-only after
relocation. This also allows us to move some of the dl_* variables
that are tunables controlled into the tunables infrastructure instead
of having two copies.
In future if we ever need specific tunables to be writable at runtime,
we can split the tunable_list into two.
Regression tested on x86_64 to verify that tests continue to pass.
* scripts/gen-tunables.awk: Add attribute_relro to
tunable_list.
Now there is a GCC 7 release branch, this patch makes
build-many-glibcs.py default to using it instead of GCC 6.
Relative to GCC 6, this fixes the MicroBlaze build but introduces ICEs
building glibc testcases for SH (GCC PRs 78459, 78460; the latter is
an out-of-memory issue so you may want to set memory limits when
running build-many-glibcs.py).
* scripts/build-many-glibcs.py (Context.checkout): Default gcc
version to 7 branch.
Since commit 8b9e9c3c0b, security_level replaces
is_secure. There were some old files need to be updated.
2017-03-23 Sunyeop Lee <sunyeop97@gmail.com>
* README.tunables: Updated descriptions.
* elf/dl-tunables.list: Fixed typo: SXID_NONE -> NONE.
* scripts/gen-tunables.awk: Updated the code related to the commit.
glibc headers include some code (not particularly consistent or
systematic) to put various declarations in C++ namespaces std and
__c99, if _GLIBCPP_USE_NAMESPACES is defined.
As noted in <https://gcc.gnu.org/ml/libstdc++/2017-03/msg00025.html>,
this macro was removed from libstdc++ in 2000. I don't expect
compilation with such old versions of libstdc++ to work with current
glibc headers anyway (whereas old *binaries* are expected to stay
working with current glibc); this patch (which should be a no-op with
any libstdc++ version postdating that removal) removes all this code
from the glibc headers.
The begin-end-check.pl test, whose comments say it is about checking
these namespace macro calls, is also removed. The code in that test
would have covered __BEGIN_DECLS / __END_DECLS as well, but if those
weren't properly matched it would show up with the
check-installed-headers-cxx tests, so I don't think there is an actual
use for keeping begin-end-check.pl with the namespace code removed.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
* misc/sys/cdefs.h (__BEGIN_NAMESPACE_STD): Remove macro.
(__END_NAMESPACE_STD): Likewise.
(__USING_NAMESPACE_STD): Likewise.
(__BEGIN_NAMESPACE_C99): Likewise.
(__END_NAMESPACE_C99): Likewise.
(__USING_NAMESPACE_C99): Likewise.
* math/math.h (_Mdouble_BEGIN_NAMESPACE): Do not define and
undefine macro.
(_Mdouble_END_NAMESPACE): Likewise.
* ctype/ctype.h: Do not handle C++ namespaces.
* libio/bits/stdio-ldbl.h: Likewise.
* libio/stdio.h: Likewise.
* locale/locale.h: Likewise.
* math/bits/mathcalls.h: Likewise.
* setjmp/setjmp.h: Likewise.
* signal/signal.h: Likewise.
* stdlib/bits/stdlib-float.h: Likewise.
* stdlib/bits/stdlib-ldbl.h: Likewise.
* stdlib/stdlib.h: Likewise.
* string/string.h: Likewise.
* sysdeps/x86/fpu/bits/mathinline.h: Likewise.
* time/bits/types/clock_t.h: Likewise.
* time/bits/types/struct_tm.h: Likewise.
* time/bits/types/time_t.h: Likewise.
* time/time.h: Likewise.
* wcsmbs/bits/wchar-ldbl.h: Likewise.
* wcsmbs/uchar.h: Likewise.
* wcsmbs/wchar.h: Likewise.
[_GLIBCPP_USE_NAMESPACES] (wint_t): Remove conditional definition.
* wctype/wctype.h: Do not handle C++ namespaces.
* scripts/begin-end-check.pl: Remove.
* Makefile (installed-headers): Likewise.
(tests-special): Do not add $(objpfx)begin-end-check.out.
($(objpfx)begin-end-check.out): Remove.
Now with d40dbe7 SH build does not require more the no_isolate gcc
options to correct build glibc (since SH build now does not generate
a trap anymore). This patch removes the unrequired options from
SH config.
Checked with a build for sh3-linux-gnu, sh3eb-linux-gnu, sh4-linux-gnu,
and sh4eb-linux-gnu.
* scripts/build-many-glibcs.py (Context.add_all_configs): Remove
no_isolate usage for SH.
The 'bot-cycle' action for build-many-glibcs is a convenient way to
not have to remember all the steps in keeping a many-glibcs tree up
to date ... or it would be, if the script could send mail _optionally_.
Make it so by skipping the mail step if mail isn't configured.
* scripts/build-many-glibcs.py (bot_build_mail): If the
bot_config does not contain all of the necessary email-
related settings, just print a warning and continue.
Florian Weimer pointed out that we have three different kinds of
environment variables (and hence tunables):
1. Variables that are removed for setxid processes
2. Variables that are ignored in setxid processes but is passed on to
child processes
3. Variables that are passed on to child processes all the time
Tunables currently only does (2) and (3) when it should be doing (1)
for MALLOC_CHECK_. This patch enhances the is_secure flag in tunables
to an enum value that can specify which of the above three categories
the tunable (and its envvar alias) belongs to.
The default is for tunables to be in (1). Hence, all of the malloc
tunables barring MALLOC_CHECK_ are explicitly specified to belong to
category (2). There were discussions around abolishing category (2)
completely but we can do that as a separate exercise in 2.26.
Tested on x86_64 to verify that there are no regressions.
[BZ #21073]
* elf/dl-tunable-types.h (tunable_seclevel_t): New enum.
* elf/dl-tunables.c (tunables_strdup): Remove.
(get_next_env): Also return the previous envp.
(parse_tunables): Erase tunables of category
TUNABLES_SECLEVEL_SXID_ERASE.
(maybe_enable_malloc_check): Make MALLOC_CHECK_
TUNABLE_SECLEVEL_NONE if /etc/setuid-debug is accessible.
(__tunables_init)[TUNABLES_FRONTEND ==
TUNABLES_FRONTEND_valstring]: Update GLIBC_TUNABLES envvar
after parsing.
[TUNABLES_FRONTEND != TUNABLES_FRONTEND_valstring]: Erase
tunable envvars of category TUNABLES_SECLEVEL_SXID_ERASE.
* elf/dl-tunables.h (struct _tunable): Change member is_secure
to security_level.
* elf/dl-tunables.list: Add security_level annotations for all
tunables.
* scripts/gen-tunables.awk: Recognize and generate enum values
for security_level.
* elf/tst-env-setuid.c: New test case.
* elf/tst-env-setuid-tunables: new test case.
* elf/Makefile (tests-static): Add them.
A recent build-many-glibcs.py build
<https://sourceware.org/ml/libc-testresults/2017-q1/msg00067.html> ran
into what proves to be an old known bug
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=42980> with parallel
install of GCC (one which as discussed there might require automake
changes to fix). This patch makes build-many-glibcs.py avoid such
intermittent failures from parallel install by using -j1 for GCC make
install (the code in question also applies to binutils make install,
but it doesn't seem worth trying to avoid -j1 there; the builds and
installs of different toolchains are still fully parallel with each
other, this is only about the case when there are few enough of those
that multiple jobs can get used within a single make install).
* scripts/build-many-glibcs.py (Config.build_cross_tool): Use -j1
for make install.
I used this patch to run the new build script with python3.2, it may be worth
adding this hack if python3.5 is not widespread (might work with older python,
i haven't tested that).
This patch make build-many-glibcs.py work with python 3.2 by
adding fallback implementation to python 3.5 facilities if they
are not present.
Checked building a x86_64-linux-gnu toolchain with python 3.2.
2016-11-22 Szabolcs Nagy <szabolcs.nagy@arm.com>
* scripts/build-many-glibcs.py (os.cpu_count): Add compatibility definition.
(re.fullmatch, subprocess.run): Likewise.
The 32-bit powerpc configurations in build-many-glibcs.py were failing
to cover the powerpc32 multiarch code at all, because that code is
only built for power4 and above configurations. This patch adds a
32-bit power4 configuration so that at least some of that multiarch
code gets build-tested. (This is preparation for reviewing the w_*
file renaming, which affects such powerpc32 multiarch files.)
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
power4 glibc for powerpc-linux-gnu.
Now that a release branch exists for binutils 2.28, this patch makes
build-many-glibcs.py use that by default in place of 2.27.
* scripts/build-many-glibcs.py (Context.checkout): Default
binutils version to 2.28 branch.
The tunables framework allows us to uniformly manage and expose global
variables inside glibc as switches to users. tunables/README has
instructions for glibc developers to add new tunables.
Tunables support can be enabled by passing the --enable-tunables
configure flag to the configure script. This patch only adds a
framework and does not pose any limitations on how tunable values are
read from the user. It also adds environment variables used in malloc
behaviour tweaking to the tunables framework as a PoC of the
compatibility interface.
* manual/install.texi: Add --enable-tunables option.
* INSTALL: Regenerate.
* README.tunables: New file.
* Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
(before-compile): Generate dl-tunable-list.h early.
* config.h.in: Add HAVE_TUNABLES.
* config.make.in: Add have-tunables.
* configure.ac: Add --enable-tunables option.
* configure: Regenerate.
* csu/init-first.c (__libc_init_first): Move
__libc_init_secure earlier...
* csu/init-first.c (LIBC_START_MAIN):... to here.
Include dl-tunables.h, libc-internal.h.
(LIBC_START_MAIN) [!SHARED]: Initialize tunables for static
binaries.
* elf/Makefile (dl-routines): Add dl-tunables.
* elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE
namespace.
* elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_
only when !HAVE_TUNABLES.
* elf/rtld.c (process_envvars): Likewise.
* elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h
(_dl_sysdep_start): Call __tunables_init.
* elf/dl-tunable-types.h: New file.
* elf/dl-tunables.c: New file.
* elf/dl-tunables.h: New file.
* elf/dl-tunables.list: New file.
* malloc/tst-malloc-usable-static.c: New test case.
* malloc/Makefile (tests-static): Add it.
* malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h.
Define TUNABLE_NAMESPACE.
(DL_TUNABLE_CALLBACK (set_mallopt_check)): New function.
(DL_TUNABLE_CALLBACK_FNDECL): New macro. Use it to define
callback functions.
(ptmalloc_init): Set tunable values.
* scripts/gen-tunables.awk: New file.
* sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h.
(_dl_sysdep_start): Call __tunables_init.
If GDB prints an error message for a "python" command, include
that error message in the test log output, to simplify diagnosing
GDB/Python detection issues.
This patch updates texinfo.tex and various miscellaneous scripts to
their latest upstream versions. (There may be another update in early
January to bring in 2017 copyright dates, if the upstream versions get
updated with such dates promptly.)
Tested for x86_64.
* manual/texinfo.tex: Update to version 2016-09-18.18 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2016-10-02.
* scripts/config.sub: Update to version 2016-11-19.
* scripts/install-sh: Update to version 2016-01-11.22.
* scripts/mkinstalldirs: Update to version 2016-01-11.22.
* scripts/move-if-change: Update to version 2016-01-11 22:04.
When build-many-glibcs.py re-execs itself with execv, any buffered
output on stdout may be lost (in particular, messages intended to go
to a bot's log about the re-exec taking place). This patch makes it
flush stdout before execv, similar to the flush before running a
subprocess from the bot that is done to ensure output appears in the
right order.
* scripts/build-many-glibcs.py (Context.exec_self): Flush stdout
before calling execv.
This patch updates build-many-glibcs.py to use Linux 4.9 for kernel
headers unless another version is explicitly specified. Note that
when a version changes like this you'll need to use --replace-sources
when updating an existing checkout to tell build-many-glibcs.py it's
OK to delete and replace the sources of a component for which the
version used has changed.
* scripts/build-many-glibcs.py (Context.checkout): Default Linux
kernel version to 4.9.
For many years, the only effect of these macros has been to make
unistd.h declare getlogin_r. _POSIX_C_SOURCE >= 199506L also causes
this function to be declared. However, people who don't carefully
read all the headers might be confused into thinking they need to
define _REENTRANT for any threaded code (as was indeed the case a long
time ago).
Therefore, remove __USE_REENTRANT, and make _REENTRANT and _THREAD_SAFE
into synonyms for _POSIX_C_SOURCE=199506L. This will only affect
programs that don't select a higher conformance level some other way.
For instance, -std=c89 -D_REENTRANT will see a change in visible
declarations, but -std=c99 -D_POSIX_C_SOURCE=200809L -D_REENTRANT won't,
and -D_REENTRANT all by itself also won't, because _DEFAULT_SOURCE
implies _POSIX_C_SOURCE > 199506.
* include/features.h: Remove __USE_REENTRANT. Treat _REENTRANT
and _THREAD_SAFE the same as _POSIX_C_SOURCE=199506L, if a higher
POSIX conformance level has not been selected by other macros.
* NEWS, manual/creature.texi: Document this change.
* posix/unistd.h, posix/bits/unistd.h: Don't check __USE_REENTRANT.
* include/libc-symbols.h: Don't define _REENTRANT.
* scripts/check-installed-headers.sh: Don't undefine _REENTRANT.
This patch adds pretty printers for the following NPTL types:
- pthread_mutex_t
- pthread_mutexattr_t
- pthread_cond_t
- pthread_condattr_t
- pthread_rwlock_t
- pthread_rwlockattr_t
To load the pretty printers into your gdb session, do the following:
python
import sys
sys.path.insert(0, '/path/to/glibc/build/nptl/pretty-printers')
end
source /path/to/glibc/source/pretty-printers/nptl-printers.py
You can check which printers are registered and enabled by issuing the
'info pretty-printer' gdb command. Printers should trigger automatically when
trying to print a variable of one of the types mentioned above.
The printers are architecture-independent, and were tested on an AMD64 running
Ubuntu 14.04 and an x86 VM running Fedora 24.
In order to work, the printers need to know the values of various flags that
are scattered throughout pthread.h and pthreadP.h as enums and #defines. Since
replicating these constants in the printers file itself would create a
maintenance burden, I wrote a script called gen-py-const.awk that Makerules uses
to extract the constants. This script is pretty much the same as gen-as-const.awk,
except it doesn't cast the constant values to 'long' and is thorougly documented.
The constants need only to be enumerated in a .pysym file, which is then referenced
by a Make variable called gen-py-const-headers.
As for the install directory, I discussed this with Mike Frysinger and Siddhesh
Poyarekar, and we agreed that it can be handled in a separate patch, and shouldn't
block merging of this one.
In addition, I've written a series of test cases for the pretty printers.
Each lock type (mutex, condvar and rwlock) has two test programs, one for itself
and other for its related 'attributes' object. Each test program in turn has a
PExpect-based Python script that drives gdb and compares its output to the
expected printer's. The tests run on the glibc host, which is assumed to have
both gdb and PExpect; if either is absent the tests will fail with code 77
(UNSUPPORTED). For cross-testing you should use cross-test-ssh.sh as test-wrapper.
I've tested the printers on both native builds and a cross build using a Beaglebone
Black running Debian, with the build system's filesystem shared with the board
through NFS.
Finally, I've written a README that explains all this and more.
* INSTALL: Regenerated.
* Makeconfig: Add comments and whitespace to make the control flow
clearer.
(+link-printers-tests, +link-pie-printers-tests, CFLAGS-printers-tests,
installed-rtld-LDFLAGS, built-rtld-LDFLAGS, link-libc-rpath,
link-libc-tests-after-rpath-link, link-libc-printers-tests): New.
(rtld-LDFLAGS, rtld-tests-LDFLAGS, link-libc-tests-rpath-link,
link-libc-tests): Use the new variables as required.
* Makerules ($(py-const)): New rule.
generated: Add $(py-const).
* README.pretty-printers: New file.
* Rules (tests-printers-programs, tests-printers-out, py-env): New.
(others): Depend on $(py-const).
(tests): Depend on $(tests-printers-programs) or $(tests-printers-out),
as required. Pass $(tests-printers) to merge-test-results.sh.
* manual/install.texi: Add requirements for testing the pretty printers.
* nptl/Makefile (gen-py-const-headers, pretty-printers, tests-printers,
CFLAGS-test-mutexattr-printers.c CFLAGS-test-mutex-printers.c,
CFLAGS-test-condattr-printers.c, CFLAGS-test-cond-printers.c,
CFLAGS-test-rwlockattr-printers.c CFLAGS-test-rwlock-printers.c,
tests-printers-libs): Define.
* nptl/nptl-printers.py: New file.
* nptl/nptl_lock_constants.pysym: Likewise.
* nptl/test-cond-printers.c: Likewise.
* nptl/test-cond-printers.py: Likewise.
* nptl/test-condattr-printers.c: Likewise.
* nptl/test-condattr-printers.py: Likewise.
* nptl/test-mutex-printers.c: Likewise.
* nptl/test-mutex-printers.py: Likewise.
* nptl/test-mutexattr-printers.c: Likewise.
* nptl/test-mutexattr-printers.py: Likewise.
* nptl/test-rwlock-printers.c: Likewise.
* nptl/test-rwlock-printers.py: Likewise.
* nptl/test-rwlockattr-printers.c: Likewise.
* nptl/test-rwlockattr-printers.py: Likewise.
* scripts/gen-py-const.awk: Likewise.
* scripts/test_printers_common.py: Likewise.
* scripts/test_printers_exceptions.py: Likewise.
This patch adds a --strip option to build-many-glibcs.py, to make it
strip the installed shared libraries after installation. This is for
convenience if you want to compare installed stripped shared libraries
before and after a patch that was not meant to result in any code
changes: you can run with this option, copy the install/glibcs
directory, run again with the patch and compare the */lib*
subdirectory contents.
(It might make sense for the option to strip libraries in other
directories, including stripping debug information from static
libraries, with a view to making it possible for a
no-generated-code-changes patch to result in completely identical
install/glibcs directories, so simplifying comparison, though that may
need other build determinism changes, e.g. to build deterministic .a
files.)
* scripts/build-many-glibcs.py (Context.__init__): Take strip
argument.
(Glibc.build_glibc): Strip installed shared libraries if
requested.
(get_parser): Add --strip option.
(main): Update Context call.
This patch makes build-many-glibcs.py support a "bot" action, for
repeatedly running a checkout and build cycle.
Two new configuration variables are used in bot-config.json. "delay"
indicates the time to sleep after each bot-cycle round (regardless of
whether that round actually ran any builds); "run" is a boolean, which
is false if the bot should just exit (the point of this is that you
can edit bot-config.json to set this to false to cause a running bot
to exit cleanly between builds) and true if the bot should run. The
bot does not exit if the bot-cycle process exits with error status
(that can occur when sourceware's load limiting means anonymous
version control access fails, for example), just sleeps until it's
time to try again.
The script is changed to flush stdout before running a subprocess in
bot-cycle, so that when output is redirected (as expected for a bot)
the status messages from bot-cycle appear in their proper position in
its redirected output relative to the output from the subprocesses
run, and to copy the logs directory before running builds in bot-cycle
so that the logs from at least one complete build are always available
for looking at how something failed, even while the next build is
running.
* scripts/build-many-glibcs.py: Add bot to usage message. Import
time module.
(Context.__init__): Initialize self.logsdir_old.
(Context.run_builds): Handle bot action.
(Context.bot_cycle): Copy logs directory before running builds.
(Context.bot_run_self): Take argument for whether to check
subprocess result. Flush stdout before running subprocess.
(Context.bot): New function.
(get_parser): Allow bot action.
This patch continues the process of setting up build-many-glibcs.py to
run as a bot monitoring for and reporting on build issues by adding a
bot-cycle action to the script. When this action is used, it will run
the checkout action (re-execing itself if it was changed by that
action), then rebuild whichever of host-libraries, compilers, glibcs
should be rebuilt based on changed versions, time elapsed and state of
previous builds. Email is sent with the results of the build (for
each build action done).
The rebuild logic is: if previous build time or versions aren't
recorded, rebuild that component. If the script has changed, rebuild
everything. If any relevant component version has changed, rebuild,
except for not rebuilding compilers if the time indicated in the bot
configuration has not passed since the last build of the compilers.
If one piece is rebuilt then rebuild subsequent pieces as well.
Using bot-cycle requires a configuration file bot-config.json in the
toplevel directory used by build-many-glibcs.py. It might contain
e.g.
{
"compilers-rebuild-delay": 604800,
"email-from": "Example Name <user@example.org>",
"email-server": "localhost",
"email-subject": "GCC 6 %(action)s %(build-time)s build results",
"email-to": "libc-testresults@sourceware.org"
}
My next intended step is adding a further action "bot" which loops
running bot-cycle then sleeping for an amount of time given in
bot-config.json. Then I'll set up a bot using that action (building
with GCC 6 branch; a bot using GCC mainline may wait until the SH
out-of-memory issues
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78460> are fixed; I
expect the bot to mail to me until it seems ready to switch to mailing
to gcc-testresults).
* scripts/build-many-glibcs.py: Add bot-cycle to usage message.
Import email.mime.text, email.utils and smtplib modules.
(Context.__init__): Initialize self.bot_config_json.
(Context.run_builds): Handle bot-cycle action.
(Context.load_bot_config_json): New function.
(Context.part_build_old): Likewise.
(Context.bot_cycle): Likewise.
(Context.bot_build_mail): Likewise.
(Context.bot_run_self): Likewise.
(get_parser): Allow bot-cycle action.
This patch makes build-many-glibcs.py store information about builds
in JSON format. This is part of preparing it for use in a bot
checking for regressions.
The information stored is: time of last build (of host-libraries,
compilers or glibcs); versions of components used in the last build
(for compilers, host library versions are properly copied from those
used for the previous host-libraries build, and for glibcs, component
versions other than that of glibc are similarly copied from the last
compilers build); PASS/FAIL/UNRESOLVED results of the individual build
steps; a list of changed results; a list of tests (that are still run
at all) that have ever been recorded to PASS.
The first pieces of information are intended to be used by a bot to
decide whether a rebuild is appropriate (based on some combination of
elapsed time and changes to versions; a bot might want to rebuild
glibcs if there had been any change but only rebuild compilers after
enough time had elapsed, for example). All the information is
intended to be used in generating mails with results information.
This state is specifically for full builds (no individual configs for
building compilers or glibcs specified). If individual configs are
specified, build-time and build-versions information is cleared (since
it will no longer accurately reflect the install directory contents),
while the other information is left unchanged. This reflects the
motivation of providing information for a bot checking for
regressions; the contents of build-state.json in a tree used for
manual builds that may be only for some configurations are not
particularly important.
* scripts/build-many-glibcs.py: Import datetime module.
(Context.__init__): Load JSON build state. Initialize list of
status logs.
(Context.run_builds): Update saved build state.
(Context.add_makefile_cmdlist): Update list of status logs.
(Context.load_build_state_json): New function.
(Context.store_build_state_json): Likewise.
(Context.clear_last_build_state): Likewise.
(Context.update_build_state): Likewise.
(CommandList.status_logs): Likewise.
Updating build-many-glibcs.py may result in changes to the default
versions of components, or to the set of components (if e.g. Hurd
support is added and that requires a new component).
It's desirable for the checkout process to leave a source tree that is
ready to use. If the checkout updated the script itself, that means
it needs to be rerun to cause any new versions or components
referenced by the new script version to be properly checked out. This
patch makes the script check if it was modified by the checkout
process, and re-exec itself (with the same arguments) if so.
* scripts/build-many-glibcs.py (Context.__init__): Save text of
script being executed.
(Context.get_script_text): New function.
(Context.exec_self): Likewise.
(Context.checkout): Re-exec script if changed by checkout process.
This patch makes build-many-glibcs.py record the component versions
checked out, and whether those versions were explicitly requested or
defaults, in a file versions.json in the source directory.
The preferred version of a component is the first of: one explicitly
specified on the command line; one explicitly specified on the command
line in a previous run of build-many-glibcs.py; a default version for
that component. Thus, once you've run build-many-glibcs.py checkout
once with the updated script (making sure to specify the right
versions of any components previously checked out with a non-default
version), in future you can just run it without version specifiers and
it will know when a default-version component has changed its default
version and so should be checked out again.
Because you might have local changes and not want a default-version
component checkout replaced, you need to pass the --replace-sources
option to allow the script to delete and replace a component source
directory automatically; otherwise, it will give an error if a version
has changed. The script does not try to change branches of git or SVN
checkouts without checking out from scratch; if the version number
requested has changed and --replace-sources is used, the relevant
source directory will be removed completely and a new one checked out
from scratch.
Apart from allowing automatic updates of components with default
versions, this also facilitates bots reporting on the versions used in
a given build. versions.json contains not just information on the
version number and whether that was requested explicitly, but also git
or SVN revision information intended to be used in email reports from
bots.
* scripts/build-many-glibcs.py: Import json module.
(Context.__init__): Take replace_sources argument. Load
versions.json.
(Context.load_versions_json): New function.
(Context.store_json): Likewise.
(Context.store_versions_json): Likewise.
(Context.set_component_version): Likewise.
(Context.checkout): Update versions.json. Check for and handle
changes of version. Prefer previously explicitly specified
version to default version.
(Context.checkout_vcs): Return a revision identifier.
(Context.git_checkout): Likewise.
(Context.gcc_checkout): Likewise.
(get_parser): Add --replace-sources option.
(main): Pass replace_sources argument to Context call.
My most recent build-many-glibcs.py build with GCC mainline showed
build failures for tilepro with the symptoms (multiple definitions of
symbols building ld.so, see the build log referenced in the GCC bug
referenced in the comment for an example) that correspond to the
isolate-erroneous-paths optimization not being suitable for building
glibc unless the GCC port provides a trap pattern (so __builtin_trap
expands to an inline instruction rather than a call to abort). Since
tilepro indeed lacks such as pattern in GCC, this patch duly arranges
for this optimization to be disabled when building for tilepro, as it
is for sh.
Tested (compilation only) for tilepro.
* scripts/build-many-glibcs.py (Context.add_all_configs): Also use
-fno-isolate-erroneous-paths options for tilepro.
As requested in
<https://sourceware.org/ml/libc-alpha/2016-11/msg00664.html>, this
patch makes the commands recorded in build-many-glibcs.py quote words
so they can be cut-and-pasted back into a shell. (Note that these
logs are generated by the wrapper script generated to run commands
with logs, hence the needs for quoting logic to be implemented in that
shell script.)
* scripts/build-many-glibcs.py (Context.write_files): Make wrapper
script quote words in command output to log suitably for input to
the shell.
This patch adds the missing GCC configure options required to make use
of the newly built host libraries in build-many-glibcs.py.
* scripts/build-many-glibcs.py (Config.build_gcc): Configure with
newly built gmp, mpfr and mpc.
* scripts/build-many-glibcs.py (os.path): Do not import.
(Context): Inherit explicitly from object. Remove blank line
between class and docstring.
(Config): Likewise.
(Glibc): Likewise.
(Command): Likewise.
(CommandList): Likewise.
(Context.write_files): Store chmod mode in a variable.
This patch adds a Python (3.5 or later) script to build many different
configurations of glibc, including building the required cross
compilers first. It's not intended to change any patch testing
requirements, although some people may wish to use it for high-risk
patches such as adding warning options (and it can also be used to
test building, including compiling tests, for an individual
configuration, if e.g. you wish to do such a compilation test of a
patch for an architecture it touches).
The configurations include all the GNU/Linux ABI variants in
<https://sourceware.org/glibc/wiki/ABIList> (although some do not yet
build cleanly) and it would be desirable to cover enough other
variants e.g. for CPUs using different sysdeps directories to test
building each piece of code in glibc at least once. It would also be
desirable to extend it to cover Hurd and NaCl, which might best be
done by people familiar with those configurations.
You call the script as
build-many-glibcs.py /some/where thing-to-do <other-arguments>
where /some/where is a working directory for the script. It will
create and use subdirectories build, install, logs therein. You can
use it with thing-to-do being "checkout" to create a subdirectory src
therein, with subdirectories binutils, gcc, glibc, gmp, linux, mpc,
mpfr with the sources of those components, or create those directories
manually (all except glibc can be symlinks to sources elsewhere). In
the checkout case, by default it checks out GCC 6 branch, binutils
2.27 branch, glibc mainline and releases of other components. You can
specify <component>-<version> to choose a version to check out, where
<version> is "vcs-mainline" or "vcs-<branch>" to check out from
version control (only supported for gcc, binutils, glibc) and
otherwise a release version number to download and use a tarball;
components not specified on the command line have default versions
checked out. If you rerun "checkout" (with the same version
specifications) it will update checkouts from version control, but
will not detect cases where the location something is expected to be
checked out from has changed.
Other than "checkout", thing-to-do is one of host-libraries,
compilers, glibcs. So you run, in that order:
build-many-glibcs.py /some/where host-libraries
build-many-glibcs.py /some/where compilers
build-many-glibcs.py /some/where glibcs
host-libraries is run once and then those libraries are used for all
the compilers. compilers can be run once and then used many times for
testing different glibc versions (so a bot only needs to update glibc
and rerun the glibcs task, if using stable GCC / binutils; if testing
the latest versions of the whole toolchain together including mainline
GCC, it would probably want to update everything and rerun both
compilers and glibcs). You can also name particular variants after
"compilers" or "glibcs" to build just those variants (the possible
variants are hardcoded in the script).
I may add support for allowing the set of configurations to depend on
the GCC version (to get cleaner default results), and optionally
looping over architecture-independent glibc variants of CFLAGS and
configure options as well, for every glibc configuration listed
(e.g. -Os).
GCC versions before 4.9 are not expected to work (the code uses
--with-glibc-version to get the bootstrap GCC appropriately
configured). There are various problems for particular configurations
as well.
Command-line options to the script: -jN to run N jobs in parallel
(default the number of CPU cores reported by the system); --keep=all
or --keep=failed to control keeping around build directories (default
--keep=none).
* scripts/build-many-glibcs.py: New file.
check-installed-headers tests were failing for x32 because of the x86
bits/sysctl.h containing a #error for x32. This patch makes the tests
ignore sys/sysctl.h for x32, similar to the other special-casing of
particular headers.
Tested for x86_64 (full testing for -m64, compile-only for x32).
* scripts/check-installed-headers.sh: Ignore sys/sysctl.h for x32.
This adds a test to ensure that the problems fixed in the last several
patches do not recur. Each directory checks the headers that it
installs for two properties: first, each header must be compilable in
isolation, as both C and C++, under a representative combination of
language and library conformance levels; second, there is a blacklist
of identifiers that may not appear in any installed header, currently
consisting of the legacy BSD typedefs. (There is an exemption for the
headers that define those typedefs, and for the RPC headers. It may be
necessary to make this more sophisticated if we add more stuff to the
blacklist in the future.)
In order for this test to work correctly, every wrapper header
that actually defines something must guard those definitions with
#ifndef _ISOMAC. This is the existing mechanism used by the conform/
tests to tell wrapper headers not to define anything that the public
header wouldn't, and not to use anything from libc-symbols.h. conform/
only cares for headers that we need to check for standards conformance,
whereas this test applies to *every* header. (Headers in include/ that
are either installed directly, or are internal-use-only and do *not*
correspond to any installed header, are not affected.)
* scripts/check-installed-headers.sh: New script.
* Rules: In each directory that defines header files to be installed,
run check-installed-headers.sh on them as a special test.
* Makefile: Likewise for the headers installed at top level.
* include/aliases.h, include/alloca.h, include/argz.h
* include/arpa/nameser.h, include/arpa/nameser_compat.h
* include/elf.h, include/envz.h, include/err.h
* include/execinfo.h, include/fpu_control.h, include/getopt.h
* include/gshadow.h, include/ifaddrs.h, include/libintl.h
* include/link.h, include/malloc.h, include/mcheck.h
* include/mntent.h, include/netinet/ether.h
* include/nss.h, include/obstack.h, include/printf.h
* include/pty.h, include/resolv.h, include/rpc/auth.h
* include/rpc/auth_des.h, include/rpc/auth_unix.h
* include/rpc/clnt.h, include/rpc/des_crypt.h
* include/rpc/key_prot.h, include/rpc/netdb.h
* include/rpc/pmap_clnt.h, include/rpc/pmap_prot.h
* include/rpc/pmap_rmt.h, include/rpc/rpc.h
* include/rpc/rpc_msg.h, include/rpc/svc.h
* include/rpc/svc_auth.h, include/rpc/xdr.h
* include/rpcsvc/nis_callback.h, include/rpcsvc/nislib.h
* include/rpcsvc/yp.h, include/rpcsvc/ypclnt.h
* include/rpcsvc/ypupd.h, include/shadow.h
* include/stdio_ext.h, include/sys/epoll.h
* include/sys/file.h, include/sys/gmon.h, include/sys/ioctl.h
* include/sys/prctl.h, include/sys/profil.h
* include/sys/statfs.h, include/sys/sysctl.h
* include/sys/sysinfo.h, include/ttyent.h, include/utmp.h
* sysdeps/arm/nacl/include/bits/setjmp.h
* sysdeps/mips/include/sys/asm.h
* sysdeps/unix/sysv/linux/include/sys/sysinfo.h
* sysdeps/unix/sysv/linux/include/sys/timex.h
* sysdeps/x86/fpu/include/bits/fenv.h:
Add #ifndef _ISOMAC guard around internal declarations.
Add multiple-inclusion guard if not already present.
rtld only needs shared objects, so the other patterns are pointless and
significantly increase the work make has to perform while identifying
which pattern rule to apply.
This reverts commit 62ce266b0b.
The change is not mature enough because it needs the following fixes:
1. Redirect test output to a file like other tests
2. Eliminate the need to use a .gdbinit because distributions will
break without it. I should have caught that but I was in too much
of a hurry to get the patch in :/
3. Feature checking during configure to determine things like minimum
required gdb version, python-pexpect version, etc. to make sure
that tests work correctly.
This patch adds pretty printers for the following NPTL types:
- pthread_mutex_t
- pthread_mutexattr_t
- pthread_cond_t
- pthread_condattr_t
- pthread_rwlock_t
- pthread_rwlockattr_t
To load the pretty printers into your gdb session, do the following:
python
import sys
sys.path.insert(0, '/path/to/glibc/build/nptl/pretty-printers')
end
source /path/to/glibc/source/pretty-printers/nptl-printers.py
You can check which printers are registered and enabled by issuing the
'info pretty-printer' gdb command. Printers should trigger automatically when
trying to print a variable of one of the types mentioned above.
The printers are architecture-independent, and were manually tested on both
the gdb CLI and Eclipse CDT.
In order to work, the printers need to know the values of various flags that
are scattered throughout pthread.h and pthreadP.h as enums and #defines. Since
replicating these constants in the printers file itself would create a
maintenance burden, I wrote a script called gen-py-const.awk that Makerules uses
to extract the constants. This script is pretty much the same as gen-as-const.awk,
except it doesn't cast the constant values to 'long' and is thorougly documented.
The constants need only to be enumerated in a .pysym file, which is then referenced
by a Make variable called gen-py-const-headers.
As for the install directory, I discussed this with Mike Frysinger and Siddhesh
Poyarekar, and we agreed that it can be handled in a separate patch, and it shouldn't
block merging of this one.
In addition, I've written a series of test cases for the pretty printers.
Each lock type (mutex, condvar and rwlock) has two test programs, one for itself
and other for its related 'attributes' object. Each test program in turn has a
PExpect-based Python script that drives gdb and compares its output to the
expected printer's. The tests run on the glibc host, which is assumed to have
both gdb and PExpect; if either is absent the tests will fail with code 77
(UNSUPPORTED). For cross-testing you should use cross-test-ssh.sh as test-wrapper.
I've tested the printers on both a native build and a cross build using a Beaglebone
Black, with the build system's filesystem shared with the board through NFS.
Finally, I've written a README that explains all this and more.
Hopefully this should be good to go in now. Thanks.
ChangeLog:
2016-07-04 Martin Galvan <martin.galvan@tallertechnologies.com>
* Makeconfig (build-hardcoded-path-in-tests): Set to 'yes' for shared builds
if tests-need-hardcoded-path is defined.
(all-subdirs): Add pretty-printers.
* Makerules ($(py-const)): New rule.
* Rules (others): Add $(py-const), if defined.
* nptl/Makefile (gen-py-const-headers): Define.
* nptl/nptl-printers.py: New file.
* nptl/nptl_lock_constants.pysym: Likewise.
* pretty-printers/Makefile: Likewise.
* pretty-printers/README: Likewise.
* pretty-printers/test-condvar-attributes.c: Likewise.
* pretty-printers/test-condvar-attributes.p: Likewise.
* pretty-printers/test-condvar-printer.c: Likewise.
* pretty-printers/test-condvar-printer.py: Likewise.
* pretty-printers/test-mutex-attributes.c: Likewise.
* pretty-printers/test-mutex-attributes.py: Likewise.
* pretty-printers/test-mutex-printer.c: Likewise.
* pretty-printers/test-mutex-printer.py: Likewise.
* pretty-printers/test-rwlock-attributes.c: Likewise.
* pretty-printers/test-rwlock-attributes.py: Likewise.
* pretty-printers/test-rwlock-printer.c: Likewise.
* pretty-printers/test-rwlock-printer.py: Likewise.
* pretty-printers/test_common.py: Likewise.
* scripts/gen-py-const.awk: Likewise.
* scripts/check-local-headers.sh (exclude): Add hurd/ihash.h, and
include .*-.*/ in addition to .*-.*-.*/ (i.e. i386-gnu in addition to
i386-linux-gnu).
This patch updates texinfo.tex, config.guess, config.sub and
move-if-change from their respective upstream sources.
* manual/texinfo.tex: Update to version 2016-01-04.21 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2016-01-01.
* scripts/config.sub: Update to version 2016-01-01.
* scripts/move-if-change: Update from gnulib.
The new format lists the version on each line, as in:
VERSION SYMBOL TYPE [VALUE]
This makes it easier to process the files with line-oriented tools.
The abilist files were converted with this awk script:
/^[^ ]/ { version = $1 }
/^ / { print version, substr($0, 2) }
And sorted under the "C" locale with sort.
Linker in binutils 2.26 and newer generate GOT references instead
PLT references when -z now is passed to linker. We need to extend
scripts/localplt.awk to allow PLT or GOT references.
[BZ #19007]
* scripts/localplt.awk: Also allow GOT references.
* sysdeps/unix/sysv/linux/i386/localplt.data: Mark
_Unwind_Find_FDE, calloc, memalign, realloc and __libc_memalign
with "+ REL R_386_GLOB_DAT".
* sysdeps/x86_64/localplt.data: Mark calloc, memalign, realloc
and __libc_memalign with "+ RELA R_X86_64_GLOB_DAT".
The file scripts/rpm2dynsym.sh appears to be unused anywhere in glibc.
This patch removes this script.
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
* scripts/rpm2dynsym.sh: Remove file.
The third arg can either be a string with "g" or "G", or it is a number.
The empty string elicits a warning with newer versions like so:
gawk: scripts/sysd-rules.awk:56: warning: gensub: third argument `' treated as 1
On x86, linker in binutils 2.26 and newer consolidates R_*_JUMP_SLOT with
R_*_GLOB_DAT relocation against the same symbol. This patch extends
local PLT reference check to support alternate relocations.
[BZ #18078]
* scripts/check-localplt.awk: Support alternate relocations.
* scripts/localplt.awk: Also check relocations in DT_RELA/DT_REL
sections.
* sysdeps/unix/sysv/linux/i386/localplt.data: Mark free and
malloc entries with + REL R_386_GLOB_DAT.
* sysdeps/x86_64/localplt.data: New file.
Since 2014-11-24 binutils git commit bb4d2ac2, readelf has appended
the symbol version to symbols shown in reloc dumps.
[BZ #16512]
* scripts/localplt.awk: Strip off symbol version.
* NEWS: Mention bug fix.
True multi-dimensional arrays were introduced in awk 4.0 and we
support awk versions as early as 3.12. Use a single subscript of the
form prefix_conf instead of two dimensions to work around this
limitation. We also need one additional array of just the conf names
subscripted by the prefix_conf to print the names for the
specifications.
* scripts/gen-posix-conf-vars.awk: Don't use multi-dimensional
arrays.
This patch adds support to generate the spec array in getconf from the
conf.list. The generated code is mostly unchanged. the only changes
are due to the change in layout of the spec and val arrays in the ELF.
The val array can also be auto-generated from posix-conf-vars.list
once the remaining macros are added to it.
* posix/posix-conf-vars.list (SPEC:XBS5): Add sysconf prefix.
* posix/confstr.c: Define NEED_SPEC_ARRAY to 0.
* posix/posix-envs.def: Likewise.
* sysdeps/posix/sysconf.c: Likewise.
* posix/getconf.c: Define NEED_SPEC_ARRAY to 1.
(specs): Remove array.
* scripts/gen-posix-conf-vars.awk: Support generation of specs
array.
This patch adds a file posix-conf-vars.list that is used to generate
macros to determine if a macro is defined as set, unset or not
defined. gen-posix-conf-vars.awk processes this file and generates a
header (posix-conf-vars-def.h) with these macros. A new header
posix-conf-vars.h includes this generated header and defines accessor
macros for the generated macros.
Tested on x86_64.
* posix/Makefile (before-compile): Add posix-conf-vars-def.h.
($(objpfx)posix-conf-vars-def.h): New target.
* posix/posix-conf-vars.list: New file.
* posix/posix-conf-vars.h: New file.
* posix/confstr.c: Include posix-conf-vars.h.
(confstr): Use CONF_IS_* macros.
* posix/posix-envs.def: Include posix-conf-vars.h. Use
CONF_IS_* macros.
* scripts/gen-posix-conf-vars.awk: New file.
Remove libc-modules.h from the tree and auto-generate it from
soversions.i and the list of modules in the built-modules variable
defined in Makeconfig. Macros generated have increasing numbered
values, with built-modules having lower values starting from 1,
following which a separator value LIBS_BEGIN is added and then finally
the library names from soversions.i are appended to the list. This
allows us to conveniently differentiate between the versioned
libraries and other built modules, which is needed in errno.h and
netdb.h to decide whether to use an internal symbol or an external
one.
Verified that generated code remains unchanged on x86_64.
* Makeconfig (built-modules): List non-library modules to be
built.
(module-cppflags): Include libc-modules.h for
everything except shlib-versions.v.i.
(CPPFLAGS): Use it.
(before-compile): Add libc-modules.h.
($(common-objpfx)libc-modules.h,
$(common-objpfx)libc-modules.stmp): New targets.
(common-generated): Add libc-modules.h and libc-modules.stmp.
($(common-objpfx)Versions.v.i): Depend on libc-modules.h.
* include/libc-symbols.h: Don't include libc-modules.h.
* include/libc-modules.h: Remove file.
* scripts/gen-libc-modules.awk: New script to generate
libc-modules.h.
* sysdeps/unix/Makefile ($(common-objpfx)sysd-syscalls):
Depend on libc-modules.stmp.
This patch eliminates the mixture of SONAME information in
shlib-versions files and SONAME information used to generate
gnu/lib-names.h in makefiles, with the information in the makefiles
being removed so all this information comes from the shlib-versions
files.
So that gnu/lib-names.h supports multiple ABIs, it is changed to be
generated on the same basis as gnu/stubs.h: when there are multiple
ABIs, gnu/lib-names.h is a wrapper header (the same header installed
whatever ABI is being built) and separate headers such as
gnu/lib-names-64.h contain the substantive contents (only one such
header being installed by any glibc build).
The rules for building gnu/lib-names.h were moved from Makeconfig to
Makerules because they need to come after sysdeps makefiles are
included (now that "ifndef abi-variants" is a toplevel conditional on
the rules rather than $(abi-variants) being evaluated later inside the
commands for a rule).
Tested for x86_64 and x86 that the installed shared libraries are
unchanged by this patch, and examined the installed gnu/lib-names*.h
headers by hand. Also tested the case of a single ABI (where there is
just a single header installed, again like stubs.h) by hacking
abi-variants to empty for x86_64.
[BZ #14171]
* Makeconfig [$(build-shared) = yes]
($(common-objpfx)soversions.mk): Don't handle SONAMEs specified in
makefiles.
[$(build-shared) = yes && $(soversions.mk-done) = t]
($(common-objpfx)gnu/lib-names.h): Remove rule.
[$(build-shared) = yes && $(soversions.mk-done) = t]
($(common-objpfx)gnu/lib-names.stmp): Likewise. Split and moved
to Makerules.
[$(build-shared) = yes && $(soversions.mk-done) = t]
(before-compile): Don't append $(common-objpfx)gnu/lib-names.h
here.
[$(build-shared) = yes && $(soversions.mk-done) = t]
(common-generated): Don't append gnu/lib-names.h and
gnu/lib-names.stmp here.
* Makerules [$(build-shared) = yes && $(soversions.mk-done) = t]
(lib-names-h-abi): New variable.
[$(build-shared) = yes && $(soversions.mk-done) = t]
(lib-names-stmp-abi): Likewise.
[$(build-shared) = yes && $(soversions.mk-done) = t &&
abi-variants] (before-compile): Append
$(common-objpfx)$(lib-names-h-abi).
[$(build-shared) = yes && $(soversions.mk-done) = t &&
abi-variants] (common-generated): Append gnu/lib-names.h.
[$(build-shared) = yes && $(soversions.mk-done) = t &&
abi-variants] (install-others-nosubdir): Depend on
$(inst_includedir)/$(lib-names-h-abi).
[$(build-shared) = yes && $(soversions.mk-done) = t &&
abi-variants] ($(common-objpfx)gnu/lib-names.h): New rule.
[$(build-shared) = yes && $(soversions.mk-done) = t]
($(common-objpfx)$(lib-names-h-abi)): New rule.
[$(build-shared) = yes && $(soversions.mk-done) = t]
($(common-objpfx)$(lib-names-stmp-abi)): Likewise.
[$(build-shared) = yes && $(soversions.mk-done) = t]
(common-generated): Append $(lib-names-h-abi) and
$(lib-names-stmp-abi).
* scripts/lib-names.awk: Do not handle multi being set.
* sysdeps/unix/sysv/linux/aarch64/Makefile (abi-lp64-ld-soname):
Remove variable.
(abi-lp64_be-ld-soname): Likewise.
* sysdeps/unix/sysv/linux/arm/Makefile (abi-soft-ld-soname):
Likewise.
(abi-hard-ld-soname): Likewise.
* sysdeps/unix/sysv/linux/i386/shlib-versions: New file.
* sysdeps/unix/sysv/linux/mips/Makefile (abi-o32_soft-ld-soname):
Remove variable.
(abi-o32_hard-ld-soname): Likewise.
(abi-o32_soft_2008-ld-soname): Likewise.
(abi-o32_hard_2008-ld-soname): Likewise.
(abi-n32_soft-ld-soname): Likewise.
(abi-n32_hard-ld-soname): Likewise.
(abi-n32_soft_2008-ld-soname): Likewise.
(abi-n32_hard_2008-ld-soname): Likewise.
(abi-n64_soft-ld-soname): Likewise.
(abi-n64_hard-ld-soname): Likewise.
(abi-n64_soft_2008-ld-soname): Likewise.
(abi-n64_hard_2008-ld-soname): Likewise.
* sysdeps/unix/sysv/linux/powerpc/Makefile (abi-64-v1-ld-soname):
Likewise.
(abi-64-v2-ld-soname): Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/shlib-versions: Add
ld.so entries.
* sysdeps/unix/sysv/linux/s390/Makefile (abi-64-ld-soname): Remove
variable.
* sysdeps/unix/sysv/linux/s390/s390-64/shlib-versions: Add ld.so
entry.
* sysdeps/unix/sysv/linux/x86/Makefile (abi-32-ld-soname): Remove
variable.
(abi-64-ld-soname): Likewise.
(abi-x32-ld-soname): Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/shlib-versions: Add ld.so
entry.
* sysdeps/unix/sysv/linux/x86_64/x32/shlib-versions: Likewise.
This patch removes the --enable-oldest-abi configure option, which has
long been bitrotten (as reported in bug 6652). The principle of
removing this option was agreed in the thread starting at
<https://sourceware.org/ml/libc-alpha/2013-07/msg00174.html>.
Tested for x86_64 and x86 that the installed shared libraries other
than libc.so are unchanged by this patch and that libc.so disassembly
and symbol versions are unchanged (debug info changes because of
changed line numbers in csu/version.c).
[BZ #6652]
* Makeconfig (soversions-default-setname): Remove variable.
($(common-objpfx)soversions.i): Don't pass default_setname to
soversions.awk.
* Makerules ($(common-objpfx)abi-versions.h): Don't pass
oldest_abi to abi-versions.awk.
* config.h.in (GLIBC_OLDEST_ABI): Remove macro undefine.
* config.make.in (oldest-abi): Remove variable.
* configure.ac (--enable-oldest-abi): Remove configure option.
* configure: Regenerated.
* csu/version.c (banner) [GLIBC_OLDEST_ABI]: Remove conditional
text.
* scripts/abi-versions.awk: Do not handle oldest_abi variable.
* scripts/soversions.awk: Do not handle default_setname variable.
* sysdeps/mach/hurd/configure.ac: Do not handle oldest_abi
variable.
* sysdeps/mach/hurd/configure: Regenerated.
* sysdeps/unix/sysv/linux/configure.ac: Do not handle oldest_abi
variable.
* sysdeps/unix/sysv/linux/configure: Regenerated.
This patch removes the first column (patterns matching configuration
names) from shlib-versions, leaving shlib-versions entry selection
based purely on sysdeps directories.
An implication of this removal is that the default for any non-Linux
ports using NPTL will be the same SONAMEs for NPTL libraries as for
Linux (as those defaults, previously limited to .*-.*-linux.*, are
left in nptl/shlib-versions and nptl_db/shlib-versions).
Special host_os handling in configure.ac that was purely for
shlib-versions is removed. (The host_os setting is still used for
libc-abis - see
<https://sourceware.org/ml/libc-alpha/2014-01/msg00375.html> regarding
that - but no entries there are affected by this change.)
Tested on x86_64 and x86 that the installed shared libraries are
unchanged by this patch.
* scripts/soversions.awk: Do not handle configuration names.
* Makeconfig ($(common-objpfx)soversions.i): Do not pass cpu,
vendor and os variables to soversions.awk.
* configure.ac: Do not modify gnu-* host_os.
* configure: Regenerated
* shlib-versions: Remove first column with configuration names.
* nptl/shlib-versions: Likewise.
* nptl_db/shlib-versions: Likewise.
* sysdeps/hppa/shlib-versions: Likewise.
* sysdeps/m68k/shlib-versions: Likewise.
* sysdeps/mach/hurd/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/aarch64/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/alpha/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/arm/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/hppa/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/ia64/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/microblaze/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/mips/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/shlib-versions:
Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/sh/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/tile/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/shlib-versions: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/shlib-versions: Likewise.
libidn/ChangeLog:
* shlib-versions: Remove first column with configuration names.
shlib-versions files can contain ABI lines that map triplets to a
canonical ABI name. This name was once used for various purposes
where test baseline files for different ABIs went in a single
directory; now these purposes use sysdeps files, generation of headers
which have per-ABI variants uses abi-variants and related Makefile
variables and the shlib-versions ABI names are unused. This patch
duly removes those lines and associated build system support for them.
Tested for x86_64 (both a full testsuite run and confirming the
installed shared libraries are unchanged by the patch).
* Makeconfig ($(common-objpfx)soversions.mk): Do not generate
abi-name definition.
* scripts/soversions.awk: Do not handle or generate ABI lines.
* shlib-versions: Remove ABI entries.
* sysdeps/powerpc/nofpu/shlib-versions: Remove file.
* sysdeps/x86_64/x32/shlib-versions: Remove ABI entry.
scripts/list-sources.sh includes handling of ports that has been
obsolete ever since the ports repository was merged into the libc
repository. This patch removes that handling.
Tested by regenerating libc.pot and examining the resulting changes.
* scripts/list-sources.sh: Do not handle ports specially.
This patch updates miscellaneous files from their upstream sources:
texinfo.tex from Texinfo, config.guess and config.sub from config.git,
install-sh from automake and move-if-change from gnulib.
Tested x86_64 that installed shared libraries are unchanged by the
patch; also looked at the generated libc.pdf manual.
* manual/texinfo.tex: Update to version 2014-05-05.10 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2014-03-23.
* scripts/config.sub: Update to version 2014-05-01
* scripts/install-sh: Update to version 2013-12-25.23.
* scripts/move-if-change: Update from gnulib.
One wart in the original support for test wrappers for cross testing,
as noted in
<https://sourceware.org/ml/libc-alpha/2012-10/msg00722.html>, is the
requirement for test wrappers to pass a poorly-defined set of
environment variables from the build system to the system running the
glibc under test. Although some variables are passed explicitly via
$(test-wrapper-env), including LD_* variables that simply can't be
passed implicitly because of the side effects they'd have on the build
system's dynamic linker, others are passed implicitly, including
variables such as GCONV_PATH and LOCPATH that could potentially affect
the build system's libc (so effectively relying on any such effects
not breaking the wrappers). In addition, the code in
cross-test-ssh.sh for preserving environment variables is fragile (it
depends on how bash formats a list of exported variables, and could
well break for multi-line variable definitions where the contents
contain things looking like other variable definitions).
This patch moves to explicitly passing environment variables via
$(test-wrapper-env). Makefile variables that previously used
$(test-wrapper) are split up into -before-env and -after-env parts
that can be passed separately to the various .sh files used in
testing, so those files can then insert environment settings between
the two parts.
The common default environment settings in make-test-out are made into
a separate makefile variable that can also be passed to scripts,
rather than many scripts duplicating those settings (for testing an
installed glibc, it is desirable to have the GCONV_PATH setting on
just one place, so just that one place needs to support it pointing to
an installed sysroot instead of the build tree). The default settings
are included in the variables such as $(test-program-prefix), so that
if tests do not need any non-default settings they can continue to use
single variables rather than the split-up variables.
Although this patch cleans up LC_ALL=C settings (that being part of
the common defaults), various LANG=C and LANGUAGE=C settings remain.
Those are generally unnecessary and I propose a subsequent cleanup to
remove them. LC_ALL takes precedence over LANG, and while LANGUAGE
takes precedence over LC_ALL, it only does so for settings other than
LC_ALL=C. So LC_ALL=C on its own is sufficient to ensure the C
locale, and anything that gets LC_ALL=C does not need the other
settings.
While preparing this patch I noticed some tests with .sh files that
appeared to do nothing beyond what the generic makefile support for
tests can do (localedata/tst-wctype.sh - the makefiles support -ENV
variables and .input files - and localedata/tst-mbswcs.sh - just runs
five tests that could be run individually from the makefile). So I
propose another subsequent cleanup to move those to using the generic
support instead of special .sh files.
Tested x86_64 (native) and powerpc32 (cross).
* Makeconfig (run-program-env): New variable.
(run-program-prefix-before-env): Likewise.
(run-program-prefix-after-env): Likewise.
(run-program-prefix): Define in terms of new variables.
(built-program-cmd-before-env): New variable.
(built-program-cmd-after-env): Likewise.
(built-program-cmd): Define in terms of new variables.
(test-program-prefix-before-env): New variable.
(test-program-prefix-after-env): Likewise.
(test-program-prefix): Define in terms of new variables.
(test-program-cmd-before-env): New variable.
(test-program-cmd-after-env): Likewise.
(test-program-cmd): Define in terms of new variables.
* Rules (make-test-out): Use $(run-program-env).
* scripts/cross-test-ssh.sh (env_blacklist): Remove variable.
(help): Do not mention environment variables. Mention
--timeoutfactor option.
(timeoutfactor): New variable.
(blacklist_exports): Remove function.
(exports): Remove variable.
(command): Do not include ${exports}.
* manual/install.texi (Configuring and compiling): Do not mention
test wrappers preserving environment variables. Mention that last
assignment to a variable must take precedence.
* INSTALL: Regenerated.
* benchtests/Makefile (run-bench): Use $(run-program-env).
* catgets/Makefile ($(objpfx)test1.cat): Use
$(built-program-cmd-before-env), $(run-program-env) and
$(built-program-cmd-after-env).
($(objpfx)test2.cat): Do not specify environment variables
explicitly.
($(objpfx)de/libc.cat): Use $(built-program-cmd-before-env),
$(run-program-env) and $(built-program-cmd-after-env).
($(objpfx)test-gencat.out): Use $(test-program-cmd-before-env),
$(run-program-env) and $(test-program-cmd-after-env).
($(objpfx)sample.SJIS.cat): Do not specify environment variables
explicitly.
* catgets/test-gencat.sh: Use test_program_cmd_before_env,
run_program_env and test_program_cmd_after_env arguments.
* elf/Makefile ($(objpfx)tst-pathopt.out): Use $(run-program-env).
* elf/tst-pathopt.sh: Use run_program_env argument.
* iconvdata/Makefile ($(objpfx)iconv-test.out): Use
$(test-wrapper-env) and $(run-program-env).
* iconvdata/run-iconv-test.sh: Use test_wrapper_env and
run_program_env arguments.
* iconvdata/tst-table.sh: Do not set GCONV_PATH explicitly.
* intl/Makefile ($(objpfx)tst-gettext.out): Use
$(test-program-prefix-before-env), $(run-program-env) and
$(test-program-prefix-after-env).
($(objpfx)tst-gettext2.out): Likewise.
* intl/tst-gettext.sh: Use test_program_prefix_before_env,
run_program_env and test_program_prefix_after_env arguments.
* intl/tst-gettext2.sh: Likewise.
* intl/tst-gettext4.sh: Do not set environment variables
explicitly.
* intl/tst-gettext6.sh: Likewise.
* intl/tst-translit.sh: Likewise.
* malloc/Makefile ($(objpfx)tst-mtrace.out): Use
$(test-program-prefix-before-env), $(run-program-env) and
$(test-program-prefix-after-env).
* malloc/tst-mtrace.sh: Use test_program_prefix_before_env,
run_program_env and test_program_prefix_after_env arguments.
* math/Makefile (run-regen-ulps): Use $(run-program-env).
* nptl/Makefile ($(objpfx)tst-tls6.out): Use $(run-program-env).
* nptl/tst-tls6.sh: Use run_program_env argument. Set LANG=C
explicitly with each use of ${test_wrapper_env}.
* posix/Makefile ($(objpfx)wordexp-tst.out): Use
$(test-program-prefix-before-env), $(run-program-env) and
$(test-program-prefix-after-env).
* posix/tst-getconf.sh: Do not set environment variables
explicitly.
* posix/wordexp-tst.sh: Use test_program_prefix_before_env,
run_program_env and test_program_prefix_after_env arguments.
* stdio-common/tst-printf.sh: Do not set environment variables
explicitly.
* stdlib/Makefile ($(objpfx)tst-fmtmsg.out): Use
$(test-program-prefix-before-env), $(run-program-env) and
$(test-program-prefix-after-env).
* stdlib/tst-fmtmsg.sh: Use test_program_prefix_before_env,
run_program_env and test_program_prefix_after_env arguments.
Split $test calls into $test_pre and $test.
* timezone/Makefile (build-testdata): Use
$(built-program-cmd-before-env), $(run-program-env) and
$(built-program-cmd-after-env).
localedata/ChangeLog:
* Makefile ($(addprefix $(objpfx),$(CTYPE_FILES))): Use
$(built-program-cmd-before-env), $(run-program-env) and
$(built-program-cmd-after-env).
($(objpfx)sort-test.out): Use $(test-program-prefix-before-env),
$(run-program-env) and $(test-program-prefix-after-env).
($(objpfx)tst-fmon.out): Use $(run-program-prefix-before-env),
$(run-program-env) and $(run-program-prefix-after-env).
($(objpfx)tst-locale.out): Use $(built-program-cmd-before-env),
$(run-program-env) and $(built-program-cmd-after-env).
($(objpfx)tst-trans.out): Use $(run-program-prefix-before-env),
$(run-program-env), $(run-program-prefix-after-env),
$(test-program-prefix-before-env) and
$(test-program-prefix-after-env).
($(objpfx)tst-ctype.out): Use $(test-program-cmd-before-env),
$(run-program-env) and $(test-program-cmd-after-env).
($(objpfx)tst-wctype.out): Likewise.
($(objpfx)tst-langinfo.out): Likewise.
($(objpfx)tst-langinfo-static.out): Likewise.
* gen-locale.sh: Use localedef_before_env, run_program_env and
localedef_after_env arguments.
* sort-test.sh: Use test_program_prefix_before_env,
run_program_env and test_program_prefix_after_env arguments.
* tst-ctype.sh: Use tst_ctype_before_env, run_program_env and
tst_ctype_after_env arguments.
* tst-fmon.sh: Use run_program_prefix_before_env, run_program_env
and run_program_prefix_after_env arguments.
* tst-langinfo.sh: Use tst_langinfo_before_env, run_program_env
and tst_langinfo_after_env arguments.
* tst-locale.sh: Use localedef_before_env, run_program_env and
localedef_after_env arguments.
* tst-mbswcs.sh: Do not set environment variables explicitly.
* tst-numeric.sh: Likewise.
* tst-rpmatch.sh: Likewise.
* tst-trans.sh: Use run_program_prefix_before_env,
run_program_env, run_program_prefix_after_env,
test_program_prefix_before_env and test_program_prefix_after_env
arguments.
* tst-wctype.sh: Use tst_wctype_before_env, run_program_env and
tst_wctype_after_env arguments.
GNU_IFUNC are shown by readelf in 'Relocation section' value as
"symbol()" instead of expected hexadecimal value. This causes the
check-localplt script to ignore potential PLT stub begin generated
by wrong IFUNC usage. This patch changes the localplt script to
emit such PLT cases.
This patch is an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00198.html> and
<https://sourceware.org/ml/libc-alpha/2014-03/msg00180.html>.
Normal practice for software testsuites is that rather than
terminating immediately when a test fails, they continue running and
report at the end on how many tests passed or failed.
The principle behind the glibc testsuite stopping on failure was
probably that the expected state is no failures and so any failure
indicates a problem such as miscompilation. In practice, while this
is fairly close to true for native testing on x86_64 and x86 (kernel
bugs and race conditions can still cause intermittent failures), it's
less likely to be the case on other platforms, and so people testing
glibc run the testsuite with "make -k" and then examine the logs to
determine whether the failures are what they expect to fail on that
platform, possibly with some automation for the comparison.
This patch switches the glibc testsuite to the normal convention of
not stopping on failure - unless you use stop-on-test-failure=y, in
which case it behaves essentially as it did before (and does not
generate overall test summaries on failure). Instead, the summary
tests.sum may contain tests that FAILed. At the end of the test run,
any FAIL or ERROR lines from tests.sum are printed, and then it exits
with error status if there were any such lines. In addition, build
failures will also cause the test run to stop - this has the
justification that those *do* indicate serious problems that should be
promptly fixed and aren't generally hard to fix (but apart from that,
avoiding the build stopping on those failures seems harder).
Note that unlike the previous patches in this series, this *does*
require people with automation around testing glibc to change their
processes - either to start using tests.sum / xtests.sum to track
failures and compare them with expectations (with or without also
using "make -k" and examining "make" logs to identify build failures),
or else to use stop-on-test-failure=y and ignore the new tests.sum /
xtests.sum mechanism. (If all you check is the exit status from "make
check", no changes are needed unless you want to avoid test runs
continuing after the first failure.)
Tested x86_64.
* scripts/evaluate-test.sh: Handle fourth argument to determine
whether test run should stop on failure.
* Makeconfig (stop-on-test-failure): New variable.
(evaluate-test): Pass fourth argument to evaluate-test.sh based on
$(stop-on-test-failure).
* Makefile (tests): Give a summary of results from testing and
exit with failure status if they include an ERROR or FAIL.
(xtests): Likewise.
* manual/install.texi (Configuring and compiling): Mention
stop-on-test-failure=y.
* INSTALL: Regenerated.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00197.html>, makes
testsuite runs generate an overall summary of test results.
A new script merge-test-results.sh deals both with collecting results
within a directory to a file with all the results from that directory,
and collecting the results from subdirectories into a single overall
file (there's not much in common between the two modes of operation of
the script, but it seemed silly to have two separate scripts for
this). Within a directory, missing results produce UNRESOLVED lines;
at top level, missing results for a whole directory produce an ERROR
line (since toplevel can't identify what the specific missing tests
are in this case).
Note that this does not change the rules for when "make" considers
there has been an error, or terminates, so unexpected failures will
still cause make to terminate, or, with -k, mean the commands for
"tests" don't get run because of failure of a dependency.
Tested x86_64, including that the summary does in fact reflect all the
tests with .test-result files.
* scripts/merge-test-results.sh: New file.
* Makefile (tests-special-notdir): New variable.
(tests): Run merge-test-results.sh.
(xtests): Likewise.
* Rules (tests-special-notdir): New variable.
(xtests-special-notdir): Likewise.
(tests): Run merge-test-results.sh
(xtests): Likewise.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00195.html>, makes it
possible for .test-result files for individual tests to contain XPASS
and XFAIL rather than PASS and FAIL in cases where failure is
expected. This replaces the marking of two individual tests with "-"
to cause them to be expected at makefile level to fail;
evaluate-test.sh will ensure it exits with status 0 for an expected
failure.
Tested x86_64.
* scripts/evaluate-test.sh: Take new argument indicating whether
failure is expected.
* Makeconfig (evaluate-test): Pass argument to evaluate-test.sh
indicating whether failure is expected.
* conform/Makefile (test-xfail-run-conformtest): New variable.
($(objpfx)run-conformtest.out): Don't expect to fail at makefile
level.
* posix/Makefile (test-xfail-annexc): New variable.
($(objpfx)annexc.out): Don't expect to fail at makefile level.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00193.html>, starts
the process of generating explicit PASS or FAIL status for individual
glibc tests. It's based on Tomas Dohnalek's patch
<https://sourceware.org/ml/libc-alpha/2012-10/msg00278.html>, but is
deliberately more minimal: it doesn't try to cover any tests outside
of $(tests) / $(xtests) (that's for a later patch), nor does it put
the result together in an overall summary file (again, a later patch):
it just generates the .test-result files.
Thus, this patch keeps the overall logic for when a testsuite run
finishes completely unchanged: a test failing will terminate the run.
I think we *should* move to a more conventional approach where plain
"make check" does not terminate for an individual test failure, unless
e.g. you say "make stop-on-test-failure=y check", but that sort of
policy change is best done as a separate patch once the infrastructure
is in place to generate summary files for completed test runs (which
will entirely consist of PASS and XFAIL lines if the testsuite run
reaches the point of generating them, until such a policy change is
made).
Tested x86_64.
2014-02-14 Tomas Dohnalek <tdohnale@redhat.com>
Joseph Myers <joseph@codesourcery.com>
* Makeconfig (test-name): New variable.
(evaluate-test): Likewise.
* Makerules (do-test-clean): Remove .test-result files.
(common-mostlyclean): Likewise.
* Rules ($(objpfx)%.out): Use $(evaluate-test) in both rules.
* scripts/evaluate-test.sh: New file.
This patch updates various miscellaneous files we take from upstream
GNU sources (texinfo.texi, config.guess, config.sub - various others
haven't changed upstream since we last updated them) to their current
upstream versions.
Tested x86_64.
* manual/texinfo.tex: Update to version 2013-11-26.10 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2013-11-29.
* scripts/config.sub: Update to version 2013-10-01.
`volatile int` means the same as 'int volatile', but that's not the
case for 'volatile char *' and 'char * volatile'. We won't need a
'char volatile *' or other complicated semantics for now.
This patch adds the ability to accept output arguments to functions
being benchmarked, by nesting the argument type in <> in the args
directive. It includes the sincos implementation as an example, where
the function would have the following args directive:
## args: double:<double *>:<double *>
This simply adds a definition for a static variable whose pointer gets
passed into the function, so it's not yet possible to pass something
more complicated like a pre-allocated string or array. That would be
a good feature to add if a function needs it.
The values in the input file will map only to the input arguments. So
if I had a directive like this for a function foo:
## args: int:<int *>:int:<int *>
and I have a value list like this:
1, 2
3, 4
5, 6
then the function calls generated would be:
foo (1, &out1, 2, &out2);
foo (3, &out1, 4, &out2);
foo (5, &out1, 6, &out2);
This adds the "include-sources" directive to scripts/bench.pl. This
allows for including source code (vs including headers, which might get
a different search path) after the inclusion of any headers.
This patch adds some more directives to the benchmark inputs file,
moving functionality from the Makefile and making the code generation
script a bit cleaner. The function argument and return types that
were earlier added as variables in the makefile and passed to the
script via command line arguments are now the 'args' and 'ret'
directive respectively. 'args' should be a colon separated list of
argument types (skipped if the function doesn't accept any arguments)
and 'ret' should be the return type.
Additionally, an 'includes' directive may have a comma separated list
of headers to include in the source. For example, the pow input file
now looks like this:
42.0, 42.0
1.0000000000000020, 1.5
I did this to unclutter the benchtests Makefile a bit and eventually
eliminate dependency of the tests on the Makefile and have tests
depend on their respective include files only.
Resolves: #15424
The compiler would optimize the benchmark function call out of the
loop and call it only once, resulting in blazingly fast times for some
benchmarks (notably atan, sin and cos). Mark the inputs as volatile
so that the code is forced to read again from the input for each
iteration.
Some math functions have distinct performance characteristics in
specific domains of inputs, where some inputs return via a fast path
while other inputs require multiple precision calculations, that too
at different precision levels. The way to implement different domains
was to have a separate source file and benchmark definition, resulting
in separate programs.
This clutters up the benchmark, so this change allows these domains to
be consolidated into the same input file. To do this, the input file
format is now enhanced to allow comments with a preceding # and
directives with two # at the begining of a line. A directive that
looks like:
tells the benchmark generation script that what follows is a different
domain of inputs. The value of the 'name' directive (in this case,
foo) is used in the output. The two input domains are then executed
sequentially and their results collated separately. with the above
directive, there would be two lines in the result that look like:
func(): ....
func(foo): ...
The idea to run benchmarks for a constant number of iterations is
problematic. While the benchmarks may run for 10 seconds on x86_64,
they could run for about 30 seconds on powerpc and worse, over 3
minutes on arm. Besides that, adding a new benchmark is cumbersome
since one needs to find out the number of iterations needed for a
sufficient runtime.
A better idea would be to run each benchmark for a specific amount of
time. This patch does just that. The run time defaults to 10 seconds
and it is configurable at command line:
make BENCH_DURATION=5 bench
The tile architecture's Linux port installs headers in an
<arch> directory; these headers are in part shared with glibc.
Ignore these headers for check-local-headers like we ignore
all the other Linux headers.
