This patch makes build-many-glibcs.py default to binutils 2.33 branch.
Tested with build-many-glibcs.py (compilers and glibcs builds).
* scripts/build-many-glibcs.py (Context.checkout): Default
binutils version to 2.33 branch.
Co-authored-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Gabriel F. T. Gomes <gabriel@inconstante.net.br>
Reviewed-by: Joseph Myers <joseph@codesourcery.com>
The utility of a ChangeLog file has been discussed in various mailing
list threads and GNU Tools Cauldrons in the past years and the general
consensus is that while the file may have been very useful in the past
when revision control did not exist or was not as powerful as it is
today, it's current utility is fast diminishing. Further, the
ChangeLog format gets in the way of modernisation of processes since
it almost always results in rewriting of a commit, thus preventing use
of any code review tools to automatically manage patches in the glibc
project.
There is consensus in the glibc community that documentation of why a
change was done (i.e. a detailed description in a git commit) is more
useful than what changed (i.e. a ChangeLog entry) since the latter can
be deduced from the patch. The GNU community would however like to
keep the option of ascertaining what changed through a ChangeLog-like
output and as a compromise, it was proposed that a script be developed
that generates this output.
The script below is the result of these discussions. This script
takes two git revisions references as input and generates the git log
between those revisions in a form that resembles a ChangeLog. Its
capabilities and limitations are listed in a comment in the script.
On a high level it is capable of parsing C code and telling what
changed at the top level, but not within constructs such as functions.
Design
------
At a high level, the script analyses the raw output of a VCS, parses
the source files that have changed and attempts to determine what
changed. The script driver needs three distinct components to be
fully functional for a repository:
- A vcstocl_quirks.py file that helps it parse weird patterns in
sources that may result from preprocessor defines.
- A VCS plugin backend; the git backend is implemented for glibc
- A programming language parser plugin. C is currently implemented.
Additional programming language parsers can be added to give more
detailed output for changes in those types of files.
For input in languages other than those that have a parser, the script
only identifies if a file has been added, removed, modified,
permissions changed, etc. but cannot understand the change in content.
The C Parser
------------
The C parser is capable of parsing C programs with preprocessor macros
in place, as if they were part of the language. This presents some
challenges with parsing code that expands macros on the fly and to
help work around that, a vcstocl_quirks.py file has transformations to
ease things.
The C parser currently can identify macro definitions and scopes and
all global and static declarations and definitions. It cannot parse
(and compare) changes inside functions yet, it could be a future
enhancement if the need for it arises.
Testing
-------
The script has been tested with the glibc repository up to glibc-2.29
and also in the past with emacs. While it would be ideal to have
something like this in a repository like gnulib, that should not be a
bottleneck for glibc to start using this, so this patch proposes to
add these scripts into glibc.
And here is (hopefully!) one of the last ChangeLog entries we'd have
to write for glibc:
* scripts/gitlog_to_changelog.py: New script to auto-generate
ChangeLog.
* scripts/vcs_to_changelog/frontend_c.py: New file.
* scripts/vcs_to_changelog/misc_util.py: New file.
* scripts/vcs_to_changelog/vcs_git.py: New file.
* scripts/vcs_to_changelog/vcstocl_quirks.py: Likewise.
This patch makes build-many-glibcs.py use Linux 5.3.
Tested with build-many-glibcs.py (host-libraries, compilers and glibcs
builds).
* scripts/build-many-glibcs.py (Context.checkout): Default Linux
version to 5.3.
Now that there are no internal users of __sysctl left, it is possible
to add an unconditional deprecation warning to <sys/sysctl.h>.
To avoid a test failure due this warning in check-install-headers,
skip the test for sys/sysctl.h.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
GCC 9 dropped support for the SPE extensions to PowerPC, which means
powerpc*-*-*gnuspe* configurations are no longer buildable with that
compiler. This ISA extension was peculiar to the “e500” line of
embedded PowerPC chips, which, as far as I can tell, are no longer
being manufactured, so I think we should follow suit.
This patch was developed by grepping for “e500”, “__SPE__”, and
“__NO_FPRS__”, and may not eliminate every vestige of SPE support.
Most uses of __NO_FPRS__ are left alone, as they are relevant to
normal embedded PowerPC with soft-float.
* sysdeps/powerpc/preconfigure: Error out on powerpc-*-*gnuspe*
host type.
* scripts/build-many-glibcs.py: Remove powerpc-*-linux-gnuspe
and powerpc-*-linux-gnuspe-e500v1 from list of build configurations.
* sysdeps/powerpc/powerpc32/e500: Recursively delete.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/e500: Recursively delete.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/context-e500.h:
Delete.
* sysdeps/powerpc/fpu_control.h: Remove SPE variant.
Issue an #error if used with a compiler in SPE-float mode.
* sysdeps/powerpc/powerpc32/__longjmp_common.S
* sysdeps/powerpc/powerpc32/setjmp_common.S
* sysdeps/unix/sysv/linux/powerpc/powerpc32/getcontext-common.S
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/getcontext.S
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/setcontext.S
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/swapcontext.S
* sysdeps/unix/sysv/linux/powerpc/powerpc32/setcontext-common.S
* sysdeps/unix/sysv/linux/powerpc/powerpc32/swapcontext-common.S:
Remove code to preserve SPE register state.
* sysdeps/unix/sysv/linux/powerpc/elision-lock.c
* sysdeps/unix/sysv/linux/powerpc/elision-trylock.c
* sysdeps/unix/sysv/linux/powerpc/elision-unlock.c
Remove __SPE__ ifndefs.
A few of our installed headers contain UTF-8 in comments.
check-obsolete-constructs opened files without explicitly specifying
their encoding, so it would barf on these headers if “make check” was
run in a non-UTF-8 locale.
* scripts/check-obsolete-constructs.py (HeaderChecker.check):
Specify encoding="utf-8" when opening headers to check.
This patch makes build-many-glibcs.py use Linux 5.0 in place of 4.20
(now that the test change required to avoid false positives with ulong
in kernel headers has been committed). This includes adjusting the
logic to compute a tarball URL to handle different major version
numbers (rather than changing the path to hardcode v5.x in place of
v4.x, as someone might still wish to check out a v4.x version).
Tested that build-many-glibcs.py successfully checks out Linux 5.0
sources after this patch.
* scripts/build-many-glibcs.py (Context.checkout): Default Linux
version to 5.0.
(Context.checkout_tar): Handle variable major version for Linux
kernel.
The test for obsolete typedefs in installed headers was implemented
using grep, and could therefore get false positives on e.g. “ulong”
in a comment. It was also scanning all of the headers included by
our headers, and therefore testing headers we don’t control, e.g.
Linux kernel headers.
This patch splits the obsolete-typedef test from
scripts/check-installed-headers.sh to a separate program,
scripts/check-obsolete-constructs.py. Being implemented in Python,
it is feasible to make it tokenize C accurately enough to avoid false
positives on the contents of comments and strings. It also only
examines $(headers) in each subdirectory--all the headers we install,
but not any external dependencies of those headers. Headers whose
installed name starts with finclude/ are ignored, on the assumption
that they contain Fortran.
It is also feasible to make the new test understand the difference
between _defining_ the obsolete typedefs and _using_ the obsolete
typedefs, which means posix/{bits,sys}/types.h no longer need to be
exempted. This uncovered an actual bug in bits/types.h: __quad_t and
__u_quad_t were being used to define __S64_TYPE, __U64_TYPE,
__SQUAD_TYPE and __UQUAD_TYPE. These are changed to __int64_t and
__uint64_t respectively. This is a safe change, despite the comments
in bits/types.h claiming a difference between __quad_t and __int64_t,
because those comments are incorrect. In all current ABIs, both
__quad_t and __int64_t are ‘long’ when ‘long’ is a 64-bit type, and
‘long long’ when ‘long’ is a 32-bit type, and similarly for __u_quad_t
and __uint64_t. (Changing the types to be what the comments say they
are would be an ABI break, as it affects C++ name mangling.) This
patch includes a minimal change to make the comments not completely
wrong.
sys/types.h was defining the legacy BSD u_intN_t typedefs using a
construct that was not necessarily consistent with how the C99 uintN_t
typedefs are defined, and is also too complicated for the new script to
understand (it lexes C relatively accurately, but it does not attempt
to expand preprocessor macros, nor does it do any actual parsing).
This patch cuts all of that out and uses bits/types.h's __uintN_t typedefs
to define u_intN_t instead. This is verified to not change the ABI on
any supported architecture, via the c++-types test, which means u_intN_t
and uintN_t were, in fact, consistent on all supported architectures.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* scripts/check-obsolete-constructs.py: New test script.
* scripts/check-installed-headers.sh: Remove tests for
obsolete typedefs, superseded by check-obsolete-constructs.py.
* Rules: Run scripts/check-obsolete-constructs.py over $(headers)
as a special test. Update commentary.
* posix/bits/types.h (__SQUAD_TYPE, __S64_TYPE): Define as __int64_t.
(__UQUAD_TYPE, __U64_TYPE): Define as __uint64_t.
Update commentary.
* posix/sys/types.h (__u_intN_t): Remove.
(u_int8_t): Typedef using __uint8_t.
(u_int16_t): Typedef using __uint16_t.
(u_int32_t): Typedef using __uint32_t.
(u_int64_t): Typedef using __uint64_t.
The check for "/finclude/" fails with the actual location of
Fortran headers because they are now stored in the "finclude"
subdirectory of the top-level include directory, so a relative path
does not contain a slash '/' before the "finclude" string.
If building on a subset of architectures only, it is easy to miss
wrapper headers which are required by other architectures because
they lack the corresponding sysdeps header. The check ensures
that every installed header which is not itself a sysdeps header
has a header under include/ (that presumably wraps the header,
and perhaps also adding declarations and definitions for !_ISOMAC).
Also check for the absence of the sysdeps/generic/bits directory
removed in commit c72565e5f1, to make
accidental re-introduction more difficult.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
In some cases, sensitive to readline version and the user's
environment, gdb might emit escape codes while run under python's
pexpect (i.e. testing pretty printers). This patch, suggested
by Jan, helps isolate the test from the user's environment.
Tested on RHEL 7 x86_64 with DTS 7 and EPEL, which is one
magic combination of components that triggers this bug.
This patch updates some miscellaneous files from their upstream
sources (thereby bringing in copyright date updates for some of those
files).
Tested for x86_64, including "make pdf".
* manual/texinfo.tex: Update to version 2018-12-28.17 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2019-01-01.
* scripts/config.sub: Update to version 2019-01-01.
* scripts/move-if-change: Update from gnulib.
Continuing the process of building up and using Python infrastructure
for extracting and using values in headers, this patch adds a test
that MAP_* constants from sys/mman.h agree with those in the Linux
kernel headers. (Other sys/mman.h constants could be added to the
test separately.)
This set of constants has grown over time, so the generic code is
enhanced to allow saying extra constants are OK on either side of the
comparison (where the caller sets those parameters based on the Linux
kernel headers version, compared with the version the headers were
last updated from). Although the test is a custom Python file, my
intention is to move in future to a single Python script for such
tests and text files it takes as inputs, once there are enough
examples to provide a guide to the common cases in such tests (I'd
like to end up with most or all such sets of constants copied from
kernel headers having such tests, and likewise for structure layouts
from the kernel).
The Makefile code is essentially the same as for tst-signal-numbers,
but I didn't try to find an object file to depend on to represent the
dependency on the headers used by the test (the conform/ tests don't
try to represent such header dependencies at all, for example).
Tested with build-many-glibcs.py, and also for x86_64 with older
kernel headers.
* scripts/glibcextract.py (compare_macro_consts): Take parameters
to allow extra macros from first or second sources.
* sysdeps/unix/sysv/linux/tst-mman-consts.py: New file.
* sysdeps/unix/sysv/linux/Makefile [$(subdir) = misc]
(tests-special): Add $(objpfx)tst-mman-consts.out.
($(objpfx)tst-mman-consts.out): New makefile target.
This patch eliminates the gen-py-const.awk variant of gen-as-const,
switching to use of gnu-as-const.py (with a new --python option) to
process .pysym files (i.e., to generate nptl_lock_constants.py), as
the syntax of those files is identical to that of .sym files.
Note that the generated nptl_lock_constants.py is *not* identical to
the version generated by the awk script. Apart from the trivial
changes (comment referencing the new script, and output being sorted),
the constant FUTEX_WAITERS, PTHREAD_MUTEXATTR_FLAG_BITS,
PTHREAD_MUTEXATTR_FLAG_PSHARED and PTHREAD_MUTEX_PRIO_CEILING_MASK are
now output as positive rather than negative constants (on x86_64
anyway; maybe not necessarily on 32-bit systems):
< FUTEX_WAITERS = -2147483648
---
> FUTEX_WAITERS = 2147483648
< PTHREAD_MUTEXATTR_FLAG_BITS = -251662336
< PTHREAD_MUTEXATTR_FLAG_PSHARED = -2147483648
---
> PTHREAD_MUTEXATTR_FLAG_BITS = 4043304960
> PTHREAD_MUTEXATTR_FLAG_PSHARED = 2147483648
< PTHREAD_MUTEX_PRIO_CEILING_MASK = -524288
---
> PTHREAD_MUTEX_PRIO_CEILING_MASK = 4294443008
This is because gen-as-const has a cast of the constant value to long
int, which gen-py-const lacks.
I think the positive values are more logically correct, since the
constants in question are in fact unsigned in C. But to reliably
produce gen-as-const.py output for constants that always (in C and
Python) reflects the signedness of values with the high bit of "long
int" set would mean more complicated logic needs to be used in
computing values.
The more correct positive values by themselves produce a failure of
nptl/test-mutexattr-printers, because masking with
~PTHREAD_MUTEXATTR_FLAG_BITS & ~PTHREAD_MUTEX_NO_ELISION_NP now leaves
a bit -1 << 32 in the Python value, resulting in a KeyError exception.
To avoid that, places masking with ~ of one of the constants in
question are changed to mask with 0xffffffff as well (this reflects
how ~ in Python applies to an infinite-precision integer whereas ~ in
C does not do any promotions beyond the width of int).
Tested for x86_64.
* scripts/gen-as-const.py (main): Handle --python option.
* scripts/gen-py-const.awk: Remove.
* Makerules (py-const-script): Use gen-as-const.py.
($(py-const)): Likewise.
* nptl/nptl-printers.py (MutexPrinter.read_status_no_robust): Mask
with 0xffffffff together with ~(PTHREAD_MUTEX_PRIO_CEILING_MASK).
(MutexAttributesPrinter.read_values): Mask with 0xffffffff
together with ~PTHREAD_MUTEXATTR_FLAG_BITS and
~PTHREAD_MUTEX_NO_ELISION_NP.
* manual/README.pretty-printers: Update reference to
gen-py-const.awk.
This patch converts the tst-signal-numbers test from shell + awk to
Python.
As with gen-as-const, the point is not so much that shell and awk are
problematic for this code, as that it's useful to build up general
infrastructure in Python for use of a range of code involving
extracting values from C headers. This patch moves some code from
gen-as-const.py to a new glibcextract.py, which also gains functions
relating to listing macros, and comparing the values of a set of
macros from compiling two different pieces of code.
It's not just signal numbers that should have such tests; pretty much
any case where glibc copies constants from Linux kernel headers should
have such tests that the values and sets of constants agree except
where differences are known to be OK. Much the same also applies to
structure layouts (although testing those without hardcoding lists of
fields to test will be more complicated).
Given this patch, another test for a set of macros would essentially
be just a call to glibcextract.compare_macro_consts (plus boilerplate
code - and we could move to having separate text files defining such
tests, like the .sym inputs to gen-as-const, so that only a single
Python script is needed for most such tests). Some such tests would
of course need new features, e.g. where the set of macros changes in
new kernel versions (so you need to allow new macro names on the
kernel side if the kernel headers are newer than the version known to
glibc, and extra macros on the glibc side if the kernel headers are
older). tst-syscall-list.sh could become a Python script that uses
common code to generate lists of macros but does other things with its
own custom logic.
There are a few differences from the existing shell + awk test.
Because the new test evaluates constants using the compiler, no
special handling is needed any more for one signal name being defined
to another. Because asm/signal.h now needs to pass through the
compiler, not just the preprocessor, stddef.h is included as well
(given the asm/signal.h issue that it requires an externally provided
definition of size_t). The previous code defined __ASSEMBLER__ with
asm/signal.h; this is removed (__ASSEMBLY__, a different macro,
eliminates the requirement for stddef.h on some but not all
architectures).
Tested for x86_64, and with build-many-glibcs.py.
* scripts/glibcextract.py: New file.
* scripts/gen-as-const.py: Do not import os.path, re, subprocess
or tempfile. Import glibcexctract.
(compute_c_consts): Remove. Moved to glibcextract.py.
(gen_test): Update reference to compute_c_consts.
(main): Likewise.
* sysdeps/unix/sysv/linux/tst-signal-numbers.py: New file.
* sysdeps/unix/sysv/linux/tst-signal-numbers.sh: Remove.
* sysdeps/unix/sysv/linux/Makefile
($(objpfx)tst-signal-numbers.out): Use tst-signal-numbers.py.
Redirect stderr as well as stdout.
This patch updates various miscellaneous files from their upstream
sources.
Tested for x86_64, including "make pdf".
* manual/texinfo.tex: Update to version 2018-09-21.20 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2018-11-28.
* scripts/config.sub: Update to version 2018-11-28.
* scripts/install-sh: Update to version 2018-03-11.20.
* scripts/mkinstalldirs: Update to version 2018-03-07.03.
* scripts/move-if-change: Update to version 2018-03-07 03:47.
It was reported in
<https://sourceware.org/ml/libc-alpha/2018-12/msg00045.html> that
gen-as-const.py fails to generate test code in the case where a .sym
file has no symbols in it, so resulting in a test failing to link for
Hurd.
The relevant difference from the old awk script is that the old script
treated '--' lines as indicating that the text to do at the start of
the test (or file used to compute constants) should be output at that
point if not already output, as well as treating lines with actual
entries for constants like that. This patch changes gen-as-const.py
accordingly, making it the sole responsibility of the code parsing
.sym files to determine when such text should be output and ensuring
it's always output at some point even if there are no symbols and no
'--' lines, since not outputting it means the test fails to link.
Handling '--' like that also avoids any problems that would arise if
the first entry for a symbol were inside #ifdef (since the text in
question must not be output inside #ifdef).
Tested for x86_64, and with build-many-glibcs.py for i686-gnu. Note
that there are still compilation test failures for i686-gnu
(linknamespace tests, possibly arising from recent posix_spawn-related
changes).
* scripts/gen-as-const.py (compute_c_consts): Take an argument
'START' to indicate that start text should be output.
(gen_test): Likewise.
(main): Generate 'START' for first symbol or '--' line, or at end
of input if not previously generated.
hurd's jmp_buf-ssp.sym does not define any symbol.
scripts/gen-as-const.py currently was emitting an empty line in that
case, and the gawk invocation was prepending "asconst_" to it, ending up
with:
.../build/glibc/setjmp/test-as-const-jmp_buf-ssp.c:1:2: error: expected « = », « , », « ; », « asm » or
« __attribute__ » at end of input
1 | asconst_
| ^~~~~~~~
* scripts/gen-as-const.py (main): Avoid emitting empty line when
there is no element in `consts'.
This patch replaces gen-as-const.awk, and some fragments of the
Makefile code that used it, by a Python script. The point is not such
much that awk is problematic for this particular script, as that I'd
like to build up a general Python infrastructure for extracting
information from C headers, for use in writing tests of such headers.
Thus, although this patch does not set up such infrastructure, the
compute_c_consts function in gen-as-const.py might be moved to a
separate Python module in a subsequent patch as a starting point for
such infrastructure.
The general idea of the code is the same as in the awk version, but no
attempt is made to make the output files textually identical. When
generating a header, a dict of constant names and values is generated
internally then defines are printed in sorted order (rather than the
order in the .sym file, which would have been used before). When
generating a test that the values computed match those from a normal
header inclusion, the test code is made into a compilation test using
_Static_assert, where previously the comparisons were done only when
the test was executed. One fragment of test generation (converting
the previously generated header to use asconst_* prefixes on its macro
names) is still in awk code in the makefiles; only the .sym processing
and subsequent execution of the compiler to extract constants have
moved to the Python script.
Tested for x86_64, and with build-many-glibcs.py.
* scripts/gen-as-const.py: New file.
* scripts/gen-as-const.awk: Remove.
* Makerules ($(common-objpfx)%.h $(common-objpfx)%.h.d): Use
gen-as-const.py.
($(objpfx)test-as-const-%.c): Likewise.
These files were both auto-generated and shipped in the source tree.
We can assume that sed is available and always generate the files
during the build.
Now that build-many-glibcs.py touches at checkout time all files that
might get rebuilt in the glibc source directory in a normal glibc
build and test run, this patch stops the script from copying the glibc
source directory, so that all builds use the original directory
directly (and less disk space is used, less I/O is involved and cached
copies of the sources in memory can be shared between all the builds -
as well as avoiding spurious failures from copying while "git gc" is
running). This is similar to how all other components were already
handled. Any bugs involving writing into the source directory can be
dealt with in future as normal bugs, just as such bugs already are
handled.
Tested with build-many-glibcs.py runs with a read-only glibc source
directory, with all files not touched by the script having timestamps
in forwards alphabetical order and separately with all files not
touched by the script having timestamps in backwards alphabetical
order.
* scripts/build-many-glibcs.py (Glibc.build_glibc): Use original
source directory instead of a copy.
(CommandList.create_copy_dir): Remove.
Mathieu Desnoyers ran into an issue with his rseq patch where he
was the first person to add weak thread-local data and this
resulted in an ABI list update with entries like this:
"GLIBC_2.29 w ? D .tdata 0000000000000020".
The weakness of the symbol has nothing to do with the DSOs ABI
and so we should not write anything about weak symbols here. The
.tdata entries should be treated exactly like .tbss entries and
the output should have been: "GLIBC_2.29 __rseq_abi T 0x20"
This change makes abilist.awk handle .tdata just like .tbss,
while at the same time adding an error case for the default, and
the unknown line cases. We never want anyone to be able to add
such entries to any ABI list files and should see an immediate
error and consult with experts.
Tested by Mathieu Desnoyers <mathieu.desnoyers@efficios.com> with
the rseq patch set and 'make update-all-abi'.
Tested myself with 'make update-all-abi' on x86_64 with no
changes.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
build-many-glibcs.py currently copies the source tree to avoid issues
with parallel builds trying to write into it. This copying can result
in occasional spurious build failures from bots, when a "git gc" is in
progress that changes .git contents while copying is taking place, and
it would also be desirable to avoid the need to copy to save on disk
space, I/O and memory used in build-many-glibcs.py builds.
In preparation for removing the copying, this patch arranges for
build-many-glibcs.py to touch more files on checkout so their
timestamps do not result in make attempting to rebuild them. Before
actually removing the copying, I intend to do further tests to ensure
I haven't missed any other such makefile dependencies.
This is of course without prejudice to possibly moving more of these
files to being generated in the build directory rather than being
checked in at all, where that can be done using build tools already
required for the build. For sysdeps files (installed and otherwise)
it would be necessary to make sure this does not affect the search
ordering, for headers used in the build it would be necessary to
ensure they are generated early enough, and for errlist.c there may be
dual licensing reasons for keeping it checked in.
Tested that a checkout with build-many-glibcs.py does touch the
expected files and that a glibcs build for aarch64-linux-gnu succeeds.
* scripts/build-many-glibcs.py (Context.fix_glibc_timestamps):
Touch additional files.
Since we have consensus on requiring Python 3.4 or later to build
glibc, it follows that compatibility with older Python versions is
also no longer relevant to auxiliary Python scripts for use in glibc
development. This patch removes such compatibility code from
build-many-glibcs.py (compatibility code needed for 3.4, which lacks
the newer subprocess interface, is kept). Because
build-many-glibcs.py is not itself called from the glibc build system,
this patch is independent of the configure checks for having a
new-enough Python version, which are only relevant to uses of Python
from the main build and test process.
Tested with build-many-glibcs.py building glibc for aarch64-linux-gnu
(with Python 3.4 to make sure that still works).
* scripts/build-many-glibcs.py: Remove compatibility for missing
os.cpu_count and re.fullmatch.
For architectures and ABIs that are added in version 2.29 or later the
option --enable-obsolete-nsl is no longer available, and no libnsl
compatibility library is built.
Every so often we get libsanitizer or libgo builds breaking with new
glibc because of some change in the glibc headers.
glibc's build-many-glibcs.py deliberately disables libsanitizer and
GCC languages other than C and C++ because the point is to test glibc
and find glibc problems (including problems shown up by new compiler
warnings in new GCC), not to test libsanitizer or libgo; if the
compiler build fails because of libsanitizer or libgo failing to
build, that could hide the existence of new problems in glibc.
However, it seems reasonable to have a non-default mode where
build-many-glibcs.py does build those additional pieces, which this
patch adds.
Note that I do not intend to run a build-many-glibcs.py bot with this
new option. If people concerned with libsanitizer, libgo or other
potentially affected GCC libraries wish to find out about such
problems more quickly, they may wish to run such a bot or bots (and to
monitor the results and fix issues found - obviously there will be
some overlap with issues found by my bots not using that option).
Note also that building a non-native Ada compiler requires a
sufficiently recent native (or build-x-host, in general) Ada compiler
to be used, possibly more or less the same version as being built.
That needs to be in the PATH when build-many-glibcs.py --full-gcc is
run; the script does not deal with setting up such a compiler (or any
of the other host tools needed for building GCC and glibc, beyond the
GMP / MPFR / MPC libraries), but perhaps it should, to avoid the need
to keep updating such a compiler manually when running a bot.
Tested by running build-many-glibcs.py with the new option, with
mainline GCC. There are build failures for various configurations,
which may be of interest to Go / Ada people even if you're not
interested in running such a bot:
* mips64 / mips64el (all configuration): ICE building libstdc++, as
seen without using the new option
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87156>.
* aarch64_be: error building libgo (little-endian aarch64 works fine):
version.go:67:13: error: expected ';' or ')' or newline
67 | BigEndian =
| ^
version.go:67:3: error: reference to undefined name 'BigEndian'
67 | BigEndian =
| ^
* arm (all configurations): error building libgo:
/scratch/jmyers/glibc/many9/src/gcc/libgo/go/internal/syscall/unix/getrandom_linux.go:29:5: error: reference to undefined name 'randomTrap'
29 | if randomTrap == 0 {
| ^
/scratch/jmyers/glibc/many9/src/gcc/libgo/go/internal/syscall/unix/getrandom_linux.go:38:34: error: reference to undefined name 'randomTrap'
38 | r1, _, errno := syscall.Syscall(randomTrap,
| ^
What's happening there is, I think, that the arm*b*-*-* case in
libgo/configure.ac is wrongly matching arm-glibc-linux-gnueabi with
the 'b' in the vendor part, and then something else is failing to
handle GOARCH=armbe. Given that you can have configurations with
multilibs of both endiannesses, endianness should always be detected
by configure.ac, for all architectures, using a compile test of
whether __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__, not based on textual
matches to the host (= target at top-level) triplet.
* armeb (all configurations): error building libada (for some reason
the Arm libada configuration seems to do different things for EH for
big-endian, which makes no sense to me and doesn't actually work):
a-exexpr.adb:87:06: "System.Exceptions.Machine" is not a predefined library unit
a-exexpr.adb:87:06: "Ada.Exceptions (body)" depends on "Ada.Exceptions.Exception_Propagation (body)"
a-exexpr.adb:87:06: "Ada.Exceptions.Exception_Propagation (body)" depends on "System.Exceptions.Machine (spec)"
* hppa: error building libgo (same error as for aarch64_be).
* ia64: ICE building libgo. I've filed
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87281> for this.
* m68k: ICE in the Go front end building libgo
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84948>.
* microblaze, microblazeel, nios2, sh3, sh3eb: build failure in libada
for lack of a libada port to those systems (I'm not sure sh3 would
actually need anything different from sh4):
a-cbdlli.ads:38:14: violation of restriction "No_Finalization" at system.ads:47
* i686-gnu: build failure in libada, might be fixed by the patch
attached to <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81103>
(not tested):
terminals.c:1115:13: fatal error: termio.h: No such file or directory
* scripts/build-many-glibcs.py (Context.__init__): Add full_gcc
argument.
(Config.build_gcc): Use --disable-libsanitizer for first GCC
build, but not for second build if --full-gcc. Use
--enable-languages=all for second build if --full-gcc.
(get_parser): Add --full-gcc option.
(main): Update call to Context.
We've had issues before with build failures (with new GCC) in code
only built with --enable-obsolete-rpc or --enable-obsolete-nsl not
being reported for a while because build-many-glibcs.py does not test
those configure options. This patch adds configurations (32-bit and
64-bit) using those options so that in future we can notice quickly if
they start failing to build.
Tested the new configurations do build with GCC 8.
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
x86_64 and i686 configs using --enable-obsolete-rpc
--enable-obsolete-nsl.
* scripts/check-execstack.awk: Consider `xfail' variable containing a
list
of libraries whose stack executability is expected.
* elf/Makefile ($(objpfx)check-execstack.out): Pass
$(check-execstack-xfail) to check-execstack.awk through `xfail'
variable.
* sysdeps/mach/hurd/i386/Makefile (check-execstack-xfail): Set to ld.so
libc.so libpthread.so.
As noted in bug 13888, and as I noted previously in
<https://sourceware.org/ml/libc-alpha/2000-10/msg00111.html>, various
tests used hardcoded paths in /tmp, so posing issues for simultaneous
test runs from different build directories.
This patch fixes such uses of hardcoded file names to put them in the
build directory instead (in the case of stdio-common/bug5 the file
names are changed as well, to avoid a conflict with the name bug5.out
also used for the automatic test output redirection). It also fixes
test-installation.pl likewise (that was using filenames with $$ in
them rather than strictly hardcoded names, but that's still not good
practice for temporary file naming).
Note that my list of files changed is not identical to that in bug
13888. I added tst-spawn3.c and test-installation.pl, and removed
some tests that seem to me (now) to create temporary files securely
(simply using /tmp is not itself a problem if the temporary files are
handled properly with mkstemp; I haven't checked whether those tests
used to do things insecurely). conformtest is not changed because the
makefiles always pass a --tmpdir option so the /tmp default is
irrelevant, and for the same reason there is no actual problem with
nptl/tst-umask1.c because again the makefiles always override the
default.
nptl/sockperf.c is ignored because there is no code to run it;
probably that file should actually be removed.
Some tests use the mktemp function, but I think they all use it in a
way that *is* secure (for generating names for directories / sockets /
fifos / symlinks, where the operation using the name will not follow
symlinks and so there is no potential for a symlink attack on the
account running the testsuite).
Some tests use the tmpnam function to generate temporary file names.
This is in principle insecure, but not addressed by this patch (I
consider it a separate issue from the fully hardcoded paths).
Tested for x86_64.
[BZ #13888]
* posix/Makefile (CFLAGS-tst-spawn3.c): New variable.
* posix/tst-spawn3.c (do_test): Put tst-spwan3.pid in OBJPFX, not
/tmp.
* scripts/test-installation.pl: Put temporary files in build
directory, not /tmp.
* stdio-common/Makefile (CFLAGS-bug3.c): New variable.
(CFLAGS-bug4.c): Likewise.
(CFLAGS-bug5.c): Likewise.
(CFLAGS-test-fseek.c): Likewise.
(CFLAGS-test-popen.c): Likewise.
(CFLAGS-test_rdwr.c): Likewise.
* stdio-common/bug3.c (main): Put temporary file in OBJPFX, not
/tmp.
* stdio-common/bug4.c (main): Likewise.
* stdio-common/bug5.c (main): Likewise.
* stdio-common/test-fseek.c (TESTFILE): Likewise.
* stdio-common/test-popen.c (do_test): Likewise.
* stdio-common/test_rdwr.c (main): Likewise.
Commit b289cd9db8 (“Ignore absolute
symbols in ABI tests.”) broke “make update-all-abi” because an empty
list of files is now passed to scripts/update-abilist.sh.
_init and _fini are special functions provided by glibc for linker to
define DT_INIT and DT_FINI in executable and shared library. They
should never be put in dynamic symbol table. This patch marks them as
hidden to remove them from dynamic symbol table.
Tested with build-many-glibcs.py.
[BZ #23145]
* elf/Makefile (tests-special): Add $(objpfx)check-initfini.out.
($(all-built-dso:=.dynsym): New target.
(common-generated): Add $(all-built-dso:$(common-objpfx)%=%.dynsym).
($(objpfx)check-initfini.out): New target.
(generated): Add check-initfini.out.
* scripts/check-initfini.awk: New file.
* sysdeps/aarch64/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/alpha/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/arm/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/hppa/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/i386/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/ia64/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/m68k/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/microblaze/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/mips/mips32/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/mips/mips64/n32/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/mips/mips64/n64/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/nios2/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/powerpc/powerpc32/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/powerpc/powerpc64/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/s390/s390-32/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/s390/s390-64/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/sh/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/sparc/crti.S (_init): Mark as hidden.
(_fini): Likewise.
* sysdeps/x86_64/crti.S (_init): Mark as hidden.
(_fini): Likewise.
Since tile support has been removed from the Linux kernel for 4.17,
this patch removes the (unmaintained) port to tilegx from glibc (the
tilepro support having been previously removed). This reflects the
general principle that a glibc port needs upstream support for the
architecture in all the components it build-depends on (so binutils,
GCC and the Linux kernel, for the normal case of a port supporting the
Linux kernel but no other OS), in order to be maintainable.
Apart from removal of sysdeps/tile and sysdeps/unix/sysv/linux/tile,
there are updates to various comments referencing tile for which
removal of those references seemed appropriate. The configuration is
removed from README and from build-many-glibcs.py. contrib.texi keeps
mention of removed contributions, but I updated Chris Metcalf's entry
to reflect that he also contributed the non-removed support for the
generic Linux kernel syscall interface.
__ASSUME_FADVISE64_64_NO_ALIGN support is removed, as it was only used
by tile.
* sysdeps/tile: Remove.
* sysdeps/unix/sysv/linux/tile: Likewise.
* README (tilegx-*-linux-gnu): Remove from list of supported
configurations.
* manual/contrib.texi (Contributors): Mention Chris Metcalf's
contribution of support for generic Linux kernel syscall
interface.
* scripts/build-many-glibcs.py (Context.add_all_configs): Remove
tilegx configurations.
(Config.install_linux_headers): Do not handle tile.
* sysdeps/unix/sysv/linux/aarch64/ldsodefs.h: Do not mention Tile
in comment.
* sysdeps/unix/sysv/linux/nios2/Makefile: Likewise.
* sysdeps/unix/sysv/linux/posix_fadvise.c: Likewise.
[__ASSUME_FADVISE64_64_NO_ALIGN] (__ALIGNMENT_ARG): Remove
conditional undefine and redefine.
* sysdeps/unix/sysv/linux/posix_fadvise64.c: Do not mention Tile
in comment.
[__ASSUME_FADVISE64_64_NO_ALIGN] (__ALIGNMENT_ARG): Remove
conditional undefine and redefine.
Now that GCC 8 has branched, this patch makes build-many-glibcs.py
default to using GCC 8 branch instead of GCC 7. The effect should be
that all builds complete cleanly and the compilation parts of the
glibc testsuite complete cleanly except for on i686-gnu (with GCC 7
there were testsuite failures for some other configurations as well).
I've replaced my bot building using GCC 6 branch with one using GCC 8
branch. (Of course glibc should continue building with GCC 6 - and
for that matter GCC 5 and 4.9, which are no longer maintained, are
supported versions as well - but building with GCC 6 will no longer be
included in my bot testing.)
* scripts/build-many-glibcs.py (Context.checkout): Default GCC
version to GCC 8 branch.
If e.g. the testcase nptl/test-mutex-printers is run
with enabled lock-elision, it fails on s390x with:
Error: Response does not match the expected pattern.
Command: print *mutex
Expected pattern: pthread_mutex_t
Response: No symbol "mutex" in current context.
(gdb)
See https://www.sourceware.org/ml/libc-alpha/2018-03/msg00583.html
for more details.
In fact the mutex pretty printer tests rely on looking at the
internal details of the lock, thus we disable it by setting up
the GLIB_TUNABLES environment variable inside gdb.
ChangeLog:
* scripts/test_printers_common.py (init_test): Disable lock elision.
The powerpcspe GCC port has been obsoleted in GCC 8 for not having had
the removal of code for non-SPE processors completed. This patch
accordingly arranges for build-many-glibcs.py to configure GCC with
--enable-obsolete for affected configurations. This is temporary;
either the port gets cleaned up and unobsoleted in GCC and the
configure option can be removed, or the port gets removed in GCC and
we should remove the corresponding glibc support.
Tested with build-many-glibcs.py for the affected configurations.
* scripts/build-many-glibcs.py (Context.add_all_configs): Use
--enable-obsolete for powerpc-linux-gnuspe.
mig_support does not actually inline the stpncpy any more.
* mach/mach/mig_support.h [defined __USE_GNU]: Do not #error out.
* scripts/check-installed-headers.sh: Do not ignore Hurd and Mach
headers.
We shipped 2.27 with libio.h and _G_config.h still installed but
issuing warnings when used. Let's stop installing them early in 2.28
so that we have plenty of time to think of another plan if there are
problems.
The public stdio.h had a genuine dependency on libio.h for the
complete definitions of FILE and cookie_io_functions_t, and a genuine
dependency on _G_config.h for the complete definitions of fpos_t and
fpos64_t; these are moved to single-type headers.
bits/types/struct_FILE.h also provides a handful of accessor and
bitflags macros so that code is not duplicated between bits/stdio.h
and libio.h. All the other _IO_ and _G_ names used by the public
stdio.h can be replaced with either public names or __-names.
In order to minimize the risk of breaking our own compatibility code,
bits/types/struct_FILE.h preserves the _IO_USE_OLD_IO_FILE mechanism
exactly as it was in libio.h, but you have to define _LIBC to use it,
or it'll error out. Similarly, _IO_lock_t_defined is preserved
exactly, but will error out if used without defining _LIBC.
Internally, include/stdio.h continues to include libio.h, and libio.h
scrupulously provides every _IO_* and _G_* name that it always did,
perhaps now defined in terms of the public names. This is how this
patch avoids touching dozens of files throughout glibc and becoming
entangled with the _IO_MTSAFE_IO mess. The remaining patches in this
series eliminate most of the _G_ names.
Tested on x86_64-linux; in addition to the test suite, I installed the
library in a sysroot and verified that a simple program that uses
stdio.h could be compiled against the installed library, and I also
verified that installed stripped libraries are unchanged.
* libio/bits/types/__fpos_t.h, libio/bits/types/__fpos64_t.h:
New single-type headers split from _G_config.h.
* libio/bits/types/cookie_io_functions_t.h
* libio/bits/types/struct_FILE.h
New single-type headers split from libio.h.
* libio/Makefile: Install the above new headers. Don't install
libio.h, _G_config.h, bits/libio.h, bits/_G_config.h, or
bits/libio-ldbl.h.
* libio/_G_config.h, libio/libio.h: Delete file.
* libio/bits/libio.h: Remove improper-inclusion guard.
Include stdio.h and don't repeat anything that it does.
Define _IO_fpos_t as __fpos_t, _IO_fpos64_t as __fpos64_t,
_IO_BUFSIZ as BUFSIZ, _IO_va_list as __gnuc_va_list,
__io_read_fn as cookie_read_function_t,
__io_write_fn as cookie_write_function_t,
__io_seek_fn as cookie_seek_function_t,
__io_close_fn as cookie_close_function_t,
and _IO_cookie_io_functions_t as cookie_io_functions_t.
Define _STDIO_USES_IOSTREAM, __HAVE_COLUMN, and _IO_file_flags
here, in the "compatibility defines" section. Remove an #if 0
block. Use the "body" macros from bits/types/struct_FILE.h to
define _IO_getc_unlocked, _IO_putc_unlocked, _IO_feof_unlocked,
and _IO_ferror_unlocked.
Move prototypes of __uflow and __overflow...
* libio/stdio.h: ...here. Don't include bits/libio.h.
Don't define _STDIO_USES_IOSTREAM. Get __gnuc_va_list
directly from stdarg.h. Include bits/types/__fpos_t.h,
bits/types/__fpos64_t.h, bits/types/struct_FILE.h,
and, when __USE_GNU, bits/types/cookie_io_functions_t.h.
Use __gnuc_va_list, not _G_va_list; __fpos_t, not _G_fpos_t;
__fpos64_t, not _G_fpos64_t; FILE, not struct _IO_FILE;
cookie_io_functions_t, not _IO_cookie_io_functions_t;
__ssize_t, not _IO_ssize_t. Unconditionally define
BUFSIZ as 8192 and EOF as (-1).
* libio/bits/stdio.h: Add multiple-include guard. Use the "body"
macros from bits/types/struct_FILE.h instead of _IO_* macros
from libio.h; use __gnuc_va_list instead of va_list and __ssize_t
instead of _IO_ssize_t.
* libio/bits/stdio2.h: Similarly.
* libio/iolibio.h: Add multiple-include guard.
Include bits/libio.h after stdio.h.
* libio/libioP.h: Add multiple-include guard.
Include stdio.h and bits/libio.h before iolibio.h.
* include/bits/types/__fpos_t.h, include/bits/types/__fpos64_t.h
* include/bits/types/cookie_io_functions_t.h
* include/bits/types/struct_FILE.h: New wrappers.
* bits/_G_config.h, sysdeps/unix/sysv/linux/_G_config.h:
Get definitions of _G_fpos_t and _G_fpos64_t from
bits/types/__fpos_t.h and bits/types/__fpos64_t.h
respectively. Remove improper-inclusion guards.
* conform/data/stdio.h-data: Update expectations of va_list.
* scripts/check-installed-headers.sh: Remove special case for
libio.h and _G_config.h.
With the git checkouts of Hurd components in build-many-glibcs.py
involving running autoreconf, there's a risk that generated files
could be left behind by an old autoreconf run (if an old version of
the sources generates those files in the source directory but a new
version does not).
This patch avoids that by using git clean -dxfq when updating git
checkouts. In this patch, that's conditional on --replace-sources, to
avoid removing any local not-checked-in files someone may have in
their checkout unless the option has been specifically passed that
says it's OK to blow old checkouts away, complete with any local
changes to them.
* scripts/build-many-glibcs.py (Context.git_checkout): Use git
clean -dxfq for git updates when replacing sources.
The disabling of libcilkrts in build-many-glibcs.py has some
peculiarities. It's only for the final GCC build, not the initial
bootstrap one, whereas normally anything disabled for the final build
should be disabled for the bootstrap one as well. And it's only for
Hurd, when it's more natural by analogy with the libsanitizer
disabling to disable this library unconditionally, not only for
targets where it's known to break. This patch cleans up that
disabling accordingly, adding a comment so it's obvious it can be
removed once GCC 7 is too old to build glibc.
* scripts/build-many-glibcs.py (Config.build_gcc): Use
--disable-libcilkrts unconditionally, not just for the final GCC
build for Hurd.
This patch makes build-many-glibcs.py use Linux 4.15. Other glibc
updates for Linux 4.15 can wait until after the 2.27 release.
* scripts/build-many-glibcs.py (Context.checkout): Default Linux
version to 4.15.
Some warnings need a couple of fixes in the gnumach headers.
* scripts/build-many-glibcs.py (checkout_vcs): Add gnumach
repository URLs, run autoreconf, and make it the default for now.
Some warnings come from code generated by mig, so we need a very recent
version for now.
* scripts/build-many-glibcs.py (checkout_vcs): Add mig repository
URL, and run autoreconf, make it the default for now.
gcc's libcilkrts has never actually supported GNU/Hurd, and doesn't
automatically disable it, and the support was actually removed in gcc trunk,
so that will never actually be fixed there.
* scripts/build-many-glibcs.py [os == gnu] (build_gcc): Pass
--disable-libcilkrts to gcc configure.
Since it turns out soft-float ColdFire has a different glibc ABI to
hard-float ColdFire, as well as various differences in which glibc
code gets built, this patch adds such a configuration to
build-many-glibcs.py to (hopefully) complete the set of ABIs being
tested. (Note that the build for soft-float ColdFire is currently
broken even with GCC mainline - I have a glibc patch to fix this, but
it needs before-and-after build-many-glibcs.py comparison of stripped
binaries for all configurations before being committed.)
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
soft-float ColdFire configuration.
This patch adds build-many-glibcs.py support for GNU Hurd. Builds of
the i686-gnu configuration will fail until sufficient support is
merged to master, so completing build-many-glibcs.py coverage of all
glibc ABIs and making results accurately reflect the broken state of
builds for Hurd.
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
i686-gnu configurations.
(Context.run_builds): Include mig, gnumach and hurd in components
considered.
(Context.checkout): Add mig, gnumach and hurd to components.
(Context.checkout_tar): Add URL mappings for mig, gnumach and
hurd.
(Context.bot_cycle): Check for changes to mig, gnumach and hurd.
(Config.build): Install gnumach headers, build mig and install
hurd headers for 'gnu' OS.
(Config.install_gnumach_headers): New function.
(Config.install_hurd_headers): Likewise.
(Glibc.build_glibc): Do not use /usr for 'gnu' OS. Specifiy MIG
when building for 'gnu' OS.
The RISC-V port will have libraries in subdirectories of lib, like
"lib64/lp64d". This adds support for stripping these installed
libraries.
2018-01-06 Palmer Dabbelt <palmer@sifive.com>
* scripts/build-many-glibcs.py (class Glibc): Strip shared objects
in subdirectories of lib.
This patch updates various files from their upstream sources. This
brings in copyright date updates for some of those files.
Tested for x86_64.
* manual/texinfo.tex: Update to version 2017-12-26.21 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2018-01-01.
* scripts/config.sub: Update to version 2018-01-01.
* scripts/move-if-change: Update from gnulib.
libio.h was originally the header for a set of supported GNU
extensions, but they have not been maintained as such in many years,
they are now standing in the way of improvements to stdio, and we
don't think there are any remaining external users. _G_config.h was
never intended for public use, but predates the bits convention.
Move both of these headers into the bits directory and provide stubs
at top level which issue deprecation warnings.
The contents of (bits/)libio.h and (bits/)_G_config.h are still
exposed to external software via stdio.h; changing that requires more
complex surgery than I have time to attempt right now.
* libio/libio.h, libio/_G_config.h: New stub headers which issue a
deprecation warning and then include <bits/libio.h>, <bits/_G_config.h>
respectively.
* libio/libio.h: Rename the original version of this file to
libio/bits/libio.h. Error out if not included by stdio.h or the
stub libio.h.
* include/libio.h: Move to include/bits. Forward to libio/bits/libio.h.
* sysdeps/generic/_G_config.h: Move to top-level bits/. Error out
if not included by bits/libio.h or the stub _G_config.h.
* sysdeps/unix/sysv/linux/_G_config.h: Move to
sysdeps/unix/sysv/linux/bits. Error out if not included by
bits/libio.h or the stub _G_config.h.
* libio/stdio.h: Include bits/libio.h, not libio.h.
* libio/Makefile: Install bits/libio.h and bits/_G_config.h as
well as libio.h and _G_config.h.
* csu/init.c, libio/fmemopen.c, libio/iolibio.h, libio/oldfmemopen.c
* libio/strfile.h, stdio-common/vfscanf.c
* sysdeps/pthread/flockfile.c, sysdeps/pthread/funlockfile.c
Include stdio.h, not _G_config.h nor libio.h.
* libio/iofgetpos.c: Also rename fgetpos64 out of the way.
* libio/iofsetpos.c: Also rename fsetpos64 out of the way.
* scripts/check-installed-headers.sh: Skip libio.h and _G_config.h.
aarch64 has several ifuncs now so test it without multiarch support separately.
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
disable-multi-arch variant to aarch64-linux-gnu.
This patch updates various miscellaneous files from their upstream
sources.
Tested for x86_64, including "make pdf".
* manual/texinfo.tex: Update to version 2017-12-18.20 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2017-12-17.
* scripts/config.sub: Update to version 2017-11-23.
* scripts/install-sh: Update to version 2017-09-23.17.
* scripts/move-if-change: Update to version 2017-09-13 06:45.
Since the default GCC and binutils versions used by build-many-glibcs.py,
which are GCC 7 branch and binutils 2.29 branch, support static PIE on
x86_64, x32 and i686, this patch adds --enable-static-pie glibc variants
to x86_64, x32 and i686 to get some coverage for static PIE.
Tested with build-many-glibcs.py.
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
--enable-static-pie variants to x86_64, x32 and i686.
My fix to make the arm-linux-gnueabihf build-many-glibcs.py builds
actually use the hard-float ABI as intended showed up another issue
when building with mainline GCC: GCC now determines an FPU based on
the selected CPU or architecture and gives an error for
-mfloat-abi=hard when the CPU does not imply a choice of FPU. This
patch fixes all the affected configurations to specify a suitable
--with-cpu, --with-fpu or -mfpu option explicitly to avoid that error
from GCC.
Tested the relevant configurations with build-many-glibcs.py with
mainline GCC.
* scripts/build-many-glibcs.py (Context.add_all_configs): Specify
CPU or FPU for ARM hard-float configurations.
The conventional configure triplet for ARM GNU/Linux with hard-float
ABI is arm-*-linux-gnueabihf. However, GCC does not automatically use
the hard-float ABI based on that triplet. This patch fixes
build-many-glibcs.py to pass --with-float=hard so that the
arm-linux-gnueabihf configurations actually build with the intended
ABI.
Tested building the affected configurations with build-many-glibcs.py.
* scripts/build-many-glibcs.py (Context.add_all_configs): Use
--with-float=hard for arm-linux-gnueabihf configurations.
There is a configure option --without-fp that specifies that nofpu
sysdeps directories should be used instead of fpu directories.
For most glibc configurations, this option is of no use: either there
is no valid nofpu variant of that configuration, or there are no fpu
or nofpu sysdeps directories for that processor and so the option does
nothing. For a few configurations, if you are using a soft-float
compiler this option is required, and failing to use it generally
results in compilation errors from inline asm using unavailable
floating-point instructions.
We're moving away from --with-cpu to configuring glibc based on how
the compiler generates code, and it is natural to do so for
--without-fp as well; in most cases the soft-float and hard-float ABIs
are incompatible so you have no hope of building a working glibc with
an inappropriately configured compiler or libgcc.
This patch eliminates --without-fp, replacing it entirely by automatic
configuration based on the compiler. Configurations for which this is
relevant (coldfire / mips / powerpc32 / sh) define a variable
with_fp_cond in their preconfigure fragments (under the same
conditions under which those fragments do anything); this is a
preprocessor conditional which the toplevel configure script then uses
in a test to determine which sysdeps directories to use.
The config.make with-fp variable remains. It's used only by powerpc
(sysdeps/powerpc/powerpc32/Makefile) to add -mhard-float to various
flags variables. For powerpc, -mcpu= options can imply use of
soft-float. That could be an issue if you want to build for
e.g. 476fp, but are using --with-cpu=476 because there isn't a 476fp
sysdeps directory. If in future we eliminate --with-cpu and replace
it entirely by testing the compiler, it would be natural at that point
to eliminate that code as well (as the user should then just use a
compiler defaulting to 476fp and the 476 sysdeps directory would be
used automatically).
Tested for x86_64, and tested with build-many-glibcs.py that installed
shared libraries are unchanged by this patch.
* configure.ac (--with-fp): Remove configure option.
(with_fp_cond): New variable.
(libc_cv_with_fp): New configure test. Use this variable instead
of with_fp.
* configure: Regenerated.
* config.make.in (with-fp): Use @libc_cv_with_fp@.
* manual/install.texi (Configuring and compiling): Remove
--without-fp.
* INSTALL: Regenerated.
* sysdeps/m68k/preconfigure (with_fp_cond): Define for ColdFire.
* sysdeps/mips/preconfigure (with_fp_cond): Define.
* sysdeps/powerpc/preconfigure (with_fp_cond): Define for 32-bit.
* sysdeps/sh/preconfigure (with_fp_cond): Define.
* scripts/build-many-glibcs.py (Context.add_all_configs): Do not
use --without-fp to configure glibc.
Since this file is no longer checked in, update-copyrights no longer
needs to avoid changing it.
* scripts/update-copyrights: Do not handle intl/plural.c
specially.
While working on SPARC changes to use libm_alias_* I noticed that the
non-multi-arch sparc32/sparcv9/fpu/s_fabs.S was missing compat symbol
support for fabsl. This clearly shows inadequate test coverage, so
this patch adds SPARC --disable-multi-arch builds to
build-many-glibcs.py (the 32-bit one fails testing until that bug is
fixed, the 64-bit one passes testing).
* scripts/build-many-glibcs.py (Context.add_all_configs): Add
SPARC --disable-multi-arch glibc variants.
This patch adds two extra configuration for arm-linux-gnueabihf to
cover for multiarch support:
1. arm-linux-gnueabihf-v7a: enables multiarch support by using
-march=armv7a.
2. Same as 1. but with --disable-multiarch.
Check with build-many-glibcs.py for both options.
* scripts/build-many-glibcs.py (Context.add_all_configs):
Add arm-linux-gnueabihf multiarch extra_glibcs.
glibc has an add-ons mechanism to allow additional software to be
integrated into the glibc build. Such add-ons may be within the glibc
source tree, or outside it at a path passed to the --enable-add-ons
configure option.
localedata and crypt were once add-ons, distributed in separate
release tarballs, but long since stopped using that mechanism.
Linuxthreads was always an add-on. Ports spent some time as an add-on
with separate release tarballs, then was first moved into the glibc
source tree, then had its sysdeps files moved into the main sysdeps
hierarchy so the add-ons mechanism was no longer used. NPTL spent
some time as an add-on in the main glibc tree before stopping using
the add-on mechanism. libidn used to have separate release tarballs
but no longer does so, but still uses the add-ons mechanism within the
glibc source tree. Various other software has supported building with
the add-ons mechanism at times in the past, but I don't think any is
still widely used.
Add-ons involve significant, little-used complexity in the glibc build
system, and make it hard to understand what the space of possible
glibc configurations is. This patch removes the add-ons mechanism.
libidn is now built via the Subdirs mechanism to cause any
configuration using sysdeps/unix/inet to build libidn; HAVE_LIBIDN
(which effectively means shared libraries are available) is now
defined via sysdeps/unix/inet/configure. Various references to
add-ons around the source tree are removed (in the case of maint.texi,
the example list of sysdeps directories is still very out of date).
Externally maintained ports should now put their files in the normal
sysdeps directory structure rather than being arranged as add-ons;
they probably need to change e.g. elf.h anyway, rather than actually
being able to work just as a drop-in subtree. Hurd libpthread should
be arranged similarly to NPTL, so some files might go in a
hurd-pthreads (or similar) top-level directory in glibc, while sysdeps
files should go in the normal sysdeps directory structure (possibly in
hurd or hurd-pthreads subdirectories, just as there are nptl
subdirectories in the sysdeps tree).
Tested for x86_64, and with build-many-glibcs.py.
* configure.ac (--enable-add-ons): Remove option.
(machine): Do not mention add-ons in comment.
(LIBC_PRECONFIGURE): Likewise.
(add_ons): Remove variable and sanity checks and logic to locate
add-ons.
(add_ons_automatic): Remove variable.
(configured_add_ons): Likewise.
(add_ons_sfx): Likewise.
(add_ons_pfx): Likewise.
(add_on_subdirs): Likewise.
(sysnames_add_ons): Likewise. Remove loop over add-ons and
consideration of add-ons in Implies handling.
(sysdeps_add_ons): Likewise.
* configure: Regenerated.
* libidn/configure.ac: Remove.
* libidn/configure: Likewise.
* sysdeps/unix/inet/configure.ac: New file.
* sysdeps/unix/inet/configure: New generated file.
* sysdeps/unix/inet/Subdirs: Add libidn.
* Makeconfig (sysdeps-srcdirs): Remove variable.
(+sysdep_dirs): Do not include $(sysdeps-srcdirs).
($(common-objpfx)config.status): Do not depend on add-on files.
($(common-objpfx)shlib-versions.v.i): Do not mention add-ons in
comment.
(all-subdirs): Do not include $(add-on-subdirs).
* Makefile (dist-prepare): Do not use $(sysdeps-add-ons).
* config.make.in (add-ons): Remove variable.
(add-on-subdirs): Likewise.
(sysdeps-add-ons): Likewise.
* manual/Makefile (add-chapters): Remove.
($(objpfx)texis): Do not depend on $(add-chapters).
(nonexamples): Do not handle $(add-chapters).
(examples): Do not handle $(add-ons).
(chapters.% top-menu.%): Do not pass '$(add-chapters)' to
libc-texinfo.sh.
* manual/install.texi (Installation): Do not mention add-ons.
(--enable-add-ons): Do not document configure option.
* INSTALL: Regenerated.
* manual/libc-texinfo.sh: Do not handle $2 add-ons argument.
* manual/maint.texi (Hierarchy Conventions): Do not mention
add-ons.
* scripts/build-many-glibcs.py (Glibc.build_glibc): Do not use
--enable-add-ons.
* scripts/gen-sorted.awk: Do not handle Subdirs files from
add-ons.
* scripts/test-installation.pl: Do not handle glibc-compat add-on.
* sysdeps/nptl/Makeconfig: Do not mention add-ons in comment.
When configuring and building GNU libc using the Mozilla NSS library
for cryptography (--enable-nss-crypt option), also include the
NSPR header files along with the Mozilla NSS library header files.
Finally, when running the check-local-headers test, ignore the
Mozilla NSPR library header files (used by the Mozilla NSS library).
Current implementation of tunables does not set arena_max and arena_test
values. Any value provided by glibc.malloc.arena_max and
glibc.malloc.arena_test parameters is ignored.
These tunables have minval value set to 1 (see elf/dl-tunables.list file)
and undefined maxval value. In that case default value (which is 0. see
scripts/gen-tunables.awk) is being used to set maxval.
For instance, generated tunable_list[] entry for arena_max is:
(gdb) p *cur
$1 = {name = 0x7ffff7df6217 "glibc.malloc.arena_max",
type = {type_code = TUNABLE_TYPE_SIZE_T, min = 1, max = 0},
val = {numval = 0, strval = 0x0}, initialized = false,
security_level = TUNABLE_SECLEVEL_SXID_IGNORE,
env_alias = 0x7ffff7df622e "MALLOC_ARENA_MAX"}
As a result, any value of glibc.malloc.arena_max is ignored by
TUNABLE_SET_VAL_IF_VALID_RANGE macro
__type min = (__cur)->type.min; <- initialized to 1
__type max = (__cur)->type.max; <- initialized to 0!
if (min == max) <- false
{
min = __default_min;
max = __default_max;
}
if ((__type) (__val) >= min && (__type) (val) <= max) <- false
{
(__cur)->val.numval = val;
(__cur)->initialized = true;
}
Assigning correct min/max values at a build time fixes a problem.
Plus, a bit of optimization: Setting of default min/max values for the
given type at a run time might be eliminated.
* elf/dl-tunables.c (do_tunable_update_val): Range checking fix.
* scripts/gen-tunables.awk: Set unspecified minval and/or maxval
values to correct default value for given type.
