My glibc bot showed that my conformtest changes fail the build of the
conformtest execution tests for x86_64-linux-gnu-static-pie, because
linking the newly built object with the newly built libc and the
associated options normally used for linking requires it to be built
as PIE. Add $(pie-default) to the compiler command used so that PIE
options are used when required.
There's a case for using the whole of $(CFLAGS-.o) (which includes
$(pie-default)), but that raises questions of any impact from using
optimization flags from CFLAGS in these tests. So for now just use
$(pie-default) as the key part of $(CFLAGS-.o) that's definitely
needed.
Tested with build-many-glibcs.py for x86_64-linux-gnu-static-pie.
Although the conform/ header tests are built using the headers of the
glibc under test, the execution tests from conformtest (a few tests of
the values of macros evaluating to string constants) are linked and
run with system libc, not the newly built libc.
Apart from preventing testing in cross environments, this can be a
problem even for native testing. Specifically, it can be useful to do
native testing when building with a cross compiler that links with a
libc that is not the system libc; for example, on x86_64, you can test
all three ABIs that way if the kernel support is present, even if the
host OS lacks 32-bit or x32 libraries or they are older than the
libraries in the sysroot used by the compiler used to build glibc.
This works for almost all tests, but not for these conformtest tests.
Arrange for conformtest to link and run test programs similarly to
other tests, with consequent refactoring of various variables in
Makeconfig to allow passing relevant parts of the link-time command
lines down to conformtest. In general, the parts of the link command
involving $@ or $^ are separated out from the parts that should be
passed to conformtest (the variables passed to conformtest still
involve various variables whose names involve $(@F), but those
variables simply won't be defined for the conformtest makefile rules
and I think their presence there is harmless).
This is also most of the support that would be needed to allow running
those tests of string constants for cross testing when test-wrapper is
defined. That will also need changes to where conformtest.py puts the
test executables, so it puts them in the main object directory
(expected to be shared with a test system in cross testing) rather
than /tmp (not expected to be shared) as at present.
Tested for x86_64.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
The stub implementations are turned into compat symbols.
Linux actually has two reserved system call numbers (for getpmsg
and putpmsg), but these system calls have never been implemented,
and there are no plans to implement them, so this patch replaces
the wrappers with the generic stubs.
According to <https://bugzilla.redhat.com/show_bug.cgi?id=436349>,
the presence of the XSI STREAMS declarations is a minor portability
hazard because they are not actually implemented.
This commit does not change the TIRPC support code in
sunrpc/rpc_svcout.c. It uses additional XTI functionality and
therefore never worked with glibc.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
conform/Makefile creates $(@D)/scratch for the per-standard per-header
tests. That directory was formerly used by the Perl scripts for
temporary files, but the Python implementations use
tempfile.TemporaryDirectory to get such files cleaned up
automatically. This patch changes the Makefile to create only $(@D)
(required for the output redirection to work), not the scratch
subdirectory.
Tested for x86_64.
* conform/Makefile ($(conformtest-header-tests)): Create $(@D),
not $(@D)/scratch.
($(linknamespace-header-tests)): Likewise.
Continuing the consolidation on Python for various miscellaneous build
and test scripts, this patch moves conformtest from Perl to Python.
The substance of the tests run is intended to be the same as before,
except that the previous test for tags did not actually achieve the
intended purpose of verifying whether a tag was already declared, so
is changed to one that would actually fail for a tag that wasn't
declared, and a typo in the old test for variables being available
($xyzzy instead of xyzzy) would have made that test not use the
correct type (but it would have passed anyway with warnings). No
attempt is made to keep the details of what the test output looks
like; instead, tests are given names which are made to follow PASS: /
FAIL: / XFAIL: / SKIP: / MISSING: as appropriate.
In the new version, there is more consistent parsing of test lines
(into a series of words, either surrounded by {} or separated by
spaces) that applies for all kinds of test lines, rather than the old
approach of different regular expressions for every kind of test. A
few of the conform/data/ files are adjusted so their syntax works with
the new script (which now requires spaces in certain cases where the
old script tolerated them being missing, and does not allow stray
semicolons at the end of "function" lines). Similarly, common logic
is used around what happens with a second subtest if a first one fails
(e.g., a test for a symbol's type if the test for availability fails),
rather than such logic being replicated separately for each kind of
test. Common parsing also applies for test lines both when they are
lines for the header under test and when they are lines for another
header specified with allow-header, again unlike the old script.
Tested for x86_64, and with build-many-glibcs.py.
* conform/conformtest.py: New file.
* conform/conformtest.pl: Remove.
* conform/GlibcConform.pm: Likewise.
* conform/glibcconform.py (KEYWORDS_C90): New constant.
(KEYWORDS_C99): Likewise.
(KEYWORDS): Likewise.
* conform/Makefile ($(conformtest-header-tests)): Use
conformtest.py instead of conformtest.pl. Do not pass --tmpdir
option. Use --header instead of --headers.
* conform/data/arpa/inet.h-data: Remove trailing semicolons on
function entries.
* conform/data/spawn.h-data: Likewise.
* conform/data/fcntl.h-data (openat): Add space after function
name.
* conform/data/wchar.h-data (wcscasecmp): Likewise.
(wcscasecmp_l): Likewise.
* conform/data/termios.h-data (c_cc): Add space after element
name.
This patch replaces conform/linknamespace.pl with a new
conform/linknamespace.py, so continuing the consolidation on Python
instead of Perl for miscellaneous scripts used in building and testing
glibc. The new script follows the same logic as the old one; as a
recently-added script, there were no major cleanups to be made in the
course of the language conversion.
Tested for x86_64, and with build-many-glibcs.py. For x86_64 I also
tested that if the Perl and Python scripts were made to print all the
symbols in seen_where and the paths of symbol references by which
those symbols were linked in, even when those symbols were OK,
identical symbol lists appeared in the output with both versions of
the script (the differences in linknamespace.out files were only in
paths to temporary files in diagnostics for e.g. deprecated functions,
and error output for the expected compilation failures when testing
ndbm.h and varargs.h).
* conform/linknamespace.py: New file.
* conform/linknamespace.pl: Remove file.
* conform/Makefile ($(linknamespace-header-tests)): Use
linknamespace.py instead of linknamespace.pl. Do not use --tmpdir
option.
This patch makes Python 3.4 or later a required tool for building
glibc, so allowing changes of awk, perl etc. code used in the build
and test to Python code without any such changes needing makefile
conditionals or to handle older Python versions.
This patch makes the configure test for Python check the version and
give an error if Python is missing or too old, and removes makefile
conditionals that are no longer needed. It does not itself convert
any code from another language to Python, and does not remove any
compatibility with older Python versions from existing scripts.
Tested for x86_64.
* configure.ac (PYTHON_PROG): Use AC_CHECK_PROG_VER. Set
critic_missing for versions before 3.4.
* configure: Regenerated.
* manual/install.texi (Tools for Compilation): Document
requirement for Python to build glibc.
* INSTALL: Regenerated.
* Rules [PYTHON]: Make code unconditional.
* benchtests/Makefile [PYTHON]: Likewise.
* conform/Makefile [PYTHON]: Likewise.
* manual/Makefile [PYTHON]: Likewise.
* math/Makefile [PYTHON]: Likewise.
Continuing the move of test code from Perl to Python (which seems
uncontroversial, unlike dependencies on Python in the actual build of
glibc), this patch replaces conform/list-header-symbols.pl with a
Python script, as a first step in converting the conform/ tests.
(conform/glibcconform.py is an equivalent to GlibcConform.pm,
containing code that will be relevant to move than one of the conform/
scripts.)
Tested for x86_64, including verifying that the symbol lists generated
are identical to those generated by the Perl version.
* conform/glibcconform.py: New file.
* conform/list-header-symbols.py: Likewise.
* conform/list-header-symbols.pl: Remove.
* conform/Makefile (tests-special): Only add linknamespace tests
if [PYTHON].
($(linknamespace-symlists-tests)): Use list-header-symbols.py.
Remove conformace assumption of NPTL implementation for ISO C threads
and revert wrong libcrypt addition on linknamespace-libs-XPG4.
The i686-gnu target now shows two new conformance failures:
FAIL: conform/ISO11/threads.h/conform
FAIL: conform/ISO11/threads.h/linknamespace
It is expected due missing HTL ISO C threads support and both conformance
.out files indicates the reason ("#error "HTL does not implement ISO C
threads").
Checked on i686-linux-gnu and i686-gnu.
* include/threads.h: Move to ...
* sysdeps/nptl/threads.h: ... here.
* sysdeps/htl/threads.h: New file.
* conform/Makefile (linknamespace-libs-ISO11): Use
static-thread-library instead of linking libpthread.
(linknamespace-libs-XPG4): Revert wrong libcrypt.a addition.
This patch adds the thrd_* definitions from C11 threads (ISO/IEC 9899:2011),
more specifically thrd_create, thrd_curent, rhd_detach, thrd_equal,
thrd_exit, thrd_join, thrd_sleep, thrd_yield, and required types.
Mostly of the definitions are composed based on POSIX conterparts, such as
thrd_t (using pthread_t). For thrd_* function internally direct
POSIX pthread call are used with the exceptions:
1. thrd_start uses pthread_create internal implementation, but changes
how to actually calls the start routine. This is due the difference
in signature between POSIX and C11, where former return a 'void *'
and latter 'int'.
To avoid calling convention issues due 'void *' to int cast, routines
from C11 threads are started slight different than default pthread one.
Explicit cast to expected return are used internally on pthread_create
and the result is stored back to void also with an explicit cast.
2. thrd_sleep uses nanosleep internal direct syscall to avoid clobbering
errno and to handle expected standard return codes. It is a
cancellation entrypoint to be consistent with both thrd_join and
cnd_{timed}wait.
3. thrd_yield also uses internal direct syscall to avoid errno clobbering.
Checked with a build for all major ABI (aarch64-linux-gnu, alpha-linux-gnu,
arm-linux-gnueabi, i386-linux-gnu, ia64-linux-gnu, m68k-linux-gnu,
microblaze-linux-gnu [1], mips{64}-linux-gnu, nios2-linux-gnu,
powerpc{64le}-linux-gnu, s390{x}-linux-gnu, sparc{64}-linux-gnu,
and x86_64-linux-gnu).
Also ran a full check on aarch64-linux-gnu, x86_64-linux-gnu, i686-linux-gnu,
arm-linux-gnueabhf, and powerpc64le-linux-gnu.
[BZ #14092]
* conform/Makefile (conformtest-headers-ISO11): Add threads.h.
(linknamespace-libs-ISO11): Add libpthread.a.
* conform/data/threads.h-data: New file: add C11 thrd_* types and
functions.
* include/stdc-predef.h (__STDC_NO_THREADS__): Remove definition.
* nptl/Makefile (headers): Add threads.h.
(libpthread-routines): Add new C11 thread thrd_create, thrd_current,
thrd_detach, thrd_equal, thrd_exit, thrd_join, thrd_sleep, and
thrd_yield.
* nptl/Versions (libpthread) [GLIBC_2.28]): Add new C11 thread
thrd_create, thrd_current, thrd_detach, thrd_equal, thrd_exit,
thrd_join, thrd_sleep, and thrd_yield symbols.
* nptl/descr.h (struct pthread): Add c11 field.
* nptl/pthreadP.h (ATTR_C11_THREAD): New define.
* nptl/pthread_create.c (START_THREAD_DEFN): Call C11 thread start
routine with expected function prototype.
(__pthread_create_2_1): Add C11 threads check based on attribute
value.
* sysdeps/unix/sysdep.h (INTERNAL_SYSCALL_CANCEL): New macro.
* nptl/thrd_create.c: New file.
* nptl/thrd_current.c: Likewise.
* nptl/thrd_detach.c: Likewise.
* nptl/thrd_equal.c: Likewise.
* nptl/thrd_exit.c: Likewise.
* nptl/thrd_join.c: Likewise.
* nptl/thrd_priv.h: Likewise.
* nptl/thrd_sleep.c: Likewise.
* nptl/thrd_yield.c: Likewise.
* include/threads.h: Likewise.
Some Linux distributions are experimenting with a new, separately
maintained and hopefully more agile implementation of the crypt
API. To facilitate this, add a configure option which disables
glibc's embedded libcrypt. When this option is given, libcrypt.*
and crypt.h will not be built nor installed.
This patch completes the ucontext.h namespace fixes by fixing issues
related to the use of struct sigcontext as mcontext_t, and inclusion
of <bits/sigcontext.h> even when struct sigcontext is not so used.
Inclusion of <bits/sigcontext.h> by <sys/ucontext.h> is removed; the
way to get the sigcontext structure is by including <signal.h> (in a
context where __USE_MISC is defined); the sysdeps/generic version of
sys/ucontext.h keeps the inclusion by necessity, with a comment about
how this is not namespace-clean, but the only configuration that used
it, MicroBlaze, gets its own version of the header in this patch.
Where mcontext_t was typedefed to struct sigcontext, the contents of
struct sigcontext are inserted (with appropriate namespace handling to
prefix fields with __ when __USE_MISC is not defined); review should
check that this has been done correctly in each case, whether the
definition of struct sigcontext comes from glibc headers or from the
Linux kernel. This changes C++ name mangling on affected
architectures (which do not include x86_64/x86).
Tested for x86_64, and with build-many-glibcs.py.
2017-08-14 Joseph Myers <joseph@codesourcery.com>
[BZ #21457]
* sysdeps/arm/sys/ucontext.h: Do not include <bits/sigcontext.h>.
* sysdeps/generic/sys/ucontext.h: Add comment about use of struct
sigcontext and namespace requirements.
* sysdeps/i386/sys/ucontext.h: Do not include <bits/sigcontext.h>.
* sysdeps/m68k/sys/ucontext.h: Likewise.
* sysdeps/mips/sys/ucontext.h: Likewise. Include <bits/types.h>.
* sysdeps/unix/sysv/linux/aarch64/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
(__ctx): Define earlier.
(mcontext_t): Define structure contents rather than using struct
sigcontext.
* sysdeps/unix/sysv/linux/aarch64/ucontext_i.sym (oEXTENSION): Use
__glibc_reserved1 instead of __reserved.
* sysdeps/unix/sysv/linux/alpha/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
(__ctx): Define earlier.
(mcontext_t): Define structure contents rather than using struct
sigcontext.
* sysdeps/unix/sysv/linux/alpha/ucontext-offsets.sym: Use
mcontext_t instead of struct sigcontext.
* sysdeps/unix/sysv/linux/arm/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
(__ctx): Define earlier.
(mcontext_t): Define structure contents rather than using struct
sigcontext.
* sysdeps/unix/sysv/linux/hppa/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
(__ctx): Define earlier.
(mcontext_t): Define structure contents rather than using struct
sigcontext.
* sysdeps/unix/sysv/linux/ia64/makecontext.c (__makecontext): Use
mcontext_t instead of struct sigcontext.
* sysdeps/unix/sysv/linux/ia64/sigcontext-offsets.sym: Use
mcontext_t instead of struct sigcontext.
* sysdeps/unix/sysv/linux/ia64/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
(__ctx): New macro.
(struct __ia64_fpreg_mcontext): New type.
(mcontext_t): Define structure contents rather than using struct
sigcontext.
(_SC_GR0_OFFSET): Use mcontext_t instead of struct sigcontext.
(uc_sigmask): Define using __ctx.
(uc_stack): Likewise.
* sysdeps/unix/sysv/linux/ia64/sys/procfs.h: Include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/ia64/sys/ptrace.h: Likewise.
* sysdeps/unix/sysv/linux/m68k/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/microblaze/sys/ucontext.h: New file.
* sysdeps/unix/sysv/linux/mips/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/nios2/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/powerpc/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/s390/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/sh/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/sparc/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
* sysdeps/unix/sysv/linux/tile/sys/ucontext.h: Do not include
<bits/sigcontext.h>.
(__ctx): Define earlier.
(mcontext_t): Define structure contents rather than using struct
sigcontext.
* sysdeps/unix/sysv/linux/x86/sys/ucontext.h: Do not include
<bits/sigcontext.h>. Include <bits/types.h>.
* conform/Makefile (test-xfail-XPG42/signal.h/conform): Remove.
(test-xfail-XPG42/sys/wait.h/conform): Likewise.
(test-xfail-XPG42/ucontext.h/conform): Likewise.
(test-xfail-UNIX98/signal.h/conform): Likewise.
(test-xfail-UNIX98/sys/wait.h/conform): Likewise.
(test-xfail-UNIX98/ucontext.h/conform): Likewise.
(test-xfail-XOPEN2K/signal.h/conform): Likewise.
(test-xfail-XOPEN2K/sys/wait.h/conform): Likewise.
(test-xfail-XOPEN2K/ucontext.h/conform): Likewise.
(test-xfail-POSIX2008/signal.h/conform): Likewise.
(test-xfail-POSIX2008/sys/wait.h/conform): Likewise.
(test-xfail-XOPEN2K8/signal.h/conform): Likewise.
(test-xfail-XOPEN2K8/sys/wait.h/conform): Likewise.
bsd_signal was added in XPG4.2, so should be declared for
__USE_XOPEN_EXTENDED, instead of the present __USE_XOPEN condition.
This patch fixes the condition accordingly.
Tested for x86_64.
[BZ #21552]
* signal/signal.h (bsd_signal): Declare if [__USE_XOPEN_EXTENDED],
not [__USE_XOPEN].
* conform/Makefile (test-xfail-XPG4/signal.h/conform): Remove.
In sys/wait.h, waitid and associated constants and types are UX-shaded
in XPG4.2 (so not in XPG4), and XSI-shaded in POSIX before 2008, so
should be appropriately conditional in the headers. This patch fixes
the conditionals accordingly. (WCONTINUED is actually still
XSI-shaded in POSIX.1:2008, but W* is also reserved there without
XSI-shading, so nothing special needs to be done about the
conditionals on WCONTINUED to conform to POSIX.1:2008 namespace
rules.)
Tested for x86_64.
[BZ #21561]
* posix/sys/wait.h (idtype_t): Change [__USE_XOPEN] condition to
[__USE_XOPEN_EXTENDED].
(id_t): Likewise.
(include of <bits/types/siginfo_t.h): Likewise.
(waitid): Likewise.
* sysdeps/unix/sysv/linux/bits/waitflags.h (WSTOPPED): Condition
on [__USE_XOPEN_EXTENDED || __USE_XOPEN2K8].
(WEXITED): Likewise.
(WCONTINUED): Likewise.
(WNOWAIT): Likewise.
* conform/Makefile (test-xfail-XPG4/stdlib.h/conform): Remove.
(test-xfail-XPG4/sys/wait.h/conform): Likewise.
(test-xfail-POSIX/sys/wait.h/conform): Likewise.
This patch corrects the conformtest expectations for sys/wait.h for
XPG4. This does not change the test results for this header at the
makefile level since it fixes some spurious failures for this header
while introducing other failures previously hidden by incorrect
expectations. However, it shows up issues with stdlib.h for XPG4
defining W* names that are not permitted in this case (previously
wrongly allowed; the W* reservation is UX-shaded in XPG4.2 /
XSI-shaded in POSIX before 2008); that test is thus XFAILed.
Tested for x86_64.
* conform/data/sys/wait.h-data (WIFCONTINUED): Do not expect for
[XPG4].
(WEXITED): Likewise.
(WSTOPPED): Likewise.
(WCONTINUED): Likewise.
(WNOHANG): Likewise.
(WNOWAIT): Likewise.
(idtype_t): Likewise.
(P_ALL): Likewise.
(P_PID): Likewise.
(P_PGID): Likewise.
(id_t): Likewise.
(siginfo_t): Likewise.
(wait3): Likewise.
(waitid): Likewise.
(signal.h): Do not allow header for [XPG4].
(sys/resource.h): Likewise.
(si_*): Do not allow for [XPG4].
(W*): Likewise.
(P_*): Likewise.
(BUS_): Likewise.
(CLD_): Likewise.
(FPE_): Likewise.
(ILL_): Likewise.
(POLL_): Likewise.
(SEGV_): Likewise.
(SI_): Likewise.
(TRAP_): Likewise.
* conform/Makefile (test-xfail-XPG4/stdlib.h/conform): New
variable.
conform/ namespace tests of arpa/inet.h, netdb.h and netinet/in.h fail
for UNIX98 and XPG42 because of inclusion of stdint.h, which defines
macros not permitted in those headers for those standards. UNIX98
allows them to include inttypes.h, but (predating C99) has restricted
inttypes.h contents (not yet tested in the conform/ tests) not
including those macros; XPG4.2 has no such permission and no
inttypes.h / stdint.h at all.
This patch rearranges the headers to avoid this issue. intN_t
definitions move to bits/stdint-intn.h, and uintN_t definitions to
bits/stdint-uintn.h. (These are not bits/types/ headers because they
each define four types. They are separate rather than just a single
header because sys/types.h defines intN_t but u_intN_t rather than
uintN_t - and while sys/types.h could define uintN_t because of the
POSIX reservation of *_t, existing practice there is largely to
condition types on appropriate feature test macros, and indeed there
is at least one open bug report (14553) about a type that's not
so-conditioned, so maybe types there should actually have conditions
added where appropriate.) The affected network headers are then made
to include bits/stdint-uintn.h instead of stdint.h. This allows six
XFAILs to be removed.
This doesn't do anything about inttypes.h defining more than it should
for UNIX98, but we don't have conformtest expectations for that case
at present (and my inclination is that a fix for that should be as
local as possible - affecting only inttypes.h, not stdint.h, only for
the case of __USE_UNIX98 && !__USE_ISOC99).
Tested for x86_64.
[BZ #21455]
* bits/stdint-intn.h: New file.
* bits/stdint-uintn.h: Likewise.
* stdlib/Makefile (headers): Add bits/stdint-intn.h and
bits/stdint-uintn.h.
* inet/netinet/in.h: Include <bits/stdint-uintn.h> instead of
<stdint.h>.
* posix/sys/types.h: Include <bits/stdint-intn.h>.
(__int8_t_defined): Do not define here.
(int8_t): Likewise.
(int16_t): Likewise.
(int32_t): Likewise.
(int64_t): Likewise.
[__GNUC_PREREQ (2, 7)] (__intN_t): Likewise.
* resolv/netdb.h: Include <bits/stdint-uintn.h> instead of
<stdint.h>.
* include/netdb.h [_ISOMAC]: Do not include <stdint.h>.
* sysdeps/generic/stdint.h: Include <bits/stdint-intn.h> and
<bits/stdint-uintn.h>.
(int8_t): Do not define here.
(int16_t): Likewise.
(int32_t): Likewise.
(int64_t): Likewise.
(uint8_t): Likewise.
(uint16_t): Likewise.
(uint32_t): Likewise.
(uint64_t): Likewise.
* conform/Makefile (test-xfail-XPG42/arpa/inet.h/conform): Remove
variable.
(test-xfail-XPG42/netdb.h/conform): Likewise.
(test-xfail-XPG42/netinet/in.h/conform): Likewise.
(test-xfail-UNIX98/arpa/inet.h/conform): Likewise.
(test-xfail-UNIX98/netdb.h/conform): Likewise.
(test-xfail-UNIX98/netinet/in.h/conform): Likewise.
POSIX.1:2001 added sem_timedwait, but said nothing about defining
struct timespec in semaphore.h. The 2008 edition added a
corresponding permission to include <time.h> from <semaphore.h>.
Since this is the normal POSIX approach in such cases, it seems
appropriate to consider this a bug fix, and so this patch makes the
conform/ tests allow that inclusion for POSIX.1:2001 as well.
Tested for x86_64.
* conform/data/semaphore.h-data [XOPEN2K] (time.h): Allow header
inclusion.
* conform/Makefile (test-xfail-XOPEN2K/semaphore.h/conform):
Remove.
The conform/ tests test -D_XOPEN_SOURCE under the name "XPG3", and
-D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED under the name "XPG4".
That naming is misleading. _XOPEN_SOURCE_EXTENDED actually means
XPG4.2, including UX-shaded interfaces. _XOPEN_SOURCE actually means
XPG4, or XPG4.2 without UX-shaded interfaces. (Prior XPG versions
also used _XOPEN_SOURCE, but without any versioning of the values of
the macro, so XPG4.2 without UX-shaded interfaces is the only sensible
set of interfaces for glibc to expose given _XOPEN_SOURCE=1 without
_XOPEN_SOURCE_EXTENDED defined.)
This patch fixes the naming in the conform/ tests, so that what is now
called XPG4 is changed to XPG42, and what is now called XPG3 is
changed to XPG4.
Tested for x86_64 (and verified the complete set of expectations is
unchanged by the patch beyond the intended renaming).
* conform/GlibcConform.pm (XPG4): Rename standard to XPG42.
(XPG3): Rename standard to XPG4.
* conform/Makefile: Likewise.
* conform/list-header-symbols.pl: Likewise.
* conform/data/aio.h-data: Likewise.
* conform/data/arpa/inet.h-data: Likewise.
* conform/data/complex.h-data: Likewise.
* conform/data/ctype.h-data: Likewise.
* conform/data/dlfcn.h-data: Likewise.
* conform/data/fcntl.h-data: Likewise.
* conform/data/fenv.h-data: Likewise.
* conform/data/float.h-data: Likewise.
* conform/data/fmtmsg.h-data: Likewise.
* conform/data/ftw.h-data: Likewise.
* conform/data/grp.h-data: Likewise.
* conform/data/inttypes.h-data: Likewise.
* conform/data/iso646.h-data: Likewise.
* conform/data/langinfo.h-data: Likewise.
* conform/data/libgen.h-data: Likewise.
* conform/data/limits.h-data: Likewise.
* conform/data/locale.h-data: Likewise.
* conform/data/math.h-data: Likewise.
* conform/data/mqueue.h-data: Likewise.
* conform/data/ndbm.h-data: Likewise.
* conform/data/net/if.h-data: Likewise.
* conform/data/netdb.h-data: Likewise.
* conform/data/netinet/in.h-data: Likewise.
* conform/data/poll.h-data: Likewise.
* conform/data/pthread.h-data: Likewise.
* conform/data/pwd.h-data: Likewise.
* conform/data/sched.h-data: Likewise.
* conform/data/search.h-data: Likewise.
* conform/data/semaphore.h-data: Likewise.
* conform/data/signal.h-data: Likewise.
* conform/data/spawn.h-data: Likewise.
* conform/data/stdbool.h-data: Likewise.
* conform/data/stdint.h-data: Likewise.
* conform/data/stdio.h-data: Likewise.
* conform/data/stdlib.h-data: Likewise.
* conform/data/string.h-data: Likewise.
* conform/data/strings.h-data: Likewise.
* conform/data/stropts.h-data: Likewise.
* conform/data/sys/mman.h-data: Likewise.
* conform/data/sys/resource.h-data: Likewise.
* conform/data/sys/select.h-data: Likewise.
* conform/data/sys/socket.h-data: Likewise.
* conform/data/sys/stat.h-data: Likewise.
* conform/data/sys/statvfs.h-data: Likewise.
* conform/data/sys/time.h-data: Likewise.
* conform/data/sys/timeb.h-data: Likewise.
* conform/data/sys/types.h-data: Likewise.
* conform/data/sys/uio.h-data: Likewise.
* conform/data/sys/un.h-data: Likewise.
* conform/data/sys/wait.h-data: Likewise.
* conform/data/syslog.h-data: Likewise.
* conform/data/termios.h-data: Likewise.
* conform/data/tgmath.h-data: Likewise.
* conform/data/time.h-data: Likewise.
* conform/data/ucontext.h-data: Likewise.
* conform/data/unistd.h-data: Likewise.
* conform/data/utmpx.h-data: Likewise.
* conform/data/varargs.h-data: Likewise.
* conform/data/wchar.h-data: Likewise.
* conform/data/wctype.h-data: Likewise.
sys/socket.h includes sys/uio.h to get the definition of the iovec
structure.
POSIX allows sys/socket.h to make all sys/uio.h symbols visible.
However, all of sys/uio.h is XSI-shaded, so for non-XSI POSIX this
results in conformtest failures (for sys/socket.h and other headers
that include it):
Namespace violation: "UIO_MAXIOV"
Namespace violation: "readv"
Namespace violation: "writev"
Now, there is some ambiguity in POSIX about what namespace
reservations apply in this case - see
http://austingroupbugs.net/view.php?id=1127 - but glibc convention
would still avoid declaring readv and writev, for example, for feature
test macros that don't include them (if only headers from the relevant
standard are included), even if such declarations are permitted, so
there is a bug here according to glibc conventions.
This patch moves the struct iovec definition to a new
bits/types/struct_iovec.h header and includes that from sys/socket.h
instead of including the whole of sys/uio.h. This fixes the namespace
issue; however, three files in glibc that were relying on the implicit
inclusion needed to be updated to include sys/uio.h explicitly. So
there is a question of whether sys/socket.h should continue to include
sys/uio.h under some conditions, such as __USE_XOPEN or __USE_MISC or
__USE_XOPEN || __USE_MISC, for greater compatibility with code that
(wrongly) expects this optional inclusion to be present there. (I
think the three affected files in glibc should still have explicit
sys/uio.h inclusions added in any case, however.)
Tested for x86_64.
[BZ #21426]
* misc/bits/types/struct_iovec.h: New file.
* misc/Makefile (headers): Add bits/types/struct_iovec.h.
* include/bits/types/struct_iovec.h: New file.
* bits/uio.h (struct iovec): Replace by inclusion of
<bits/types/struct_iovec.h>.
* sysdeps/unix/sysv/linux/bits/uio.h (struct iovec): Likewise.
* socket/sys/socket.h: Include <bits/types/struct_iovec.h> instead
of <sys/uio.h>.
* nptl/tst-cancel4.c: Include <sys/uio.h>
* posix/test-errno.c: Likewise.
* support/resolv_test.c: Likewise.
* conform/Makefile (test-xfail-POSIX2008/arpa/inet.h/conform):
Remove.
(test-xfail-POSIX2008/netdb.h/conform): Likewise.
(test-xfail-POSIX2008/netinet/in.h/conform): Likewise.
(test-xfail-POSIX2008/sys/socket.h/conform): Likewise.
This patch enables the compilation part of the conformtest tests (the
vast bulk of them) when cross compiling, so making it easy to run them
across many configurations with build-many-glibcs.py.
Tested with build-many-glibcs.py.
* conform/Makefile (tests-special): Do not make addition of
$(conformtest-header-tests) conditional on [$(cross-compiling) = no].
(generated): Do not make addition of $(conformtest-header-base)
conditional on [$(cross-compiling) = no].
This patch makes conformtest skip execution tests when
cross-compiling, as an interim step towards running most of these
tests (presently disabled) in that case. It omits the (obvious)
Makefile change to actually enable the conformtest tests when
cross-compiling, as there are still enough failures seen with
build-many-glibcs.py that I'd like to get the results cleaner before
enabling these tests.
Tested for x86_64, and with the tests actually enabled for
cross-compilation with build-many-glibcs.py.
* conform/conformtest.pl ($cross): New variable.
(--cross): New command-line option.
(runtest): Skip test execution when cross-compiling.
* conform/Makefile (conformtest-cross): New variable.
($(conformtest-header-tests)): Pass $(conformtest-cross) to
conformtest.pl.
conformtest has an internal XFAIL mechanism to allow failures of
individual expectations to be ignored, so that known hard-to-fix
failures (e.g. those affecting ABIs or requiring kernel changes) do
not cause the overall tests to FAIL and so hide other failures from
the same (header, standard) pair.
Various such bugs are system-specific, so this patch adds a mechanism
to allow system-specific XFAILs. A system-independent XFAIL is
achieved by putting "xfail-" at the start of the relevant expectation
in the *-data files. A system-specific XFAIL instead uses
"xfail[cond]-", where "cond" is a condition listed in
conformtest-xfail-conds in a sysdeps makefile (so one for x32 might
set conformtest-xfail-conds = x86_64-x32-linux, for example, and then
an expectation for tv_nsec's type could use
xfail[x86_64-x32-linux]-). The actual names are arbitrary, just
needing to match between the makefiles and the expectations, and if
necessary you can use "xfail[cond1|cond2]-" for a test that is
expected to fail under multiple conditions. As with
system-independent XFAILs, I think system-specific ones should have a
bug filed in Bugzilla and a comment referencing that bug.
Tested for x86_64, including with test expectations and makefiles
changed to use the new facility.
* conform/conformtest.pl ($xfail_str): New variable.
(--xfail=): New command-line option.
(top level): Handle expectations starting xfail[cond]-.
* conform/Makefile (conformtest-xfail): New variable.
($(conformtest-header-tests)): Pass $(conformtest-xfail) to
conformtest.pl.
When I set up linknamespace testing, the lists of libraries that might
contain functions from various standards were based on the -l options
POSIX says may be required to find certain functions with the c99
utility.
glibc has some POSIX functions in the libdl and libcrypt libraries,
not mentioned in the definition of the c99 utility (so an
implementation of that utility using glibc would need to use -ldl
-lcrypt automatically). This patch adds those libraries to the ones
considered in linknamespace testing for relevant standards. (The
crypt functions are XSI only, present in XPG3 and above; the libdl
ones were added in UNIX98, then moved from XSI to BASE in the 2008
edition of POSIX.)
* conform/Makefile (linknamespace-libs): Rename to
linknamespace-libs-thr.
(linknamespace-libs-posix): New variable.
(linknamespace-libs-xsi): Likewise.
(linknamespace-libs-XPG3): Include libcrypt.a.
(linknamespace-libs-XPG4): Use $(linknamespace-libs-XPG3).
(linknamespace-libs-POSIX): Use $(linknamespace-libs-thr).
(linknamespace-libs-UNIX98): Use $(linknamespace-libs-xsi).
(linknamespace-libs-XOPEN2K): Likewise.
(linknamespace-libs-XOPEN2K8): Likewise.
(linknamespace-libs-POSIX2008): Use $(linknamespace-libs-posix).
Having found that with my script to build many glibc variants I could
reproduce the linknamespace test failures in parallel builds (that
various people had previously reported but I hadn't seen myself), I
investigated those failures further. This patch adds a missing
dependency to those tests.
Tested for x86_64, including the configuration where I saw those
failures and where I don't see them with this patch.
* conform/Makefile ($(linknamespace-header-tests)): Also depend on
$(linknamespace-symlists-tests).
Historically perl includes the current directory in the module search
path. Over the time this has been considered as a security issue and
the recent vulnerabilities [1] made people to reconsider this behaviour.
It is almost sure that this will be removed in the future [2], possibly
for the 5.26 release, although this is not yet firmly decided.
Debian has decided to backport the patches [3], so the perl binary in
unstable do not have '.' in @INC anymore.
This behaviour is used in the conform perl scripts to include the
GlibcConform module. This patch fixes that by calling perl with '-I.'.
This is not a security issue in this case as make ensures that the
current directory is $(srcdir)/conform/ when the scripts are called.
Passing the full path would do exactly the same.
[1] CVE-2016-1238 CVE-2016-6185
[2] https://rt.perl.org/Public/Bug/Display.html?id=127810
[3] https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html
Changelog:
* conform/Makefile (conformtest-header-tests): Pass -I. to $(PERL).
(linknamespace-symlists-tests): Likewise.
(linknamespace-header-tests): Likewise.
The conform/ test of limits.h namespace for XPG3 was failing because
of NL_* and NZERO defines. Those symbols are EX-shaded, not
UX-shaded, so it's correct for them to be defined for XPG3; this patch
corrects the expectations accordingly. (Actually it looks like they
should not be listed as optional for these standards, but that's
another matter.)
Tested for x86_64 and x86.
* conform/data/limits.h-data (CHARCLASS_NAME_MAX): Also expect for
[XPG3 || XPG4].
(NL_ARGMAX): Also expect for [XPG3].
(NL_LANGMAX): Likewise.
(NL_MSGMAX): Likewise.
(NL_NMAX): Likewise.
(NL_SETMAX): Likewise.
(NL_TEXTMAX): Likewise.
(NZERO): Likewise.
(TMP_MAX): Likewise.
* conform/Makefile (test-xfail-XPG3/limits.h/conform): Remove
variable.
This patch corrects various conformtest expectations in ftw.h for XPG3
and XPG4.
Tested for x86_64 and x86.
* conform/data/ftw.h-data (struct FTW): Do not expect for [XPG3].
(FTW_DP): Do not expect for [XPG3 || XPG4].
(FTW_SL): Do not expect for [XPG3].
(FTW_SLN): Likewise.
(FTW_PHYS): Likewise.
(FTW_MOUNT): Likewise.
(FTW_DEPTH): Likewise.
(FTW_CHDIR): Likewise.
(nftw): Likewise.
* conform/Makefile (test-xfail-XPG3/ftw.h/conform): Remove
variable.
This patch corrects some conformtest expectations for pwd.h for XPG4.
Tested for x86_64 and x86.
* conform/data/pwd.h-data (endpwent): Do not expect for [XPG3].
(getpwent): Likewise.
(setpwent): Likewise.
* conform/Makefile (test-xfail-XPG3/pwd.h/conform): Remove
variable.
This patch corrects some conformtest expectations for search.h for
XPG3.
Tested for x86_64 and x86.
* conform/data/search.h-data (insque): Do not expect for [XPG3].
(remque): Likewise.
* conform/Makefile (test-xfail-XPG3/search.h/conform): Remove
variable.
stdlib.h declares grantpt, ptsname, unlockpt for __USE_XOPEN. This
patch corrects the condition to __USE_XOPEN_EXTENDED (these functions
are new in XPG4).
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
[BZ #20094]
* stdlib/stdlib.h (grantpt): Declare if [__USE_XOPEN_EXTENDED],
not [__USE_XOPEN].
(unlockpt): Likewise.
(ptsname): Likewise.
* conform/Makefile (test-xfail-XPG3/stdlib.h/conform): Remove
variable.
sys/stat.h should define S_IFSOCK and S_ISSOCK for XPG4 (XNS), but
does not. This patch corrects the relevant header conditionals.
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
[BZ #20076]
* io/sys/stat.h (S_IFSOCK): Define for [__USE_XOPEN_EXTENDED]
instead of [__USE_UNIX98].
(S_ISSOCK): Likewise.
* conform/Makefile (test-xfail-XPG4/sys/stat.h/conform): Remove
variable.
stdlib.h declares rand_r if __USE_POSIX; i.e., POSIX.1:1990. But
rand_r was added along with threads, so the condition should be
__USE_POSIX199506. This patch corrects the condition.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20074]
* stdlib/stdlib.h (rand_r): Declare if [__USE_POSIX199506], not if
[__USE_POSIX].
* conform/Makefile (test-xfail-XPG4/stdlib.h/conform): Remove
variable.
UNIX98 and XPG4 have ttyslot in <stdlib.h>. glibc, however, has it in
<unistd.h>, for __USE_MISC || (__USE_XOPEN_EXTENDED && !__USE_UNIX98),
but no supported standard has it in <unistd.h>.
This patch adds a properly conditioned declaration to <stdlib.h> (only
enabled for the relevant standards, not for __USE_MISC or __USE_GNU).
The <unistd.h> declaration is restricted to __USE_MISC. Some relevant
XFAILs are removed.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20051]
* posix/unistd.h [__USE_XOPEN_EXTENDED && !__USE_UNIX98]
(ttyslot): Do not declare.
* stdlib/stdlib.h [__USE_XOPEN_EXTENDED && !__USE_XOPEN2K]
(ttyslot): New prototype.
* conform/Makefile (test-xfail-XPG4/unistd.h/conform): Remove
variable.
(test-xfail-UNIX98/stdlib.h/conform): Likewise.
This patch corrects a spurious conformtest expectation of strdup in
string.h for XPG3.
Tested for x86_64 and x86.
* conform/data/string.h-data (strdup): Do not expect for [XPG3].
* conform/Makefile (test-xfail-XPG3/string.h/conform): Remove
variable.
This patch corrects spurious conformtest expectations of getdate and
getdate_err for XPG3. (This is not based on a full review of the
expectations, so there may be other issues where the header and tests
agree but are both incorrect.)
Tested for x86_64 and x86.
* conform/data/time.h-data (getdate_err): Do not expect for
[XPG3].
(getdate): Likewise.
* conform/Makefile (test-xfail-XPG3/time.h/conform): Remove
variable.
The conformtest tests of unistd.h fail for XPG3 because of various
expectations that are incorrect for XPG3. This patch corrects those
bogus expectations, and one bogus expectation for XPG4. (This is not
based on a full review of the standards so there may well still be
other bugs in the expectations for this header.)
Tested for x86_64 and x86.
* conform/data/unistd.h-data (F_LOCK): Do not expect for [XPG3].
(F_ULOCK): Likewise.
(F_TEST): Likewise.
(F_TLOCK): Likewise.
(useconds_t): Likewise.
(intptr_t): Do not expect for [XPG3] or [XPG4].
(brk): Do not expect for [XPG3]
(fchown): Likewise.
(fchdir): Likewise.
(ftruncate): Likewise.
(getdtablesize): Likewise.
(gethostid): Likewise.
(getpagesize): Likewise.
(getpgid): Likewise.
(getsid): Likewise.
(getwd): Likewise.
(lchown): Likewise.
(lockf): Likewise.
(readlink): Likewise.
(sbrk): Likewise.
(setpgrp): Likewise.
(setregid): Likewise.
(setreuid): Likewise.
(symlink): Likewise.
(sync): Likewise.
(truncate): Likewise.
(ualarm): Likewise.
(usleep): Likewise.
(vfork): Likewise.
* conform/Makefile (test-xfail-XPG3/unistd.h/conform): Remove
variable.
The conform/ tests for fcntl.h are failing for XPG3 and XPG4 because
of missing S_IFSOCK.
This is a case of a bogus test. The relevant wording requiring such
constants is, in current POSIX (and this requirement dates back as far
as XPG4), "The <fcntl.h> header shall define the symbolic constants
for file modes for use as values of mode_t as described in
<sys/stat.h>.". Note that this is *file modes* not *file types*.
That makes sense, since the point is presumably for use with functions
such as open that are declared in fcntl.h, where file modes are
relevant but file types aren't. So this patch removes all those
spurious S_IF* expectations for fcntl.h (the macros are generally
still *allowed* through the permission to make everything from
sys/stat.h visible).
Tested for x86_64 and x86.
* conform/data/fcntl.h-data [!POSIX] (S_IFMT): Do not expect.
[!POSIX] (S_IFBLK): Likewise.
[!POSIX] (S_IFCHR): Likewise.
[!POSIX] (S_IFIFO): Likewise.
[!POSIX] (S_IFREG): Likewise.
[!POSIX] (S_IFDIR): Likewise.
[!POSIX] (S_IFLNK): Likewise.
[!POSIX] (S_IFSOCK): Likewise.
* conform/Makefile (test-xfail-XPG3/fcntl.h/conform): Remove
variable.
(test-xfail-XPG4/fcntl.h/conform): Likewise.
termios.h should declare tcgetsid for XPG4, but only does so for
__USE_UNIX98 || __USE_XOPEN2K8 at present. This patch fixes the
declaration conditions. A spurious conformtest expectation of this
declaration for XPG3 is removed, and two XFAILs that are fixed by
these changes are also removed.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20055]
* termios/termios.h (pid_t): Define for [__USE_XOPEN_EXTENDED]
instead of [__USE_UNIX98].
(tcgetsid): Declare for [__USE_XOPEN_EXTENDED] instead of
[__USE_UNIX98].
* conform/data/termios.h-data (tcgetsid): Do not expect for
[XPG3].
* conform/Makefile (test-xfail-XPG3/termios.h/conform): Remove
variable.
(test-xfail-XPG4/termios.h/conform): Likewise.
sys/stat.h declares fchmod if __USE_POSIX (i.e. POSIX.1:1990). But it
was actually added in 1993 and also in XPG4. This patch changes the
conditions to the correct __USE_POSIX199309 || __USE_XOPEN_EXTENDED.
Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).
[BZ #20073]
* io/sys/stat.h (fchmod): Declare for
[__USE_POSIX199309 || __USE_XOPEN_EXTENDED], not [__USE_POSIX].
* conform/Makefile (test-xfail-XPG3/sys/stat.h/conform): Remove
variable.
For UNIX98 (only), unistd.h should declare pthread_atfork, but that
declaration is missing. This patch adds it.
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20044]
* posix/unistd.h [__USE_UNIX98 && !__USE_XOPEN2K]
(pthread_atfork): New prototype.
* conform/Makefile (test-xfail-UNIX98/unistd.h/conform): Remove
variable.
The conformtest tests for sys/types.h for XPG3 and XPG4 fail because
of missing blksize_t. This is a bug in the expectations; that type is
not part of those standards. This patch stops the tests from
expecting it, and some other types that also are not part of XPG3 and
XPG4.
Tested for x86_64 and x86.
* conform/data/sys/types.h-data (blkcnt_t): Do not expect for
[XPG3 || XPG4].
(blksize_t): Likewise.
(clockid_t): Likewise.
* conform/Makefile (test-xfail-XPG3/sys/types.h/conform): Remove
variable.
(test-xfail-XPG4/sys/types.h/conform): Likewise.
For UNIX98 and older standards, sys/time.h should not define struct
timespec, but does so via the inclusion of sys/select.h (which is a
new header in the 2001 edition of POSIX, and defines struct timespec
because of the declaration of pselect, a new function in the 2001
edition of POSIX). In turn, this affects some other headers that
themselves include sys/time.h.
This patch fixes this by conditioning the __need_timespec definition
in sys/select.h on __USE_XOPEN2K, the same condition used there for
the declaration of pselect (this has no effect on direct uses of
sys/select.h with feature test macros for any standard that includes
that header, since such standards result in __USE_XOPEN2K being
defined).
Tested for x86_64 and x86 (testsuite, and that installed stripped
shared libraries are unchanged by the patch).
[BZ #20041]
* misc/sys/select.h (__need_timespec): Only define if
[__USE_XOPEN2K].
* conform/Makefile (test-xfail-XPG4/sys/time.h/conform): Remove
variable.
(test-xfail-XPG4/utmpx.h/conform): Likewise.
(test-xfail-UNIX98/sys/time.h/conform): Likewise.
(test-xfail-UNIX98/utmpx.h/conform): Likewise.
