This patch is a revised and updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html>.
In order to generate overall summaries of the results of all tests in
the glibc testsuite, we need to identify and concatenate the files
with the results of individual tests.
Tomas Dohnalek's patch used $(common-objpfx)*/*.test-result for this.
However, the normal glibc approach is explicit enumeration of the
expected set of files with a given property, rather than all files
matching some pattern like that. Furthermore, we would like to be
able to mark tests as UNRESOLVED if the file with their results is for
some reason missing, and in future we would like to be able to mark
tests as UNSUPPORTED if they are disabled for a particular
configuration (rather than simply having them missing from the list of
tests as at present). Such handling of tests that were not run or did
not record results requires an explicit enumeration of tests.
For the tests following the default makefile rules, $(tests) (and
$(xtests)) provides such an enumeration. Others, however, are added
directly as dependencies of the "tests" and "xtests" makefile
targets. This patch changes the makefiles to put them in variables
tests-special and xtests-special, with appropriate dependencies on the
tests listed there then being added centrally.
Those variables are used in Rules and so need to be set before Rules
is included in a subdirectory makefile, which is often earlier in the
makefile than the dependencies were present before. We previously
discussed the question of where to include Rules; see the question at
<https://sourceware.org/ml/libc-alpha/2012-11/msg00798.html>, and a
discussion in
<https://sourceware.org/ml/libc-alpha/2013-01/msg00337.html> of why
Rules is included early rather than late in subdirectory makefiles.
It was necessary to avoid an indirection through the check-abi target
and get the check-abi-* targets for individual libraries into the
tests-special variable. The intl/ test $(objpfx)tst-gettext.out,
previously built only because of dependencies from other tests, was
also added to tests-special for the same reason.
The entries in tests-special are the full makefile targets, complete
with $(objpfx) and .out. If a future change causes tests to be named
consistently with a .out suffix, this can be changed to include just
the path relative to $(objpfx), without .out.
Tested x86_64, including that the same set of files is generated in
the build directory by a build and testsuite run both before and after
the patch (except for changes to the
elf/tst-null-argv.debug.out.<number> file name), and a build with
run-built-tests=no to verify there aren't any more obvious instances
of the issue Marcus Shawcroft reported with a previous version in
<https://sourceware.org/ml/libc-alpha/2014-01/msg00462.html>.
* Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(tests): Depend on $(tests-special).
* Makerules (check-abi-list): New variable.
(check-abi): Depend on $(check-abi-list).
[$(subdir) = elf] (tests-special): Add
$(objpfx)check-abi-libc.out.
[$(build-shared) = yes && subdir] (tests-special): Add
$(check-abi-list).
[$(build-shared) = yes && subdir] (tests): Do not depend on
check-abi.
* Rules (tests): Depend on $(tests-special).
(xtests): Depend on $(xtests-special).
* catgets/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* conform/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* elf/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* grp/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* iconv/Makefile (xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* iconvdata/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* intl/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable. Also add
$(objpfx)tst-gettext.out.
* io/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* libio/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* malloc/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* misc/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* nptl/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* nptl_db/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* posix/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* resolv/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* stdio-common/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(do-tst-unbputc): Remove target.
(do-tst-printf): Likewise.
* stdlib/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* string/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* sysdeps/x86/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
localedata:
* Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00193.html>, starts
the process of generating explicit PASS or FAIL status for individual
glibc tests. It's based on Tomas Dohnalek's patch
<https://sourceware.org/ml/libc-alpha/2012-10/msg00278.html>, but is
deliberately more minimal: it doesn't try to cover any tests outside
of $(tests) / $(xtests) (that's for a later patch), nor does it put
the result together in an overall summary file (again, a later patch):
it just generates the .test-result files.
Thus, this patch keeps the overall logic for when a testsuite run
finishes completely unchanged: a test failing will terminate the run.
I think we *should* move to a more conventional approach where plain
"make check" does not terminate for an individual test failure, unless
e.g. you say "make stop-on-test-failure=y check", but that sort of
policy change is best done as a separate patch once the infrastructure
is in place to generate summary files for completed test runs (which
will entirely consist of PASS and XFAIL lines if the testsuite run
reaches the point of generating them, until such a policy change is
made).
Tested x86_64.
2014-02-14 Tomas Dohnalek <tdohnale@redhat.com>
Joseph Myers <joseph@codesourcery.com>
* Makeconfig (test-name): New variable.
(evaluate-test): Likewise.
* Makerules (do-test-clean): Remove .test-result files.
(common-mostlyclean): Likewise.
* Rules ($(objpfx)%.out): Use $(evaluate-test) in both rules.
* scripts/evaluate-test.sh: New file.
If you rerun "make check" in a tree where some tests have already been
run, it will rerun ABI tests because those do not create an output
file.
This patch changes those tests to create .out files so they only get
rerun if the dependencies (on the ABI baselines and the generated
.symlist files) indicate they should be rerun.
Tested x86_64.
* Makerules (check-abi-%): Change target to
$(objpfx)check-abi-%.out.
(check-abi target): Update dependencies.
(check-abi-pattern variable): Redirect output of diff to $@.
(check-abi variable): Likewise.
* elf/Makefile (check-abi): Update dependencies.
This patch creates implicit rules to match the abifiles if
abilist-pattern is defined in the architecture Makefile. This allows
machine specific Makefiles to define different abifiles names
(for instance *-le.abilist for powerpc64le).
[BZ #15915] As described in the bug, the pattern rule for lib%.so files
in Makerules includes linkobj/libc.so as a dependency. However, the
explicit rule for linkobj/libc.so is in the top-level Makefile.
Thus, the subdirectory makefiles that include Makerules end up with an
erroneous makefile pattern rule for linkobj/libc.so that includes
itself as a dependency. The result is make warnings whenever rules
for other .so files are resolved -- and, on occasion, actual makefile
failures when a race condition causes the implicit rule to actually be
used.
This patch moves the explicit rules for linkobj/libc.so into Makerules
to clear up this problem. It also elaborates a couple of comments
that I'd initially found confusing.
abilist.awk relies on English-language output to find the "file
format" header line in "objdump --dynamic-syms" output, so force the
locale.
Without this change, "make check" fails in non-English locales:
| diff -p -U 0 ../sysdeps/unix/sysv/linux/x86_64/64/nptl/libBrokenLocale.abilist [...]/locale/libBrokenLocale.symlist
| --- ../sysdeps/unix/sysv/linux/x86_64/64/nptl/libBrokenLocale.abilist 2012-10-02 18:33:50.761863147 -0700
| +++ [...]/locale/libBrokenLocale.symlist 2012-10-03 00:11:22.877234252 -0700
| @@ -0,0 +1 @@
| +Don't grok this line: [...]/locale/libBrokenLocale.so: format de fichier elf64-x86-64
| make[2]: *** [check-abi-libBrokenLocale] Erreur 1
Due to the rise of kbuild type build systems (as the Linux kernel is a
popularly emulated environment), the V variable has become common as a
knob for controlling verbosity. Unfortunately, if you run `make V=1`
with glibc during install, it fails with weird errors due to the glibc
build already using this variable for versioning information.
Granted, overriding this variable in the glibc context makes no sense
so people shouldn't be doing it, but when paired with build frameworks
that like to use one set of options for all packages, glibc starts to
stick out as an oddball (in that it fails). Considering it's easy
enough to rename (it's used in just one place), let's do so.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
* Makerules (%.abilist): Add vpath on sysdep_dirs.
(check-abi-%): Remove AWK script prerequisite and explicit
abilist directory.
(check-abi): Rewrite to just diff the symlist with the abilist.
(config-tls, config-abi-config): Delete, no longer used.
(update-abi-%): Remove AWK script and explicit abilist directory.
(update-abi): Rewrite to simply compare and conditionally copy the
symlist and the sysdep abilist file. Remove update-abi-config
checks.
* abilist/ld.abilist: Remove.
* abilist/libBrokenLocale.abilist: Remove.
* abilist/libanl.abilist: Remove.
* abilist/libcrypt.abilist: Remove.
* abilist/libdl.abilist: Remove.
* abilist/librt.abilist: Remove.
* abilist/libthread_db.abilist: Remove.
* abilist/libutil.abilist: Remove.
* scripts/extract-abilist.awk: Remove.
* scripts/merge-abilist.awk: Remove.
* sysdeps/generic/libcidn.abilist: New file.
* sysdeps/generic/libnss_compat.abilist: New file.
* sysdeps/generic/libnss_db.abilist: New file.
* sysdeps/generic/libnss_dns.abilist: New file.
* sysdeps/generic/libnss_files.abilist: New file.
* sysdeps/generic/libnss_hesiod.abilist: New file.
* sysdeps/generic/libnss_nis.abilist: New file.
* sysdeps/generic/libnss_nisplus.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libanl.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libBrokenLocale.abilist: New
file.
* sysdeps/unix/sysv/linux/i386/nptl/libc.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libcrypt.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libdl.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libm.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libnsl.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libpthread.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libresolv.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/librt.abilist: New file.
* sysdeps/unix/sysv/linux/i386/nptl/libthread_db.abilist: New
file.
* sysdeps/unix/sysv/linux/i386/nptl/libutil.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libanl.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libBrokenLocale.abilist: New
file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libc.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libcrypt.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libdl.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libm.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libnsl.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libpthread.abilist: New
file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libresolv.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/librt.abilist: New file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libthread_db.abilist: New
file.
* sysdeps/unix/sysv/linux/x86_64/nptl/libutil.abilist: New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/ld.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libanl.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libBrokenLocale.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libc.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libcrypt.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libdl.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libm.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libnsl.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libpthread.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libresolv.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/librt.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libthread_db.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powercp32/fpu/nptl/libutil.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/ld.abilist: New
file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libanl.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libBrokenLocale.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libc.abilist: New
file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libcrypt.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libdl.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libm.abilist: New
file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libnsl.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libpthread.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libresolv.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/librt.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libthread_db.abilist:
New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/nptl/libutil.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libanl.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libBrokenLocale.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libc.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libcrypt.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libdl.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libm.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libnsl.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libpthread.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libresolv.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/librt.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libthread_db.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-32/nptl/libutil.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libanl.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libBrokenLocale.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libc.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libcrypt.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libdl.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libm.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libnsl.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libpthread.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libresolv.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/librt.abilist: New
file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libthread_db.abilist:
New file.
* sysdeps/unix/sysv/linux/s390/s390-64/nptl/libutil.abilist: New
file.
* sysdeps/unix/sysv/linux/sh/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libanl.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libBrokenLocale.abilist: New
file.
* sysdeps/unix/sysv/linux/sh/nptl/libc.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libcrypt.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libdl.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libm.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libnsl.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libpthread.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libresolv.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/librt.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libthread_db.abilist: New file.
* sysdeps/unix/sysv/linux/sh/nptl/libutil.abilist: New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libanl.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libBrokenLocale.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libc.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libcrypt.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libdl.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libm.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libnsl.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libpthread.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libresolv.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/librt.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libthread_db.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/nptl/libutil.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/ld.abilist: New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libanl.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libBrokenLocale.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libc.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libcrypt.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libdl.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libm.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libnsl.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libpthread.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libresolv.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/librt.abilist: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libthread_db.abilist:
New file.
* sysdeps/unix/sysv/linux/sparc/sparc64/nptl/libutil.abilist: New
file.
