Commit Graph

153 Commits

Author SHA1 Message Date
Joseph Myers
81700b5363 conformtest: correct set of standards for which more headers are tested.
Continuing the corrections to which headers conformtest tests for
which standards, this patch corrects conformtest mistakes regarding
which headers to test for the remaining standards (XOPEN2K POSIX2008
XOPEN2K8).  Tested x86_64.

2014-03-05  Joseph Myers  <joseph@codesourcery.com>

	* conform/data/netinet/tcp.h-data [POSIX2008 || XOPEN2K8]: Enable
	whole file.
	* conform/data/sys/timeb.h-data [POSIX2008 || XOPEN2K8]: Disable
	whole file.
	* conform/data/sys/uio.h-data [POSIX2008]: Likewise.
	* conform/data/ucontext.h-data [POSIX2008 || XOPEN2K8]: Likewise.
2014-03-06 17:43:46 +00:00
Joseph Myers
2ea18461d3 conformtest: correct set of standards for which some headers are tested.
Reviewing the sets of headers for which conformtest has expectations
for various standards showed up cases where headers had expectations
for standards not including those headers, or were missing
expectations for a standard that did include the header.  This patch
fixes easy cases of this (largely disabling tests for standards for
which they are inapplicable).  Tested x86_64.

	* conform/data/aio.h-data [XPG3 || XPG4]: Disable whole file.
	* conform/data/arpa/inet.h-data [XPG3]: Likewise.
	* conform/data/dlfcn.h-data [XPG3 || XPG4]: Likewise.
	* conform/data/fmtmsg.h-data [XPG3]: Likewise.
	* conform/data/libgen.h-data [XPG3]: Likewise.
	* conform/data/mqueue.h-data [XPG3 || XPG4]: Likewise.
	* conform/data/ndbm.h-data [XPG3]: Likewise.
	* conform/data/net/if.h-data [XPG3 || XPG4 || UNIX98]: Likewise.
	* conform/data/netdb.h-data [XPG3]: Likewise.
	* conform/data/netinet/in.h-data [XPG3]: Likewise.
	* conform/data/poll.h-data [XPG3]: Likewise.
	* conform/data/spawn.h-data [XPG3 || XPG4 || UNIX98]: Likewise.
	* conform/data/strings.h-data [XPG3]: Likewise.
	* conform/data/stropts.h-data [XPG3]: Likewise.
	* conform/data/sys/mman.h-data [XPG3]: Likewise.
	* conform/data/sys/resource.h-data [XPG3]: Likewise.
	* conform/data/sys/select.h-data [XPG3 || XPG4 || UNIX98]:
	Likewise.
	* conform/data/sys/statvfs.h-data [XPG3]: Likewise.
	* conform/data/sys/time.h-data [XPG3]: Likewise.
	* conform/data/sys/timeb.h-data [XPG3]: Likewise.
	* conform/data/sys/uio.h-data [XPG3]: Likewise.
	* conform/data/sys/un.h-data [XPG3]: Likewise.
	* conform/data/syslog.h-data [XPG3]: Likewise.
	* conform/data/ucontext.h-data [XPG3]: Likewise.
	* conform/data/utmpx.h-data [XPG3]: Likewise.
	* conform/data/varargs.h-data [UNIX98]: Enable file.
2014-03-06 17:42:03 +00:00
Joseph Myers
f8c17e79fa Support expected failures in .test-result files.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00195.html>, makes it
possible for .test-result files for individual tests to contain XPASS
and XFAIL rather than PASS and FAIL in cases where failure is
expected.  This replaces the marking of two individual tests with "-"
to cause them to be expected at makefile level to fail;
evaluate-test.sh will ensure it exits with status 0 for an expected
failure.

Tested x86_64.

	* scripts/evaluate-test.sh: Take new argument indicating whether
	failure is expected.
	* Makeconfig (evaluate-test): Pass argument to evaluate-test.sh
	indicating whether failure is expected.
	* conform/Makefile (test-xfail-run-conformtest): New variable.
	($(objpfx)run-conformtest.out): Don't expect to fail at makefile
	level.
	* posix/Makefile (test-xfail-annexc): New variable.
	($(objpfx)annexc.out): Don't expect to fail at makefile level.
2014-02-27 03:25:27 +00:00
Joseph Myers
a5f891ac8d Consistently include Makeconfig after defining subdir.
In <https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html> I
noted it was necessary to add includes of Makeconfig early in various
subdirectory makefiles for the tests-special variable settings added
by that patch to be conditional on configuration information.  No-one
commented on the general question there of whether Makeconfig should
always be included immediately after the definition of subdir.

This patch implements that early inclusion of Makeconfig in each
directory (which is a lot easier than consistent placement of includes
of Rules).  Includes are added if needed, or moved up if already
present.  Subdirectory "all:" targets are removed, since Makeconfig
provides one.

There is potential for further cleanups I haven't done.  Rules and
Makerules have code such as

ifneq   "$(findstring env,$(origin headers))" ""
headers :=
endif

to override to empty any value of various variables that came from the
environment.  I think there is a case for Makeconfig setting all the
subdirectory variables (other than subdir) to empty to ensure no
outside value is going to take effect if a subdirectory fails to
define a variable.  (A list of such variables, possibly out of date
and incomplete, is in manual/maint.texi.)  Rules and Makerules would
give errors if Makeconfig hadn't already been included, instead of
including it themselves.  The special code to override values coming
from the environment would then be obsolete and could be removed.

Tested x86_64, including that installed binaries are identical before
and after the patch.

	* argp/Makefile: Include Makeconfig immediately after defining
	subdir.
	* assert/Makefile: Likewise.
	* benchtests/Makefile: Likewise.
	* catgets/Makefile: Likewise.
	* conform/Makefile: Likewise.
	* crypt/Makefile: Likewise.
	* csu/Makefile: Likewise.
	(all): Remove target.
	* ctype/Makefile: Include Makeconfig immediately after defining
	subdir.
	* debug/Makefile: Likewise.
	* dirent/Makefile: Likewise.
	* dlfcn/Makefile: Likewise.
	* gmon/Makefile: Likewise.
	* gnulib/Makefile: Likewise.
	* grp/Makefile: Likewise.
	* gshadow/Makefile: Likewise.
	* hesiod/Makefile: Likewise.
	* hurd/Makefile: Likewise.
	(all): Remove target.
	* iconvdata/Makefile: Include Makeconfig immediately after
	defining subdir.
	* inet/Makefile: Likewise.
	* intl/Makefile: Likewise.
	* io/Makefile: Likewise.
	* libio/Makefile: Likewise.
	(all): Remove target.
	* locale/Makefile: Include Makeconfig immediately after defining
	subdir.
	* login/Makefile: Likewise.
	* mach/Makefile: Likewise.
	(all): Remove target.
	* malloc/Makefile: Include Makeconfig immediately after defining
	subdir.
	(all): Remove target.
	* manual/Makefile: Include Makeconfig immediately after defining
	subdir.
	* math/Makefile: Likewise.
	* misc/Makefile: Likewise.
	* nis/Makefile: Likewise.
	* nss/Makefile: Likewise.
	* po/Makefile: Likewise.
	(all): Remove target.
	* posix/Makefile: Include Makeconfig immediately after defining
	subdir.
	* pwd/Makefile: Likewise.
	* resolv/Makefile: Likewise.
	* resource/Makefile: Likewise.
	* rt/Makefile: Likewise.
	* setjmp/Makefile: Likewise.
	* shadow/Makefile: Likewise.
	* signal/Makefile: Likewise.
	* socket/Makefile: Likewise.
	* soft-fp/Makefile: Likewise.
	* stdio-common/Makefile: Likewise.
	* stdlib/Makefile: Likewise.
	* streams/Makefile: Likewise.
	* string/Makefile: Likewise.
	* sunrpc/Makefile: Likewise.
	(all): Remove target.
	* sysvipc/Makefile: Include Makeconfig immediately after defining
	subdir.
	* termios/Makefile: Likewise.
	* time/Makefile: Likewise.
	* timezone/Makefile: Likewise.
	(all): Remove target.
	* wcsmbs/Makefile: Include Makeconfig immediately after defining
	subdir.
	* wctype/Makefile: Likewise.

libidn/ChangeLog:
	* Makefile: Include Makeconfig immediately after defining subdir.

localedata/ChangeLog:
	* Makefile: Include Makeconfig immediately after defining subdir.
	(all): Remove target.

nptl/ChangeLog:
	* Makefile: Include Makeconfig immediately after defining subdir.

nptl_db/ChangeLog:
	* Makefile: Include Makeconfig immediately after defining subdir.
2014-02-26 23:12:03 +00:00
Joseph Myers
f0881698bf Generate .test-result files for tests with special rules.
This patch, an updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00194.html> now
proposed for inclusion in glibc, extends the generation of PASS and
FAIL status in .test-result files for individual tests to cover tests
with their own custom makefile rules.  This is just adding
$(evaluate-test) calls to all such rules, since tests with multiple
commands were previously split into separate tests.

Note that the tests the makefiles expect to fail (posix/annexc and
conformtest) currently get FAIL listed in the .test-result file,
rather than XFAIL; a subsequent patch will introduce a better XFAIL
mechanism.

Tested x86_64.

	* Makefile ($(objpfx)c++-types-check.out): Use $(evaluate-test).
	($(objpfx)check-local-headers.out): Likewise.
	($(objpfx)begin-end-check.out): Likewise.
	* Makerules (check-abi-%.out): Likewise.
	* catgets/Makefile ($(objpfx)test1.cat): Likewise.
	($(objpfx)test2.cat): Likewise.
	($(objpfx)de/libc.cat): Likewise.
	($(objpfx)test-gencat.out): Likewise.
	* conform/Makefile ($(objpfx)run-conformtest.out): Likewise.
	* elf/Makefile ($(objpfx)order-cmp.out): Likewise.
	($(objpfx)noload-mem): Likewise.
	($(objpfx)tst-pathopt.out): Likewise.
	($(objpfx)tst-rtld-load-self.out): Likewise.
	($(objpfx)tst-array1-cmp.out): Likewise.
	($(objpfx)tst-array1-static-cmp.out): Likewise.
	($(objpfx)tst-array2-cmp.out): Likewise.
	($(objpfx)tst-array3-cmp.out): Likewise.
	($(objpfx)tst-array4-cmp.out): Likewise.
	($(objpfx)tst-array5-cmp.out): Likewise.
	($(objpfx)tst-array5-static-cmp.out): Likewise.
	($(objpfx)check-textrel.out): Likewise.
	($(objpfx)check-execstack.out): Likewise.
	($(objpfx)check-localplt.out): Likewise.
	($(objpfx)order2-cmp.out): Likewise.
	($(objpfx)tst-leaks1-mem): Likewise.
	($(objpfx)tst-leaks1-static-mem): Likewise.
	($(objpfx)tst-initorder-cmp.out): Likewise.
	($(objpfx)tst-initorder2-cmp.out): Likewise.
	($(objpfx)tst-unused-dep.out): Likewise.
	($(objpfx)tst-unused-dep-cmp.out): Likewise.
	* grp/Makefile ($(objpfx)tst_fgetgrent.out): Likewise.
	* iconv/Makefile (test-iconvconfig): Likewise.
	* iconvdata/Makefile ($(objpfx)mtrace-tst-loading): Likewise.
	($(objpfx)iconv-test.out): Likewise.
	($(objpfx)tst-tables.out): Likewise.
	* intl/Makefile ($(objpfx)mtrace-tst-gettext): Likewise.
	($(objpfx)tst-gettext.out): Likewise.
	($(objpfx)tst-translit.out): Likewise.
	($(objpfx)tst-gettext2.out): Likewise.
	($(objpfx)tst-gettext4.out): Likewise.
	($(objpfx)tst-gettext6.out): Likewise.
	* io/Makefile ($(objpfx)ftwtest.out): Likewise.
	* libio/Makefile ($(objpfx)test-freopen.out): Likewise.
	($(objpfx)tst-fopenloc-cmp.out): Likewise.
	($(objpfx)tst-fopenloc-mem.out): Likewise.
	* malloc/Makefile ($(objpfx)tst-mtrace.out): Likewise.
	* misc/Makefile ($(objpfx)tst-error1-mem): Likewise.
	* posix/Makefile ($(objpfx)globtest.out): Likewise.
	($(objpfx)wordexp-tst.out): Likewise.
	($(objpfx)annexc.out): Likewise.
	($(objpfx)tst-fnmatch-mem): Likewise.
	($(objpfx)bug-regex2-mem): Likewise.
	($(objpfx)bug-regex14-mem): Likewise.
	($(objpfx)bug-regex21-mem): Likewise.
	($(objpfx)bug-regex31-mem): Likewise.
	($(objpfx)tst-vfork3-mem): Likewise.
	($(objpfx)tst-rxspencer-no-utf8-mem): Likewise.
	($(objpfx)tst-pcre-mem): Likewise.
	($(objpfx)tst-boost-mem): Likewise.
	($(objpfx)tst-getconf.out): Likewise.
	($(objpfx)bug-ga2-mem): Likewise.
	($(objpfx)bug-glob2-mem): Likewise.
	* resolv/Makefile ($(objpfx)mtrace-tst-leaks): Likewise.
	($(objpfx)mtrace-tst-leaks2): Likewise.
	* stdio-common/Makefile ($(objpfx)tst-unbputc.out): Likewise.
	($(objpfx)tst-printf.out): Likewise.
	($(objpfx)tst-setvbuf1.out): Likewise.
	($(objpfx)tst-setvbuf1-cmp.out): Likewise.
	* stdlib/Makefile ($(objpfx)isomac.out): Likewise.
	($(objpfx)tst-fmtmsg.out): Likewise.
	* string/Makefile ($(objpfx)tst-svc-cmp.out): Likewise.
	* sysdeps/x86/Makefile ($(objpfx)tst-xmmymm.out): Likewise.

localedata:
	* Makefile ($(objpfx)sort-test.out): Use $(evaluate-test).
	($(objpfx)tst-fmon.out): Likewise.
	($(objpfx)tst-numeric.out): Likewise.
	($(objpfx)tst-locale.out): Likewise.
	($(objpfx)tst-rpmatch.out): Likewise.
	($(objpfx)tst-trans.out): Likewise.
	($(objpfx)tst-mbswcs.out): Likewise.
	($(objpfx)tst-ctype.out): Likewise.
	($(objpfx)tst-wctype.out): Likewise.
	($(objpfx)tst-langinfo.out): Likewise.
	($(objpfx)mtrace-tst-leaks): Likewise.

nptl:
	* Makefile ($(objpfx)tst-stack3-mem): Use $(evaluate-test).
	($(objpfx)tst-tls6.out): Likewise.
	($(objpfx)tst-cleanup0.out): Likewise.
	($(objpfx)tst-cleanup0-cmp.out): Likewise.
	($(objpfx)tst-cancel-wrappers.out): Likewise.
	($(objpfx)tst-oddstacklimit.out): Likewise.

nptl_db:
	* Makefile ($(objpfx)db-symbols.out): Use
	$(evaluate-test).
2014-02-21 21:48:08 +00:00
Allan McRae
d4697bc93d Update copyright notices with scripts/update-copyrights 2014-01-01 22:00:23 +10:00
Joseph Myers
4fd2cf9f5e conformtest: Clean up expectations for POSIX for sched.h. 2013-09-25 00:51:42 +00:00
Joseph Myers
a03d8ea77f conformtest: Clean up expectations for POSIX for pthread.h. 2013-09-24 22:09:04 +00:00
Joseph Myers
b92a20b560 conformtest: Fix namespace testing. 2013-09-04 20:47:21 +00:00
Joseph Myers
568035b787 Update copyright notices with scripts/update-copyrights. 2013-01-02 19:05:09 +00:00
Joseph Myers
116fc08a61 conformtest: Clean up expectations for POSIX for more headers. 2012-11-19 22:21:47 +00:00
Joseph Myers
a483863fc1 conformtest: Clean up expectations for POSIX for various headers. 2012-11-19 22:20:44 +00:00
Joseph Myers
5ba924e3c7 conformtest: Fix miscellaneous expectations for XPG4. 2012-11-19 22:19:11 +00:00
Joseph Myers
3d2577bba5 conformtest: Fix miscellaneous UNIX98 expectations. 2012-11-10 01:13:39 +00:00
Joseph Myers
c4b6cf53d5 conformtest: Fix pthreads expectations for XPG3 / XPG4 / UNIX98. 2012-11-07 19:42:20 +00:00
Joseph Myers
cbe6e12006 conformtest: Fix setenv / unsetenv expectations. 2012-11-07 19:41:38 +00:00
Joseph Myers
1b12644333 conformtest: Fix isnan function return type. 2012-11-07 19:41:03 +00:00
Joseph Myers
b961a5737a conformtest: Fix sys/mman.h expectations for UNIX98. 2012-11-07 19:40:18 +00:00
Joseph Myers
9e188909df conformtest: Only expect mknodat for XOPEN2K8. 2012-11-07 19:37:52 +00:00
Joseph Myers
05bcf62afb conformtest: Clean up / correct / expand C99 and C11 expectations. 2012-11-07 19:36:48 +00:00
Joseph Myers
45832f7473 conformtest: Fix typo in CLK_TCK condition. 2012-11-07 04:05:12 +00:00
Joseph Myers
a68d0680f8 conformtest: Add test data for fenv.h. 2012-11-02 23:21:36 +00:00
Joseph Myers
caf7f573e9 conformtest: Enable complex.h and inttypes.h testing for XOPEN2K8. 2012-11-02 23:20:43 +00:00
Joseph Myers
6c073ad6fe conformtest: Fix sys/wait.h expectations for struct rusage. 2012-11-02 22:12:09 +00:00
Joseph Myers
bf9e20711e conformtest: Update expectations for time.h. 2012-07-24 19:43:38 +00:00
Joseph Myers
8048311a50 conformtest: Update expectations for string.h. 2012-07-15 22:38:11 +00:00
Joseph Myers
6bcc8b3ff9 conformtest: Update expectations for stdio.h and stdlib.h. 2012-05-24 21:58:04 +00:00
Joseph Myers
b1cc2472d2 conformtest: Update expectations for more ISO C headers. 2012-05-09 10:05:14 +00:00
Joseph Myers
615605c94e conformtest: Update expectations for some ISO C headers. 2012-05-01 20:05:49 +00:00
Joseph Myers
c9140a622e conformtest: Check namespace within __attribute__. 2012-05-01 20:05:14 +00:00
Joseph Myers
9af0bf29de conformtest: Use correct keyword set for underlying C standard. 2012-05-01 20:04:44 +00:00
Joseph Myers
343222a2a0 conformtest: Don't test non-C90 headers for C90. 2012-05-01 20:04:15 +00:00
Joseph Myers
661f8cf03b conformtest: Support testing macro usability in #if. 2012-05-01 20:03:42 +00:00
Joseph Myers
ee74b9cbbd conformtest: Support specifying types before promotion. 2012-05-01 20:03:15 +00:00
Joseph Myers
aafc49b3fc conformtest: Unify "macro" and "constant" handling. 2012-05-01 20:02:48 +00:00
Joseph Myers
fefdf57412 conformtest: Unify "macro" cases. 2012-05-01 20:02:19 +00:00
Joseph Myers
f2d922feb2 conformtest: Unify "constant" and "typed-constant". 2012-05-01 20:01:48 +00:00
Joseph Myers
d22956c99b conformtest: Unify handling of "optional-" and non-optional lines. 2012-05-01 20:00:56 +00:00
Joseph Myers
028e2e3879 conformtest: Unify variants of "constant" and "optional-constant". 2012-05-01 20:00:24 +00:00
Joseph Myers
1b8f2850d8 conformtest: Always pass C standards options to compiler. 2012-05-01 19:59:47 +00:00
Joseph Myers
a05a144b13 conformtest: Improve math.h tests for X/Open versions. 2012-05-01 19:59:15 +00:00
Joseph Myers
73c5ebe344 conformtest: Use correct feature test macros for XPG4. 2012-05-01 19:58:35 +00:00
Joseph Myers
39c33b6cbc conformtest: Handle "symbol" lines for allow-header. 2012-05-01 19:58:06 +00:00
Joseph Myers
0749ff8b81 conformtest: Remove duplicate typed-constant handling. 2012-04-28 01:42:06 -07:00
Joseph Myers
41d73a1bc0 conformtest: Fix typo in handling typed-constant from allow-header. 2012-04-28 01:27:20 -07:00
David S. Miller
b855ab8539 Run conformtest script explicitly using BASH.
* conform/Makefile: Run run-conformtest.sh using $(BASH).
2012-03-27 15:02:19 -07:00
Thomas Schwinge
8149f97606 Don't tamper with $HOME. 2012-03-21 18:10:03 +01:00
Ulrich Drepper
a4300c7a4d Remove distribute variable from Makefiles 2012-03-07 05:17:13 -05:00
Ulrich Drepper
ba63ba0882 Restrict run-conformtest runs 2012-02-26 21:45:44 -05:00
Ulrich Drepper
d94a467080 Add first fixes for conformtest for POSIX2008 2012-02-26 21:32:56 -05:00