I noticed that some of the Depend files, used to determine the
subdirectory build order in sysd-sorted, still mentioned linuxthreads,
although it hasn't been supported for many years. This patch removes
those references. In the case of nscd, it substitutes an nptl
reference, since I believe there is a fact a thread library dependence
there; the others already mentioned nptl.
Note that I am not at all confident in the completeness of these
Depend files.
Note also that references to linuxthreads remain in a comment in
sysdeps/unix/sysv/linux/ia64/Versions, and in manual/maint.texi,
manual/signal.texi and scripts/documented.sh.
Tested x86_64 that the installed shared libraries are unchanged by the
patch (as is sysd-sorted).
* nscd/Depend (linuxthreads): Remove.
(nptl): Add.
* resolv/Depend (linuxthreads): Remove.
* rt/Depend (linuxthreads): Remove.
[Fixes BZ #14308, #12994, #13651]
AF_UNSPEC results in sending two queries in parallel, one for the A
record and the other for the AAAA record. If one of these is a
referral, then the query fails, which is wrong. It should return at
least the one successful response.
The fix has two parts. The first part makes the referral fall back to
the SERVFAIL path, which results in using the successful response.
There is a bug in that path however, due to which the second part is
necessary. The bug here is that if the first response is a failure
and the second succeeds, __libc_res_nsearch does not detect that and
assumes a failure. The case where the first response is a success and
the second fails, works correctly.
This condition is produced by buggy routers, so here's a crude
interposable library that can simulate such a condition. The library
overrides the recvfrom syscall and modifies the header of the packet
received to reproduce this scenario. It has two key variables:
mod_packet and first_error.
The mod_packet variable when set to 0, results in odd packets being
modified to be a referral. When set to 1, even packets are modified
to be a referral.
The first_error causes the first response to be a failure so that a
domain-appended search is performed to test the second part of the
__libc_nsearch fix.
The driver for this fix is a simple getaddrinfo program that does an
AF_UNSPEC query. I have omitted this since it should be easy to
implement.
I have tested this on x86_64.
The interceptor library source:
/* Override recvfrom and modify the header of the first DNS response to make it
a referral and reproduce bz #845218. We have to resort to this ugly hack
because we cannot make bind return the buggy response of a referral for the
AAAA record and an authoritative response for the A record. */
#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <stdbool.h>
#include <endian.h>
#include <dlfcn.h>
#include <stdlib.h>
/* Lifted from resolv/arpa/nameser_compat.h. */
typedef struct {
unsigned id :16; /*%< query identification number */
#if BYTE_ORDER == BIG_ENDIAN
/* fields in third byte */
unsigned qr: 1; /*%< response flag */
unsigned opcode: 4; /*%< purpose of message */
unsigned aa: 1; /*%< authoritive answer */
unsigned tc: 1; /*%< truncated message */
unsigned rd: 1; /*%< recursion desired */
/* fields
* in
* fourth
* byte
* */
unsigned ra: 1; /*%< recursion available */
unsigned unused :1; /*%< unused bits (MBZ as of 4.9.3a3) */
unsigned ad: 1; /*%< authentic data from named */
unsigned cd: 1; /*%< checking disabled by resolver */
unsigned rcode :4; /*%< response code */
#endif
#if BYTE_ORDER == LITTLE_ENDIAN || BYTE_ORDER == PDP_ENDIAN
/* fields
* in
* third
* byte
* */
unsigned rd :1; /*%< recursion desired */
unsigned tc :1; /*%< truncated message */
unsigned aa :1; /*%< authoritive answer */
unsigned opcode :4; /*%< purpose of message */
unsigned qr :1; /*%< response flag */
/* fields
* in
* fourth
* byte
* */
unsigned rcode :4; /*%< response code */
unsigned cd: 1; /*%< checking disabled by resolver */
unsigned ad: 1; /*%< authentic data from named */
unsigned unused :1; /*%< unused bits (MBZ as of 4.9.3a3) */
unsigned ra :1; /*%< recursion available */
#endif
/* remaining
* bytes
* */
unsigned qdcount :16; /*%< number of question entries */
unsigned ancount :16; /*%< number of answer entries */
unsigned nscount :16; /*%< number of authority entries */
unsigned arcount :16; /*%< number of resource entries */
} HEADER;
static int done = 0;
/* Packets to modify. 0 for the odd packets and 1 for even packets. */
static const int mod_packet = 0;
/* Set to true if the first request should result in an error, resulting in a
search query. */
static bool first_error = true;
static ssize_t (*real_recvfrom) (int sockfd, void *buf, size_t len, int flags,
struct sockaddr *src_addr, socklen_t *addrlen);
void
__attribute__ ((constructor))
init (void)
{
real_recvfrom = dlsym (RTLD_NEXT, "recvfrom");
if (real_recvfrom == NULL)
{
printf ("Failed to get reference to recvfrom: %s\n", dlerror ());
printf ("Cannot simulate test\n");
abort ();
}
}
/* Modify the second packet that we receive to set the header in a manner as to
reproduce BZ #845218. */
static void
mod_buf (HEADER *h, int port)
{
if (done % 2 == mod_packet || (first_error && done == 1))
{
printf ("(Modifying header)");
if (first_error && done == 1)
h->rcode = 3;
else
h->rcode = 0; /* NOERROR == 0. */
h->ancount = 0;
h->aa = 0;
h->ra = 0;
h->arcount = 0;
}
done++;
}
ssize_t
recvfrom (int sockfd, void *buf, size_t len, int flags,
struct sockaddr *src_addr, socklen_t *addrlen)
{
ssize_t ret = real_recvfrom (sockfd, buf, len, flags, src_addr, addrlen);
int port = htons (((struct sockaddr_in *) src_addr)->sin_port);
struct in_addr addr = ((struct sockaddr_in *) src_addr)->sin_addr;
const char *host = inet_ntoa (addr);
printf ("\n*** From %s:%d: ", host, port);
mod_buf (buf, port);
printf ("returned %zd\n", ret);
return ret;
}
This patch defines _STRING_ARCH_unaligned to 0 on default bits/string.h
header to avoid undefined compiler warnings on platforms that do not
define it. It also make adjustments in code where tests checked if macro
existed or not.
This patch systematically renames miscellaneous tests so their outputs
use a *.out name (unless the test is just running some glibc program
with its conventional output file name, rather than a special program
at all, as in catgets tests generating *.cat). In the case of the
iconv test test-iconvconfig, output is redirected where it wasn't
before.
In various places the "generated" variable is updated to reflect the
revised test names; in iconvdata/Makefile a typo (mmtrace-tst-loading)
is also fixed. resolv/Makefile sets both "generate" (which appears
unused) and "generated". Bitrot in the settings of these variables
could no doubt be fixed so that "make clean" after build and testing
leaves results the same as after configure (and indeed the
tests-special / xtests-special variables could be used to simplify
things, by removing those files automatically rather than listing them
manually in these variables), and "make distclean" leaves an empty
build directory, but right now it appears various files don't get
deleted. I think they are liable to continue to bitrot in the absence
of routine testing that these targets actually work, given that
building in the source directory isn't supported and that was the main
use of such makefile targets.
Tested x86_64.
* elf/Makefile (tests-special): Rename tests to end with .out.
($(objpfx)noload-mem): Likewise.
($(objpfx)tst-leaks1-mem): Likewise.
($(objpfx)tst-leaks1-static-mem.out): Likewise.
* iconv/Makefile (xtests-special): Change test-iconvconfig to
$(objpfx)test-iconvconfig.out.
(test-iconvconfig): Change to $(objpfx)test-iconvconfig.out. Use
set -e inside subshell and redirect output to file.
* iconvdata/Makefile (generated): Rename tests to end with .out.
Correct type.
(tests-special): Rename tests to end with .out.
($(objpfx)mtrace-tst-loading): Likewise.
* intl/Makefile (generated): Likewise.
(tests-special): Likewise.
($(objpfx)mtrace-tst-gettext): Likewise.
* misc/Makefile (generated): Likewise.
(tests-special): Likewise.
($(objpfx)tst-error1-mem): Likewise.
* nptl/Makefile (tests-special): Likewise.
($(objpfx)tst-stack3-mem): Likewise.
(generated): Likewise.
* posix/Makefile (generated): Likewise.
(tests-special): Likewise.
(xtests-special): Likewise.
($(objpfx)tst-fnmatch-mem): Likewise.
($(objpfx)bug-regex2-mem): Likewise.
($(objpfx)bug-regex14-mem): Likewise.
($(objpfx)bug-regex21-mem): Likewise.
($(objpfx)bug-regex31-mem): Likewise.
($(objpfx)tst-vfork3-mem): Likewise.
($(objpfx)tst-rxspencer-no-utf8-mem): Likewise.
($(objpfx)tst-pcre-mem): Likewise.
($(objpfx)tst-boost-mem): Likewise.
($(objpfx)bug-ga2-mem): Likewise.
($(objpfx)bug-glob2-mem): Likewise.
* resolv/Makefile (generate): Likewise.
(tests-special): Likewise.
(xtests-special): Likewise.
(generated): Likewise.
($(objpfx)mtrace-tst-leaks): Likewise.
($(objpfx)mtrace-tst-leaks2): Likewise.
localedata:
* Makefile (generated): Rename tests to end with .out.
(tests-special): Likewise.
($(objpfx)mtrace-tst-leaks): Likewise.
This patch is a revised and updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html>.
In order to generate overall summaries of the results of all tests in
the glibc testsuite, we need to identify and concatenate the files
with the results of individual tests.
Tomas Dohnalek's patch used $(common-objpfx)*/*.test-result for this.
However, the normal glibc approach is explicit enumeration of the
expected set of files with a given property, rather than all files
matching some pattern like that. Furthermore, we would like to be
able to mark tests as UNRESOLVED if the file with their results is for
some reason missing, and in future we would like to be able to mark
tests as UNSUPPORTED if they are disabled for a particular
configuration (rather than simply having them missing from the list of
tests as at present). Such handling of tests that were not run or did
not record results requires an explicit enumeration of tests.
For the tests following the default makefile rules, $(tests) (and
$(xtests)) provides such an enumeration. Others, however, are added
directly as dependencies of the "tests" and "xtests" makefile
targets. This patch changes the makefiles to put them in variables
tests-special and xtests-special, with appropriate dependencies on the
tests listed there then being added centrally.
Those variables are used in Rules and so need to be set before Rules
is included in a subdirectory makefile, which is often earlier in the
makefile than the dependencies were present before. We previously
discussed the question of where to include Rules; see the question at
<https://sourceware.org/ml/libc-alpha/2012-11/msg00798.html>, and a
discussion in
<https://sourceware.org/ml/libc-alpha/2013-01/msg00337.html> of why
Rules is included early rather than late in subdirectory makefiles.
It was necessary to avoid an indirection through the check-abi target
and get the check-abi-* targets for individual libraries into the
tests-special variable. The intl/ test $(objpfx)tst-gettext.out,
previously built only because of dependencies from other tests, was
also added to tests-special for the same reason.
The entries in tests-special are the full makefile targets, complete
with $(objpfx) and .out. If a future change causes tests to be named
consistently with a .out suffix, this can be changed to include just
the path relative to $(objpfx), without .out.
Tested x86_64, including that the same set of files is generated in
the build directory by a build and testsuite run both before and after
the patch (except for changes to the
elf/tst-null-argv.debug.out.<number> file name), and a build with
run-built-tests=no to verify there aren't any more obvious instances
of the issue Marcus Shawcroft reported with a previous version in
<https://sourceware.org/ml/libc-alpha/2014-01/msg00462.html>.
* Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(tests): Depend on $(tests-special).
* Makerules (check-abi-list): New variable.
(check-abi): Depend on $(check-abi-list).
[$(subdir) = elf] (tests-special): Add
$(objpfx)check-abi-libc.out.
[$(build-shared) = yes && subdir] (tests-special): Add
$(check-abi-list).
[$(build-shared) = yes && subdir] (tests): Do not depend on
check-abi.
* Rules (tests): Depend on $(tests-special).
(xtests): Depend on $(xtests-special).
* catgets/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* conform/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* elf/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* grp/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* iconv/Makefile (xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* iconvdata/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* intl/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable. Also add
$(objpfx)tst-gettext.out.
* io/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* libio/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* malloc/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* misc/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* nptl/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* nptl_db/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* posix/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* resolv/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* stdio-common/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(do-tst-unbputc): Remove target.
(do-tst-printf): Likewise.
* stdlib/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* string/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* sysdeps/x86/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
localedata:
* Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
In <https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html> I
noted it was necessary to add includes of Makeconfig early in various
subdirectory makefiles for the tests-special variable settings added
by that patch to be conditional on configuration information. No-one
commented on the general question there of whether Makeconfig should
always be included immediately after the definition of subdir.
This patch implements that early inclusion of Makeconfig in each
directory (which is a lot easier than consistent placement of includes
of Rules). Includes are added if needed, or moved up if already
present. Subdirectory "all:" targets are removed, since Makeconfig
provides one.
There is potential for further cleanups I haven't done. Rules and
Makerules have code such as
ifneq "$(findstring env,$(origin headers))" ""
headers :=
endif
to override to empty any value of various variables that came from the
environment. I think there is a case for Makeconfig setting all the
subdirectory variables (other than subdir) to empty to ensure no
outside value is going to take effect if a subdirectory fails to
define a variable. (A list of such variables, possibly out of date
and incomplete, is in manual/maint.texi.) Rules and Makerules would
give errors if Makeconfig hadn't already been included, instead of
including it themselves. The special code to override values coming
from the environment would then be obsolete and could be removed.
Tested x86_64, including that installed binaries are identical before
and after the patch.
* argp/Makefile: Include Makeconfig immediately after defining
subdir.
* assert/Makefile: Likewise.
* benchtests/Makefile: Likewise.
* catgets/Makefile: Likewise.
* conform/Makefile: Likewise.
* crypt/Makefile: Likewise.
* csu/Makefile: Likewise.
(all): Remove target.
* ctype/Makefile: Include Makeconfig immediately after defining
subdir.
* debug/Makefile: Likewise.
* dirent/Makefile: Likewise.
* dlfcn/Makefile: Likewise.
* gmon/Makefile: Likewise.
* gnulib/Makefile: Likewise.
* grp/Makefile: Likewise.
* gshadow/Makefile: Likewise.
* hesiod/Makefile: Likewise.
* hurd/Makefile: Likewise.
(all): Remove target.
* iconvdata/Makefile: Include Makeconfig immediately after
defining subdir.
* inet/Makefile: Likewise.
* intl/Makefile: Likewise.
* io/Makefile: Likewise.
* libio/Makefile: Likewise.
(all): Remove target.
* locale/Makefile: Include Makeconfig immediately after defining
subdir.
* login/Makefile: Likewise.
* mach/Makefile: Likewise.
(all): Remove target.
* malloc/Makefile: Include Makeconfig immediately after defining
subdir.
(all): Remove target.
* manual/Makefile: Include Makeconfig immediately after defining
subdir.
* math/Makefile: Likewise.
* misc/Makefile: Likewise.
* nis/Makefile: Likewise.
* nss/Makefile: Likewise.
* po/Makefile: Likewise.
(all): Remove target.
* posix/Makefile: Include Makeconfig immediately after defining
subdir.
* pwd/Makefile: Likewise.
* resolv/Makefile: Likewise.
* resource/Makefile: Likewise.
* rt/Makefile: Likewise.
* setjmp/Makefile: Likewise.
* shadow/Makefile: Likewise.
* signal/Makefile: Likewise.
* socket/Makefile: Likewise.
* soft-fp/Makefile: Likewise.
* stdio-common/Makefile: Likewise.
* stdlib/Makefile: Likewise.
* streams/Makefile: Likewise.
* string/Makefile: Likewise.
* sunrpc/Makefile: Likewise.
(all): Remove target.
* sysvipc/Makefile: Include Makeconfig immediately after defining
subdir.
* termios/Makefile: Likewise.
* time/Makefile: Likewise.
* timezone/Makefile: Likewise.
(all): Remove target.
* wcsmbs/Makefile: Include Makeconfig immediately after defining
subdir.
* wctype/Makefile: Likewise.
libidn/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
localedata/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
(all): Remove target.
nptl/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
nptl_db/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
limit
[BZ #14307]
* sysdeps/posix/getaddrinfo.c (gaih_inet): Increase the size of
the temporary buffer used to invoke __gethostbyname2_r,
__gethostbyaddr_r and gethostbyname4_r to make room for struct
host_data / struct gaih_addrtuple.
* resolv/nss_dns/dns-host.c (global scope): Move definition of
implementation constants MAX_NR_ALIASES and MAX_NR_ADDRS to
header file nss/nsswitch.h.
* nss/nsswitch.h (global scope): Add definition of implementation
constants MAX_NR_ALIASES and MAX_NR_ADDRS (moved from
resolv/nss_dns/dns-host.c).
[BZ #13928] A DNS request consists of multiple resources combined into
a single hostent, including multiple CNAME records that may have been
assigned different TTL values. In such a case, nscd should take the
least TTL among all of the resources as the timeout for the hostent
before it is reloaded in its cache so that the hostent remains stale
in the database for the least amount of time.
