This change adds a warning to nscd.conf about running multiple caching
services together and that it may lead to unexpected behaviours. Also we
add a note that enabling the 'shared' option will cause cache hit rates
to be misreported (a side effect of the implementation).
v2
- Rewrite comment to avoid implementation details.
The function uses the internal service_user type, so it is not
really usable from the outside of glibc. Rename the function
to __nss_database_lookup2 for internal use, and change
__nss_database_lookup to always indicate failure to the caller.
__nss_next already was a compatibility symbol. The new
implementation always fails and no longer calls __nss_next2.
unscd, the alternative nscd implementation, does not use
__nss_database_lookup, so it is not affected by this change.
This functionality was deprecated in glibc 2.25.
This commit only includes the core changes to remove the
functionality. It does not remove the RES_USE_INET6 handling in the
individual NSS service modules and the res_use_inet6 function.
These changes will happen in future commits.
I'm looking at the warnings from building glibc with -Wextra, to see
if we could use -Wextra by default, possibly with a few of its
warnings disabled, and so benefit from warnings in -Wextra but not in
-Wall. (The vast bulk of the extra warnings so produced are from
-Wunused-parameter -Wsign-compare -Wmissing-field-initializers
-Wtype-limits, so I expect those would be disabled at least at first.)
Various miscellaneous warnings show up with -Wextra that it clearly
seems to make sense to fix independent of whether we add -Wextra to
the normal options for building glibc. This patch fixes one:
"initialized field overwritten [-Woverride-init]" in nscd.
Tested for x86_64.
* nscd/connections.c (reqinfo): Initialize SHUTDOWN element only
once.
The IPv4 address parser in the getaddrinfo function is changed so that
it does not ignore trailing whitespace and all characters after it.
For backwards compatibility, the getaddrinfo function still recognizes
legacy name syntax, such as 192.000.002.010 interpreted as 192.0.2.8
(octal).
This commit does not change the behavior of inet_addr and inet_aton.
gethostbyname already had additional sanity checks (but is switched
over to the new __inet_aton_exact function for completeness as well).
To avoid sending the problematic query names over DNS, commit
6ca53a2453 ("resolv: Do not send queries
for non-host-names in nss_dns [BZ #24112]") is needed.
This is a major rewrite of the description of 'crypt', 'getentropy',
and 'getrandom'.
A few highlights of the content changes:
- Throughout the manual, public headers, and user-visible messages,
I replaced the term "password" with "passphrase", the term
"password database" with "user database", and the term
"encrypt(ion)" with "(one-way) hashing" whenever it was applied to
passphrases. I didn't bother making this change in internal code
or tests. The use of the term "password" in ruserpass.c survives,
because that refers to a keyword in netrc files, but it is adjusted
to make this clearer.
There is a note in crypt.texi explaining that they were
traditionally called passwords but single words are not good enough
anymore, and a note in users.texi explaining that actual passphrase
hashes are found in a "shadow" database nowadays.
- There is a new short introduction to the "Cryptographic Functions"
section, explaining how we do not intend to be a general-purpose
cryptography library, and cautioning that there _are_, or have
been, legal restrictions on the use of cryptography in many
countries, without getting into any kind of detail that we can't
promise to keep up to date.
- I added more detail about what a "one-way function" is, and why
they are used to obscure passphrases for storage. I removed the
paragraph saying that systems not connected to a network need no
user authentication, because that's a pretty rare situation
nowadays. (It still says "sometimes it is necessary" to
authenticate the user, though.)
- I added documentation for all of the hash functions that glibc
actually supports, but not for the additional hash functions
supported by libxcrypt. If we're going to keep this manual section
around after the transition is more advanced, it would probably
make sense to add them then.
- There is much more detailed discussion of how to generate a salt,
and the failure behavior for crypt is documented. (Returning an
invalid hash on failure is what libxcrypt does; Solar Designer's
notes say that this was done "for compatibility with old programs
that assume crypt can never fail".)
- As far as I can tell, the header 'crypt.h' is entirely a GNU
invention, and never existed on any other Unix lineage. The
function 'crypt', however, was in Issue 1 of the SVID and is now
in the XSI component of POSIX. I tried to make all of the
@standards annotations consistent with this, but I'm not sure I got
them perfectly right.
- The genpass.c example has been improved to use getentropy instead
of the current time to generate the salt, and to use a SHA-256 hash
instead of MD5. It uses more random bytes than is strictly
necessary because I didn't want to complicate the code with proper
base64 encoding.
- The testpass.c example has three hardwired hashes now, to
demonstrate that different one-way functions produce different
hashes for the same input. It also demonstrates how DES hashing
only pays attention to the first eight characters of the input.
- There is new text explaining in more detail how a CSPRNG differs
from a regular random number generator, and how
getentropy/getrandom are not exactly a CSPRNG. I tried not to make
specific falsifiable claims here. I also tried to make the
blocking/cancellation/error behavior of both getentropy and
getrandom clearer.
The pre-allocation of the three scratch buffers increased the initial
stack size somewhat, but if retries are needed, the previous version
used more stack space if extend_alloca could not merge allocations.
Lack of alloca accounting also means could be problematic with
extremely large NSS responses, too.
[BZ #18023]
* nscd/aicache.c (addhstaiX): Use struct scratch_buffer instead
of extend_alloca.
As indicated by BZ#23178, concurrent access on some files read by nscd
may result non expected data send through service requisition. This is
due 'sendfile' Linux implementation where for sockets with zero-copy
support, callers must ensure the transferred portions of the the file
reffered by input file descriptor remain unmodified until the reader
on the other end of socket has consumed the transferred data.
I could not find any explicit documentation stating this behaviour on
Linux kernel documentation. However man-pages sendfile entry [1] states
in NOTES the aforementioned remark. It was initially pushed on man-pages
with an explicit testcase [2] that shows changing the file used in
'sendfile' call prior the socket input data consumption results in
previous data being lost.
From commit message it stated on tested Linux version (3.15) only TCP
socket showed this issues, however on recent kernels (4.4) I noticed the
same behaviour for local sockets as well.
Since sendfile on HURD is a read/write operation and the underlying
issue on Linux, the straightforward fix is just remove sendfile use
altogether. I am really skeptical it is hitting some hotstop (there
are indication over internet that sendfile is helpfull only for large
files, more than 10kb) here to justify that extra code complexity or
to pursuit other possible fix (through memory or file locks for
instance, which I am not sure it is doable).
Checked on x86_64-linux-gnu.
[BZ #23178]
* nscd/nscd-client.h (sendfileall): Remove prototype.
* nscd/connections.c [HAVE_SENDFILE] (sendfileall): Remove function.
(handle_request): Use writeall instead of sendfileall.
* nscd/aicache.c (addhstaiX): Likewise.
* nscd/grpcache.c (cache_addgr): Likewise.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/netgroupcache.c (addgetnetgrentX, addinnetgrX): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise.
* nscd/servicescache.c (cache_addserv): Likewise.
* sysdeps/unix/sysv/linux/Makefile [$(subdir) == nscd]
(sysdep-CFLAGS): Remove -DHAVE_SENDFILE.
* sysdeps/unix/sysv/linux/kernel-features.h (__ASSUME_SENDFILE):
Remove define.
[1] http://man7.org/linux/man-pages/man2/sendfile.2.html
[2] 7b6a329977 (diff-efd6af3a70f0f07c578e85b51e83b3c3)
Contributed by
Agustina Arzille <avarzille@riseup.net>
Amos Jeffries <squid3@treenet.co.nz>
David Michael <fedora.dm0@gmail.com>
Marco Gerards <marco@gnu.org>
Marcus Brinkmann <marcus@gnu.org>
Neal H. Walfield <neal@gnu.org>
Pino Toscano <toscano.pino@tiscali.it>
Richard Braun <rbraun@sceen.net>
Roland McGrath <roland@gnu.org>
Samuel Thibault <samuel.thibault@ens-lyon.org>
Thomas DiModica <ricinwich@yahoo.com>
Thomas Schwinge <tschwinge@gnu.org>
* htl: New directory.
* sysdeps/htl: New directory.
* sysdeps/hurd/htl: New directory.
* sysdeps/i386/htl: New directory.
* sysdeps/mach/htl: New directory.
* sysdeps/mach/hurd/htl: New directory.
* sysdeps/mach/hurd/i386/htl: New directory.
* nscd/Depend, resolv/Depend, rt/Depend: Add htl dependency.
* sysdeps/mach/hurd/i386/Implies: Add mach/hurd/i386/htl imply.
* sysdeps/mach/hurd/i386/libpthread.abilist: New file.
Unlike other nscd caches, the netgroup cache contains two types of
records - those for "iterate through a netgroup" (i.e. setnetgrent())
and those for "is this user in this netgroup" (i.e. innetgr()),
i.e. full and partial records. The timeout code assumes these records
have the same key for the group name, so that the collection of records
that is "this netgroup" can be expired as a unit.
However, the keys are not the same, as the in-netgroup key is generated
by nscd rather than being passed to it from elsewhere, and is generated
without the trailing NUL. All other keys have the trailing NUL, and as
noted in the linked BZ, debug statements confirm that two keys for the
same netgroup are added to the cache with two different lengths.
The result of this is that as records in the cache expire, the purge
code only cleans out one of the two types of entries, resulting in
stale, possibly incorrect, and possibly inconsistent cache data.
The patch simply includes the existing NUL in the computation for the
key length ('key' points to the char after the NUL, and 'group' to the
first char of the group, so 'key-group' includes the first char to the
NUL, inclusive).
[BZ #22342]
* nscd/netgroupcache.c (addinnetgrX): Include trailing NUL in
key value.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP is Linux-only.
* nscd/connections.c (RWLOCK_INITIALIZER): Define to
PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP or
PTHREAD_RWLOCK_INITIALIZER if that is not available.
(dbs): Use RWLOCK_INITIALIZER instead of
PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP.
400669754d ('hurd: Fix nscd build') had the side effect of making
libc's freeaddrinfo expose freeifaddrs through __check_pf. We can just
move the renames to gai.c itself, along others.
* sysdeps/mach/hurd/check_pf.c (__getifaddrs, __freeifaddrs): Do not
define macros.
* nscd/gai.c (__getifaddrs): Define macro to getifaddrs.
(__freeifaddrs): Define macro to freeifaddrs.
Current GCC mainline detects that nscd calls readlink with the same
buffer for both input and output, which is not valid (those arguments
are both restrict-qualified in POSIX). This patch makes it use a
separate buffer for readlink's input (with a size that is sufficient
to avoid truncation, so there should be no problems with warnings
about possible truncation, though not strictly minimal, but much
smaller than the buffer for output) to avoid this problem.
Tested compilation for aarch64-linux-gnu with build-many-glibcs.py.
[BZ #22446]
* nscd/connections.c (handle_request) [SO_PEERCRED]: Use separate
buffers for readlink input and output.
Hide internal __nis_hash function to allow direct access within libc.so
and libc.a without using GOT nor PLT.
[BZ #18822]
* nscd/nscd_helper.c (__nis_hash): New prototype.
[BZ #22161]
* nscd/netgroupcache.c (addinnetgrX): Release read lock after
resetting timeout.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Andreas Schwab <schwab@suse.de>
A lock is held by mempool_allocate() when CACHEABLE is true; we
must release this lock if we exit early.
Mark internal nss symbols with attribute_hidden to allow direct access
within libc.so and libc.a without using GOT nor PLT.
Tested on x86-64 with and without --disable-nscd.
[BZ #18822]
* grp/initgroups.c (__nss_group_database): Removed.
(__nss_initgroups_database): Likewise.
* nscd/gai.c (__nss_hosts_database): Likewise.
* nss/XXX-lookup.c (DATABASE_NAME_SYMBOL): Likewise.
* posix/tst-rfc3484-2.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484-3.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484.c (__nss_hosts_database): Likewise.
* sysdeps/posix/getaddrinfo.c (__nss_hosts_database): Likewise.
* nss/getXXent.c (INTERNAL (REENTRANT_GETNAME)): Add
attribute_hidden.
* nss/nsswitch.c (__nss_database_custom): Define only if
USE_NSCD is defined.
(__nss_configure_lookup): Use __nss_database_custom only if
USE_NSCD is defined.
* nss/nsswitch.h (__nss_database_custom): Declare only if
USE_NSCD is defined. Add attribute_hidden.
(__nss_setent): Add attribute_hidden.
(__nss_endent): Likewise.
(__nss_getent_r): Likewise.
(__nss_getent): Likewise.
(DEFINE_DATABASE): Declare __nss_##arg##_database.
struct resolv_context objects provide a temporary resolver context
which does not change during a name lookup operation. Only when the
outmost context is created, the stub resolver configuration is
verified to be current (at present, only against previous res_init
calls). Subsequent attempts to obtain the context will reuse the
result of the initial verification operation.
struct resolv_context can also be extended in the future to store
data which needs to be deallocated during thread cancellation.
Many callers of __res_maybe_init also call _res_hconf_init.
Additional calls to the latter do not hurt because the function
does its work only once. (/etc/hosts.conf is not reloaded or
even checked for changes.) This means that we can simplify the
code by calling _res_hconf_init directly from __res_vinit.
cppflags-iterator.mk no longer has anything to do with CPPFLAGS; all
it does is set libof-$(foo) for a list of files. extra-modules.mk
does the same thing, but with a different input variable, and doesn't
let the caller control the module. Therefore, this patch gives
cppflags-iterator.mk a better name, removes extra-modules.mk, and
updates all uses of both.
* extra-modules.mk: Delete file.
* cppflags-iterator.mk: Rename to ...
* libof-iterator.mk: ...this. Adjust comments.
* Makerules, extra-lib.mk, benchtests/Makefile, elf/Makefile
* elf/rtld-Rules, iconv/Makefile, locale/Makefile, malloc/Makefile
* nscd/Makefile, sunrpc/Makefile, sysdeps/s390/Makefile:
Use libof-iterator.mk instead of cppflags-iterator.mk or
extra-modules.mk.
* benchtests/strcoll-inputs/filelist#en_US.UTF-8: Remove
extra-modules.mk and cppflags-iterator.mk, add libof-iterator.mk.
Simplify the Linux accept4 implementation based on the assumption
that it is available in some way. __ASSUME_ACCEPT4_SOCKETCALL was
previously unused, so remove it.
For ia64, the accept4 system call (and socket call) were backported
in kernel version 3.2.18. Reflect this in the installation
instructions.
posix/wordexp-test.c used libc-internal.h for PTR_ALIGN_DOWN; similar
to what was done with libc-diag.h, I have split the definitions of
cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and PTR_ALIGN_DOWN
to a new header, libc-pointer-arith.h.
It then occurred to me that the remaining declarations in libc-internal.h
are mostly to do with early initialization, and probably most of the
files including it, even in the core code, don't need it anymore. Indeed,
only 19 files actually need what remains of libc-internal.h. 23 others
need libc-diag.h instead, and 12 need libc-pointer-arith.h instead.
No file needs more than one of them, and 16 don't need any of them!
So, with this patch, libc-internal.h stops including libc-diag.h as
well as losing the pointer arithmetic macros, and all including files
are adjusted.
* include/libc-pointer-arith.h: New file. Define
cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and
PTR_ALIGN_DOWN here.
* include/libc-internal.h: Definitions of above macros
moved from here. Don't include libc-diag.h anymore either.
* posix/wordexp-test.c: Include stdint.h and libc-pointer-arith.h.
Don't include libc-internal.h.
* debug/pcprofile.c, elf/dl-tunables.c, elf/soinit.c, io/openat.c
* io/openat64.c, misc/ptrace.c, nptl/pthread_clock_gettime.c
* nptl/pthread_clock_settime.c, nptl/pthread_cond_common.c
* string/strcoll_l.c, sysdeps/nacl/brk.c
* sysdeps/unix/clock_settime.c
* sysdeps/unix/sysv/linux/i386/get_clockfreq.c
* sysdeps/unix/sysv/linux/ia64/get_clockfreq.c
* sysdeps/unix/sysv/linux/powerpc/get_clockfreq.c
* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c:
Don't include libc-internal.h.
* elf/get-dynamic-info.h, iconv/loop.c
* iconvdata/iso-2022-cn-ext.c, locale/weight.h, locale/weightwc.h
* misc/reboot.c, nis/nis_table.c, nptl_db/thread_dbP.h
* nscd/connections.c, resolv/res_send.c, soft-fp/fmadf4.c
* soft-fp/fmasf4.c, soft-fp/fmatf4.c, stdio-common/vfscanf.c
* sysdeps/ieee754/dbl-64/e_lgamma_r.c
* sysdeps/ieee754/dbl-64/k_rem_pio2.c
* sysdeps/ieee754/flt-32/e_lgammaf_r.c
* sysdeps/ieee754/flt-32/k_rem_pio2f.c
* sysdeps/ieee754/ldbl-128/k_tanl.c
* sysdeps/ieee754/ldbl-128ibm/k_tanl.c
* sysdeps/ieee754/ldbl-96/e_lgammal_r.c
* sysdeps/ieee754/ldbl-96/k_tanl.c, sysdeps/nptl/futex-internal.h:
Include libc-diag.h instead of libc-internal.h.
* elf/dl-load.c, elf/dl-reloc.c, locale/programs/locarchive.c
* nptl/nptl-init.c, string/strcspn.c, string/strspn.c
* malloc/malloc.c, sysdeps/i386/nptl/tls.h
* sysdeps/nacl/dl-map-segments.h, sysdeps/x86_64/atomic-machine.h
* sysdeps/unix/sysv/linux/spawni.c
* sysdeps/x86_64/nptl/tls.h:
Include libc-pointer-arith.h instead of libc-internal.h.
* elf/get-dynamic-info.h, sysdeps/nacl/dl-map-segments.h
* sysdeps/x86_64/atomic-machine.h:
Add multiple include guard.
_res_hconf.initialized was not suitable for use in a multi-threaded
environment due to the lack of atomics and memory barriers. Use of it was
also unnecessary because _res_hconf_init did the right thing by using
__libc_once. This patch fixes the glibc-internal uses by just calling
_res_hconf_init unconditionally, and switches to a release MO atomic store
for _res_hconf.initialized to fix the glibc side of the synchronization
problem (which will maintain backward compatibility, but cannot fix the
lack of acquire MO on any glibc-external loads).
[BZ #20477]
* resolv/res_hconf.c (do_init): Use atomic access.
* resolv/res_hconf.h: Add comments.
* nscd/aicache.c (addhstaiX): Call _res_hconf_init unconditionally.
* nss/getXXbyYY_r.c (REENTRANT_NAME): Likewise.
* sysdeps/posix/getaddrinfo.c (gaih_inet): Likewise.
