In bug 14906 the user complains that the inotify support in nscd
is not sufficient when it comes to detecting changes in the
configurationfiles that should be watched for the various databases.
The current nscd implementation uses inotify to watch for changes in
the configuration files, but adds watches only for IN_DELETE_SELF and
IN_MODIFY. These watches are insufficient to cover even the most basic
uses by a system administrator. For example using emacs or vim to edit
a configuration file should trigger a reload but it might not if
the editors use move to atomically update the file. This atomic update
changes the inode and thus removes the notification on the file (as
inotify is based on inodes). Thus the inotify support in nscd for
configuration files is insufficient to account for the average use
cases of system administrators and users.
The inotify support is significantly enhanced and described here:
https://www.sourceware.org/ml/libc-alpha/2015-02/msg00504.html
Tested on x86_64 with and without inotify support.
The padding bytes in the statsdata struct are not initialized, due to
which valgrind throws a warning:
==11384== Memcheck, a memory error detector
==11384== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==11384== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==11384== Command: nscd -d
==11384==
Fri 25 Apr 2014 10:34:53 AM CEST - 11384: handle_request: request received (Version = 2) from PID 11396
Fri 25 Apr 2014 10:34:53 AM CEST - 11384: GETSTAT
==11384== Thread 6:
==11384== Syscall param socketcall.sendto(msg) points to uninitialised byte(s)
==11384== at 0x4E4ACDC: send (in /lib64/libpthread-2.12.so)
==11384== by 0x11AF6B: send_stats (in /usr/sbin/nscd)
==11384== by 0x112F75: nscd_run_worker (in /usr/sbin/nscd)
==11384== by 0x4E439D0: start_thread (in /lib64/libpthread-2.12.so)
==11384== by 0x599AB6C: clone (in /lib64/libc-2.12.so)
==11384== Address 0x15708395 is on thread 6's stack
Fix the warning by initializing the structure.
This patch avoids warnings for unused results of setuid and setgid in
nscd/connections.c using an ignore_value macro along the lines
suggested by Paul in
<https://sourceware.org/ml/libc-alpha/2014-11/msg00733.html>.
Tested for x86_64.
* include/libc-internal.h (ignore_value): New macro.
* nscd/connections.c (restart): Wrap calls to setuid and setgid
with ignore_value.
The current scheme to identify which module a translation unit is
built in depends on defining multiple macros IS_IN_* and also defining
NOT_IN_libc if we're building a non-libc module. In addition, there
is an IN_LIB macro that does effectively the same thing, but for
different modules (notably the systemtap probes). This macro scheme
unifies both ideas to use just one macro IN_MODULE and assign it a
value depending on the module it is being built into. If the module
is not defined, it defaults to MODULE_libc.
Patches that follow will replace uses of IS_IN_* variables with the
IS_IN() macro. libc-symbols.h has been converted already to give an
example of how such a transition will look.
Verified that there are no relevant binary changes. One source change
that will crop up repeatedly is that of nscd_stat, since it uses the
build timestamp as a constant in its logic.
* Makeconfig (in-module): Get value of libof set for the
translation unit.
(CPPFLAGS): Use $(in-module).
* Makerules: Don't suffix routine names for nonlib.
* include/libc-modules.h: New file.
* include/libc-symbols.h: Include libc-modules.h
(IS_IN): New macro to replace IS_IN_* macros.
* elf/Makefile: Set libof-* for each routine.
* elf/rtld-Rules: Likewise.
* extra-modules.mk: Likewise.
* iconv/Makefile: Likewise.
* iconvdata/Makefile: Likewise.
* locale/Makefile: Likewise.
* malloc/Makefile: Likewise.
* nss/Makefile: Likewise.
* sysdeps/gnu/Makefile: Likewise.
* sysdeps/ieee754/ldbl-opt/Makefile: Likewise.
* sysdeps/unix/sysv/linux/Makefile: Likewise.
* sysdeps/s390/s390-64/Makefile: Likewise.
* nscd/Makefile: Set libof-* for each routine. Set CFLAGS and
CPPFLAGS for nscd instead of nonlib.
qsort_r is defined in the same file as qsort, but is not an ISO C
function, so should be a weak alias for __qsort_r. The uses in
getaddrinfo should also call __qsort_r, since getaddrinfo is a POSIX
function and qsort_r isn't. This patch implements this. Because nscd
uses the getaddrinfo sources outside libc, as do the tst-rfc3484
tests, a #define of __qsort_r to qsort_r is added there alongside the
similar defines for other libc-internal symbols used in getaddrinfo.
Tested for x86_64 (testsuite, and that disassembly of installed shared
libraries is unchanged by the patch).
[BZ #17571]
* stdlib/msort.c (qsort_r): Rename to __qsort_r and define as weak
alias of __qsort_r.
(qsort): Call __qsort_r instead of qsort_r.
* include/stdlib.h (qsort_r): Do not call libc_hidden_proto.
(__qsort_r): Declare. Call libc_hidden_proto.
* sysdeps/posix/getaddrinfo.c (getaddrinfo): Call __qsort_r
instead of qsort_r.
* nscd/gai.c (__qsort_r): Define to qsort_r.
* posix/tst-rfc3484.c (__qsort_r): Likewise.
* posix/tst-rfc3484-2.c (__qsort_r): Likewise.
* posix/tst-rfc3484-3.c (__qsort_r): Likewise.
The nscd parent process returns the result of a `wait' call rather
than the exit status of the child it waits for. These two aren't
exactly the same. In my case (and probably on most machines), the exit
status is in the 2nd LSB of the result of `wait', and so:
e.g. if the nscd child process returns 1, the parent returns 1 << 8,
which Bash happily reports as 0.
The attached patch removed the unused ‘thread_info_t’ typedef and the
‘thread_info’ variable from nscd.c. The former conflicts with a GNU Mach
typedef, and the latter conflicts with a GNU Mach function declaration:
<https://lists.gnu.org/archive/html/bug-hurd/2014-06/msg00101.html>.
Tested on x86_64-linux-gnu.
This patch makes files using __ASSUME_* macros include
<kernel-features.h> explicitly, rather than relying on some other
header (such as tls.h, lowlevellock.h or pthreadP.h) to include it
implicitly. (I omitted cases where I've already posted or am testing
the patch that stops the file from needing __ASSUME_* at all.) This
accords with the general principle of making source files include the
headers for anything they use, and also helps make it safe to remove
<kernel-features.h> includes from any file that doesn't use
__ASSUME_* (some of those may be stray includes left behind after
increasing the minimum kernel version, others may never have been
needed or may have become obsolete after some other change).
Tested x86_64 that the disassembly of installed shared libraries is
unchanged by this patch.
* nptl/pthread_cond_wait.c: Include <kernel-features.h>.
* nptl/pthread_rwlock_timedrdlock.c: Likewise.
* nptl/pthread_rwlock_timedwrlock.c: Likewise.
* nptl/sysdeps/unix/sysv/linux/lowlevelrobustlock.c: Likewise.
* nscd/nscd.c: Likewise.
* sysdeps/i386/nptl/tcb-offsets.sym: Likewise.
* sysdeps/powerpc/nptl/tcb-offsets.sym: Likewise.
* sysdeps/sh/nptl/tcb-offsets.sym: Likewise.
* sysdeps/x86_64/nptl/tcb-offsets.sym: Likewise.
The netgroups nss modules in the glibc tree use NSS_STATUS_UNAVAIL
(with errno as ERANGE) when the supplied buffer does not have
sufficient space for the result. This is wrong, because the canonical
way to indicate insufficient buffer is to set the errno to ERANGE and
the status to NSS_STATUS_TRYAGAIN, as is used by all other modules.
This fixes nscd behaviour when the nss_ldap module returns
NSS_STATUS_TRYAGAIN to indicate that a netgroup entry is too long to
fit into the supplied buffer.
I noticed that some of the Depend files, used to determine the
subdirectory build order in sysd-sorted, still mentioned linuxthreads,
although it hasn't been supported for many years. This patch removes
those references. In the case of nscd, it substitutes an nptl
reference, since I believe there is a fact a thread library dependence
there; the others already mentioned nptl.
Note that I am not at all confident in the completeness of these
Depend files.
Note also that references to linuxthreads remain in a comment in
sysdeps/unix/sysv/linux/ia64/Versions, and in manual/maint.texi,
manual/signal.texi and scripts/documented.sh.
Tested x86_64 that the installed shared libraries are unchanged by the
patch (as is sysd-sorted).
* nscd/Depend (linuxthreads): Remove.
(nptl): Add.
* resolv/Depend (linuxthreads): Remove.
* rt/Depend (linuxthreads): Remove.
The datahead structure has an unused padding field that remains
uninitialized. Valgrind prints out a warning for it on querying a
netgroups entry. This is harmless, but is a potential data leak since
it would result in writing out an uninitialized byte to the cache
file. Besides, this happens only when there is a cache miss, so we're
not adding computation to any fast path.
This patch consolidates the code to initialize the header of a dataset
into a single set of functions (one for positive and another for
negative datasets) primarily to reduce repetition of code. The
secondary reason is to simplify Patch 2/2 which fixes the problem of
an uninitialized byte in the header by initializing an unused field in
the structure and hence preventing a possible data leak into the cache
file.
The SELinux team has indicated to me that glibc's SELinux checks
in nscd are not being carried out as they would expect the API
to be used today. They would like to move away from static header
defines for class and permissions and instead use dynamic checks
at runtime that provide an answer which is dependent on the runtime
status of SELinux i.e. more dynamic.
The following patch is a minimal change that moves us forward in
this direction.
It does the following:
* Stop checking for SELinux headers that define NSCD__SHMEMHOST.
Check only for the presence or absence of the library.
* Don't encode the specific SELinux permission constants into a
table at build time, and instead use the symbolic name for the
permission as expected.
* Lookup the "What do we do if we don't know this permission?"
policy and use that if we find SELinux's policy is older than
the glibc policy e.g. we make a request for a permission that
SELinux doesn't know about.
* Lastly, translate the class and permission and then make
the permission check. This is done every time we lookup
a permission, and this is the expected way to use the API.
SELinux will optimize this for us, and we expect the network
latencies to hide these extra library calls.
Tested on x86, x86-64, and via Fedora Rawhide since November 2013.
See:
https://sourceware.org/ml/libc-alpha/2014-04/msg00179.html
This patch defines _STRING_ARCH_unaligned to 0 on default bits/string.h
header to avoid undefined compiler warnings on platforms that do not
define it. It also make adjustments in code where tests checked if macro
existed or not.
Calls to stpcpy from nscd netgroups code will have overlapping source
and destination when all three values in the returned triplet are
non-NULL and in the expected (host,user,domain) order. This is seen
in valgrind as:
==3181== Source and destination overlap in stpcpy(0x19973b48, 0x19973b48)
==3181== at 0x4C2F30A: stpcpy (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==3181== by 0x12567A: addgetnetgrentX (string3.h:111)
==3181== by 0x12722D: addgetnetgrent (netgroupcache.c:665)
==3181== by 0x11114C: nscd_run_worker (connections.c:1338)
==3181== by 0x4E3C102: start_thread (pthread_create.c:309)
==3181== by 0x59B81AC: clone (clone.S:111)
==3181==
Fix this by using memmove instead of stpcpy.
nscd works correctly when the request in innetgr is a wildcard,
i.e. when one or more of host, user or domain parameters is NULL.
However, it does not work when the the triplet in the netgroup
definition has a wildcard. This is easy to reproduce for a triplet
defined as follows:
foonet (,foo,)
Here, an innetgr call that looks like this:
innetgr ("foonet", "foohost", "foo", NULL);
should succeed and so should:
innetgr ("foonet", NULL, "foo", "foodomain");
It does succeed with nscd disabled, but not with nscd enabled. This
fix adds this additional check for all three parts of the triplet so
that it gives the correct result.
[BZ #16758]
* nscd/netgroupcache.c (addinnetgrX): Succeed if triplet has
blank values.
The buffer to query netgroup entries is allocated sufficient space for
the netgroup entries and the key to be appended at the end, but it
sends in an incorrect available length to the NSS netgroup query
functions, resulting in overflow of the buffer in some special cases.
The fix here is to factor in the key length when sending the available
buffer and buffer length to the query functions.
Currently the nscd service is installed in systemd as a simple
service, which means that it is able to handle its own errors and does
not quit. Since nscd does not fit that description, i.e. it can exit
on errors like, say, failing to parse nscd.conf, it should be declared
as forking instead.
Currently, the nscd parent process parses commandline options and
configuration, forks on startup and immediately exits with a success.
If the child process encounters some error after this, it goes
undetected and any services started up after it may have to repeatedly
check to make sure that the nscd service did actually start up and is
serving requests.
To make this process more reliable, I have added a pipe between the
parent and child process, through which the child process sends a
notification to the parent informing it of its status. The parent
waits for this status and once it receives it, exits with the
corresponding exit code. So if the child service sends a success
status (0), the parent exits with a success status. Similarly for
error conditions, the child sends the non-zero status code, which the
parent passes on as the exit code.
This, along with setting the nscd service type to forking in its
systemd configuration file, allows systemd to be certain that the nscd
service is ready and is accepting connections.
The _nss_*_getnetgrent_r query populates the netgroup results in the
allocated buffer and then sets the result triplet to point to strings
in the buffer. This is a problem when the buffer is reallocated since
the pointers to the triplet strings are no longer valid. The pointers
need to be adjusted so that they now point to strings in the
reallocated buffer.
addgetnetgrentX has a buffer which is grown as per the needs of the
requested size either by using alloca or by falling back to malloc if
the size is larger than 1K. There are two problems with the alloca
bits: firstly, it doesn't really extend the buffer since it does not
use the return value of the extend_alloca macro, which is the location
of the reallocated buffer. Due to this the buffer does not actually
extend itself and hence a subsequent write may overwrite stuff on the
stack.
The second problem is more subtle - the buffer growth on the stack is
discontinuous due to block scope local variables. Combine that with
the fact that unlike realloc, extend_alloca does not copy over old
content and you have a situation where the buffer just has garbage in
the space where it should have had data.
This could have been fixed by adding code to copy over old data
whenever we call extend_alloca, but it seems unnecessarily
complicated. This code is not exactly a performance hotspot (it's
called when there is a cache miss, so factors like network lookup or
file reads will dominate over memory allocation/reallocation), so this
premature optimization is unnecessary.
Thanks Brad Hubbard <bhubbard@redhat.com> for his help with debugging
the problem.
Usage output for option --invalidate=TABLE is not helpful without
list of tables. The list is also missing from nscd(8) manual which
made it pretty difficult to know what are the tables.
nscd incorrectly returns a success even when the netgroup in question
is not found and adds a positive result in the cache. this patch
fixes this behaviour by adding a negative lookup entry to cache and
returning an error when the netgroup is not found.
Currently, when a user looks up a netgroup that does not have any
members, nscd goes into an infinite loop trying to find members in the
group. This is because it does not handle cases when getnetgrent
returns an NSS_STATUS_NOTFOUND (which is what it does on empty group).
Fixed to handle this in the same way as NSS_STATUS_RETURN, similar to
what getgrent does by itself.
Autoconf has been deprecating configure.in for quite a long time.
Rename all our configure.in and preconfigure.in files to .ac.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
These prototypes are duplicated in many places. Add a dedicated
header for holding prototypes for program-specific functions to
avoid that.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
We refactor the inotify file checking code and the
database clearing code out of the main_loop_poll and
mail_loop_epoll functions. We document some the related
functions with appropriate comments.
* nscd/grpcache.c (cache_addgr): Rename alloca_used to
dataset_temporary. Track alloca usage into alloca_used.
If dataset is large allocate and release it via malloc/free.
Commit 61653dfb81 added support for
compilers predefining _FORTIFY_SOURCE by adding -U_FORTIFY_SOURCE to
CPPFLAGS for these compilers, but that change doesn't work quite well in
case of nscd: its Makefile sets _FORTIFY_SOURCE using CFLAGS instead of
CPPFLAGS and, thanks to compilation rules defined in Makerules, CPPFLAGS
are passed to compiler after CFLAGS, resulting to a build with
_FORTIFY_SOURCE turned off.
This change implements a more safe method of passing preprocessor and
compiler flags so that no nscd modules could be accidentally forgotten.
2012-05-09 Andreas Jaeger <aj@suse.de>
* nscd/nscd.c (run_modes): Make named enum, reorder so that
default is first entry.
(run_mode): Set type.
(main): Remove informal message about syslog.
(options): Fix typo.
For a better integration of nscd with systemd, we should run nscd not as
daemon but in the foreground. A new --foreground option should be added.
2012-05-09 Alexandre Oliva <aoliva@redhat.com>
Andreas Jaeger <aj@suse.de>
* nscd/nscd.c (go_background): Replaced with...
(run_mode): ... this.
(RUN_FOREGROUND, RUN_DAEMONIZE, RUN_DEBUG): Add.
(options): Add -F --foreground.
(main): Implement it.
(parse_opt): Parse it.
Whenever getaddrinfo needed network interface information it used the
netlink interface to read the information every single time. The
problem is that this information can change at any time.
The patch implements monitoring of the network interfaces through
nscd. If no change is detected the previously read information can
be reused (which is the norm). This timestamp information is also
made available to other processes using the shared memory segment
between nscd and those processes.
nscd can clear caches when certain files change. The list of files
was hardcoded so far and worked for nss_files and nss_dns and those
modules which need no monitoring. nss_db, for instance, has its
own set of files to monitor. Now the NSS modules themselves can
request that certain files are monitored.
When readding entries to the group and services cache and the lookup
is unsuccesful, we tried to write the notfound record. Just don't
do it in this case.
The nscd/*cache.c files contain assert()s, writeall() and sendfileall() calls
that invalidly use together &dataset->resp and total where either dataset or
dataset->head.recsize should be used instead one of the components. In the
writeall() and sendfileall() cases, it is unlikely to matter in practice, but
the assertions can fail sometimes without a proper reason.
The commit 20e498bd removes the pthread_mutex_rdlock() calls, but not the
corresponding pthread_mutex_unlock() calls. Also, the database lock is never
unlocked in one branch of the mempool_alloc() if.
I think unreproducible random assert(dh->usable) crashes in prune_cache() were
caused by this. But an easy way to make nscd threads hang with the broken
locking was.
There are two issues with the forced loop exit in the nscd lookup:
1. the estimate of the entry size isn't pessimistic enough for all
databases, resulting potentially is too early exits
2. the combination of 64-bit process and 32-bit nscd would lead to
rejecting valid records in the database.
The nscd database mapped in processes can change at any time. We
have to be more vigilant when it comes to using that memory. Test
the data entries are valid in their entire size, don't read data
again from memory once we verified it, and make sure the trailing
pointer is not going off the deep end.
Because we are not shutting down the other threads first another
thread might work on a query before the process shuts down. In this
case the now uninitialized libselinux and libaudit might be used.
Just don't free the resources. It's not necessary anyway because
the process is about to terminate.
from definition.
* sysdeps/x86_64/dl-machine.h (elf_machine_rela): Don't define
label if it is not used.
* elf/dl-profile.c (_dl_start_profile): Define real-type variant
of gmon_hist_hdr and gmon_hdr structures and use them.
* elf/dl-load.c (open_verify): Add temporary variable to avoid
warning.
* nscd/nscd_helper.c (get_mapping): Avoid casts to avoid warnings.
* sunrpc/clnt_raw.c (clntraw_private_s): Use union in definition
to avoid cast.
* inet/rexec.c (rexec_af): Make sa2 a union to avoid warnings.
* inet/rcmd.c (rcmd_af): Make from a union of the various needed types
to avoid warnings.
(iruserok_af): Use ss_family instead of casts.
* gmon/gmon.c (write_hist): Define real-type variant of
gmon_hist_hdr structure and use it.
(write_gmon): Likewise for gmon_hdr.
* sysdeps/unix/sysv/linux/readv.c: Avoid declaration of replacement
function if we are not going to define it.
* sysdeps/unix/sysv/linux/writev.c: Likewise.
* inet/inet6_option.c (optin_alloc): Add temporary variable to
avoid warning.
* libio/strfile.h (struct _IO_streambuf): Use correct type and
name of VTable element.
* libio/iovsprintf.c: Avoid casts to avoid warnings.
* libio/iovsscanf.c: Likewise.
* libio/vasprintf.c: Likewise.
* libio/vsnprintf.c: Likewise.
* stdio-common/isoc99_vsscanf.c: Likewise.
* stdlib/strfmon_l.c: Likewise.
* debug/vasprintf_chk.c: Likewise.
* debug/vsnprintf_chk.c: Likewise.
* debug/vsprintf_chk.c: Likewise.
2009-02-13 Ulrich Drepper <drepper@redhat.com>
[BZ #5381]
* nscd/nscd.h: Remove definitions and declarations for mem_in_flight.
Change mempool_alloc prototype.
* nscd/mem.c (gc): Don't handle mem_in_flight.
(mempool_alloc): Third parameter now only indicates whether this is the
first call (to allocate data) or not. If it is, get db rdlock.
Release it on error. Don't handle mem_in_flight.
* nscd/aicache.c (addhstaiX): Mark he parameter as const.
Adjust third parameter of mempool_alloc calls.
Nothing to do here in case mempool_alloc fails.
Avoid local variable shadowing parameter. No need to get db rdlock
before calling cache_add.
* nscd/cache.c (cache_add): Adjust call to mempool_alloc. There is
no mem_in_flight array anymore.
* nscd/connections.c: Remove definition and handling of mem_in_flight.
* nscd/grpcache.c (cache_addgr): Adjust third parameter of
mempool_alloc calls. Mark he parameter as const. Nothing to do here
in case mempool_alloc fails. No need to get db rdlock before calling
cache_add.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/servicescache.c (cache_addserv): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise. Remove some debugging code.
* elf/ldd.bash.in: Likewise.
* elf/sprof.c: Likewise.
* locale/programs/locale.c: Likewise.
* malloc/memusage.sh: Likewise.
* nss/getent.c: Likewise.
2009-02-06 Joseph Myers <joseph@codesourcery.com>
* debug/pcprofiledump.c (print_version,
argp_program_version_hook): New function.
* elf/ldconfig.c (more_help): New function.
(argp): Use it.
* elf/sln.c (usage): New function.
(main): Support --help and --version.
* malloc/memusagestat.c (print_version): New function.
(argp_program_version_hook): New variable.
* nscd/nscd.c (more_help): New function.
(argp): Use it.
* posix/getconf.c (main): Send --version output to stdout.
Support --help.
* sunrpc/rpc_main.c (usage, options_usage): Take STREAM and STATUS
arguments. All callers changed.
(print_version): New function.
(parseargs): Support --help and --version.
* sunrpc/rpcinfo.c (usage): Take STREAM argument. All callers
changed.
(print_version): New function.
(main): Use getopt_long. Support --help and --version.
* sysdeps/unix/sysv/linux/lddlibc4.c (main): Support --help and
--version.
2009-02-06 Ulrich Drepper <drepper@redhat.com>
* nscd/mem.c (gc): Use alloca_count to get the real stack usage.
* include/alloca.h (alloca_account): Define.
* sysdeps/x86_64/stackinfo.h (stackinfo_get_sp): Define.
(stackinfo_sub_sp): Define.
2009-01-28 Ulrich Drepper <drepper@redhat.com>
[BZ #9741]
* nscd/mem.c (gc): Fix assignment of he_data in case malloc is used.
Reported by Jun'ichi Nomura <j-nomura@ce.jp.nec.com>.
2008-12-03 Petr Baudis <pasky@suse.cz>
[BZ #7067]
* nscd/connections.c (invalidate_cache): Use prune_run_lock
instead of prune_lock.
(nscd_run_prune): Before calling prune_cache, take prune_run_lock.
* nscd/nscd.h (database_dyn): Add prune_run_cache.
