In bug 14906 the user complains that the inotify support in nscd
is not sufficient when it comes to detecting changes in the
configurationfiles that should be watched for the various databases.
The current nscd implementation uses inotify to watch for changes in
the configuration files, but adds watches only for IN_DELETE_SELF and
IN_MODIFY. These watches are insufficient to cover even the most basic
uses by a system administrator. For example using emacs or vim to edit
a configuration file should trigger a reload but it might not if
the editors use move to atomically update the file. This atomic update
changes the inode and thus removes the notification on the file (as
inotify is based on inodes). Thus the inotify support in nscd for
configuration files is insufficient to account for the average use
cases of system administrators and users.
The inotify support is significantly enhanced and described here:
https://www.sourceware.org/ml/libc-alpha/2015-02/msg00504.html
Tested on x86_64 with and without inotify support.
Currently, the nscd parent process parses commandline options and
configuration, forks on startup and immediately exits with a success.
If the child process encounters some error after this, it goes
undetected and any services started up after it may have to repeatedly
check to make sure that the nscd service did actually start up and is
serving requests.
To make this process more reliable, I have added a pipe between the
parent and child process, through which the child process sends a
notification to the parent informing it of its status. The parent
waits for this status and once it receives it, exits with the
corresponding exit code. So if the child service sends a success
status (0), the parent exits with a success status. Similarly for
error conditions, the child sends the non-zero status code, which the
parent passes on as the exit code.
This, along with setting the nscd service type to forking in its
systemd configuration file, allows systemd to be certain that the nscd
service is ready and is accepting connections.
These prototypes are duplicated in many places. Add a dedicated
header for holding prototypes for program-specific functions to
avoid that.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
nscd can clear caches when certain files change. The list of files
was hardcoded so far and worked for nss_files and nss_dns and those
modules which need no monitoring. nss_db, for instance, has its
own set of files to monitor. Now the NSS modules themselves can
request that certain files are monitored.
2009-02-13 Ulrich Drepper <drepper@redhat.com>
[BZ #5381]
* nscd/nscd.h: Remove definitions and declarations for mem_in_flight.
Change mempool_alloc prototype.
* nscd/mem.c (gc): Don't handle mem_in_flight.
(mempool_alloc): Third parameter now only indicates whether this is the
first call (to allocate data) or not. If it is, get db rdlock.
Release it on error. Don't handle mem_in_flight.
* nscd/aicache.c (addhstaiX): Mark he parameter as const.
Adjust third parameter of mempool_alloc calls.
Nothing to do here in case mempool_alloc fails.
Avoid local variable shadowing parameter. No need to get db rdlock
before calling cache_add.
* nscd/cache.c (cache_add): Adjust call to mempool_alloc. There is
no mem_in_flight array anymore.
* nscd/connections.c: Remove definition and handling of mem_in_flight.
* nscd/grpcache.c (cache_addgr): Adjust third parameter of
mempool_alloc calls. Mark he parameter as const. Nothing to do here
in case mempool_alloc fails. No need to get db rdlock before calling
cache_add.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/servicescache.c (cache_addserv): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise. Remove some debugging code.
2008-12-03 Petr Baudis <pasky@suse.cz>
[BZ #7067]
* nscd/connections.c (invalidate_cache): Use prune_run_lock
instead of prune_lock.
(nscd_run_prune): Before calling prune_cache, take prune_run_lock.
* nscd/nscd.h (database_dyn): Add prune_run_cache.
fields.
* nscd/connections.c (inotify_fd): New variable.
(nscd_init): Try to open an inotify descriptor.
If successful, watch files for databases using inotify instead of
having prune threads stat the files.
(nscd_run_prune): Recognize clear_cache flag being set and call
prune_cache appropriately.
(main_loop_poll): Add inotify descriptor to wait set and handle the
reported changes.
(main_loop_epoll): Likewise.
* nscd/cache.c (prune_cache): Don't stat files for databases if
inotify is used.
* sysdeps/unix/sysv/linux/Makefile [subdir=nscd]
(CFLAGS-connections.c): Add -DHAVE_INOTIFY.
whether this is in response of a cache refill. Check alignment
of package data. Revamp waking of pruning thread.
(prune_cache): Small optimization.
* nscd/nscd.h: Adjust cache_add prototypes.
* nscd/aicache.c: Adjust cache_add calls.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicescache.c: Likewise.
* nscd/connections.c (restart): Really disable cache use before
exec attempt. If it fails, reenable cache.
(nscd_run_prune): Initialize wakeup_time. After wakeup, set wakeup
time to max to be able to notice concurrent cache additions. Unlock
prune_lock while performing gc. Afterwards compute wakeup time with
current wakeup_time value in mind.
2008-04-15 Ulrich Drepper <drepper@redhat.com>
[BZ #5209]
* sysdeps/unix/sysv/linux/times.c: New file.
[BZ #5381]
* nscd/nscd.h: Define enum in_flight, mem_in_flight, and
mem_in_flight_list variables. Add new parameter to mempool_alloc
prototype.
* nscd/mem.c (mempool_alloc): Take additional parameter. Initialize
appropriate mem_in_flight element.
(gc): Take allocations which have not yet been committed to the
database into account.
* nscd/cache.c (cache_add): Add new parameter to mempool_alloc call.
Reset mem_in_flight before returning.
* nscd/connections.c (nscd_run_worker): Initialize mem_in_flight and
cue it up in mem_in_flight_list.
* nscd/aicache.c: Adjust mempool_alloc call.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicescache.c: Likewise.
* nscd/Makefile (nscd-flags): Until ld is fixed, use -fpic instead
of -fpie.
* nscd/connections.c (handle_request): Provide better error message
in case SELinux forbids the service.
* version.h (VERSION): Bump to 2.8.90.
* nscd/connections.c (dbs): Add initializers for .suggested_module.
(verify_persistent_db): Remove one unnecessary test and add a new one
for bad configuration.
(nscd_init): Improve error reported when persistent database cannot
be reused.
* nscd/nscd.h (DEFAULT_SUGGESTED_MODULE): Define.
* nscd/nscd_conf.c (nscd_parse_file): Provide default values for
.suggested_module and .max_db_size and case config file says the
values are zero.
* nscd/nscd_helper.c (get_mapping): Fail if hash table module is zero.
* nscd/mem.c (MAX_STACK_USE): Remove definition here.
(gc): Initialize stack_used based on allocation in prune_cache.
* nscd/cache.c (prune_cache): Use heap for mark array if necessary.
Clear array bfore use.
* nscd/aicache.c (addhstaiX): Update statistics counter in case
memory allocate failed.
* nscd/nscd.h (NSCD_THREAD_STACKSIZE): Define.
* nscd/connections.c (start_threads): Use NSCD_THREAD_STACKSIZE.
* nscd/mem.c (gc): Don't allocate arrays on stack if it can
overflow it.
* sysdeps/unix/sysv/linux/nscd_setup_thread.c (setup_thread):
Return zero in case thread library is not NPTL.
when the lookup call failed.
* nscd/nscd.h (struct database_dyn): Rename prunelock to prune_lock.
Add prune_cond and wakeup_time.
(CACHE_PRUNE_INTERNAL): Define.
Update declarations of prune_cache and setup_thread.
* nscd/connections.c (dbs): Update initializers.
(CACHE_PRUNE_INTERNAL): Moved to nscd.h.
(nscd_init): Default number of threads is now 4.
(invalidate_cache): Take lock before calling prune_cache.
(handle_request): If SELinux forbids the request, say so.
(readylist_cond): Use static initializer.
(nscd_run_prune): New function. Used only by pruning threads.
(nscd_run_worder): Renamed from nscd_run. Remove support for pruning
here.
(fd_ready): Update nscd_run reference.
(start_threads): No need to initialize readylist_cond.
Start pruning threads separately.
* nscd/nscd_setup_thread.c: Change return value type to int and always
return 0.
* sysdeps/unix/sysv/linux/nscd_setup_thread.c: Change return value type
to int and return nonzero value if we can use the TID address hack.
* nscd/cache.c (cache_add): If next wakeup time of cleanup thread for
the database is later than the new entry's timeout, update the
wakeup time and wake the cleanup thread.
(prune_cache): Return seconds the next entry in the database is still
valid. Remove locking for pruning here.
* nscd/nscd.conf: Document default number of threads.
* nscd/nscd.c (parse_opt): Read response from INVALIDATE request
to make sure the database has been already invalidated.
* nscd/cache.c (prune_cache): Add fd argument. Write response to fd
after the cache has been invalidated. Use pthread_mutex_lock rather
than pthread_mutex_trylock if fd != -1.
* nscd/connections.c (invalidate_cache): Add fd argument, write
response to fd if not calling prune_cache, pass fd to prune_cache.
(handle_request): Adjust invalidate_cache caller.
(nscd_run): Pass -1 as fd to prune_cache.
* nscd/cache.c (prune_cache): Take prunelock before starting the
work. Just return in case it is already taken.
(inet6_option_alloc): Now a simple wrapper around option_alloc.
* nscd/nscd_conf.c (nscd_parse_file): Parse auto-propagate lines.
* nscd/nscd.conf: Add auto-propagate lines.
* nscd/connections.c (dbs): Initialize .propagate fields.
* nscd/grpcache.c (cache_addgr): Do not add ID entry for name lookups
and vice versa if propagation is disabled for the database.
* nscd/pwdcache.c (cache_addpw): Likewise.
(dbs): Initialize max_db_size fields.
(nscd_init): When mapping the database, use max_db_size as the
mapping size even if it is bigger than the file size.
* nscd/mem.c (mempool_alloc): When resizing the file make sure the
limit in max_db_size is not exceeded. Don't use mremap, just
posix_fallocate is enough (according to Linus). Use posix_fallocate
correctly.
* nscd/nscd.conf: Add max-db-size parameters.
* nscd/nscd.h (struct database_dyn): Add max_db_size field.
Define DEFAULT_MAX_DB_SIZE and DEFAULT_DATASIZE_PER_BUCKET.
Temporarily define TEMP_FAILURE_RETRY_VAL here.
* nscd/nscd_conf.c (nscd_parse_file): Parse max-db-size parameter
and add sanity checks for it.
* nscd/aicache.c (addhstaiX): Use send with MSG_NOSIGNAL not write to
send reply.
* nscd/connection.c (writeall): Likewise.
(handle_request): Likewise.
* nscd/grpcache.c (cache_addgr): Likewise.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/nscd.c (parse_opt): Likewise.
* nscd/nscd_stat.c (send_stats): Likewise.
(receive_print_stats): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise.
2004-10-15 Ulrich Drepper <drepper@redhat.com>
* nscd/nscd.h (_PATH_NSCD_PASSWD_DB): Move to /var/db.
(_PATH_NSCD_GROUP_DB): Likewise.
(_PATH_NSCD_HOSTS_DB): Likewise.
(_PATH_NSCD_XYZ_DB_TMP): New #define, point to /var/run.
* nscd/connections.c (nscd_init): Non-persistent database files
are created with the _PATH_NSCD_XYZ_DB_TMP path.
* nscd/nscd.init: Create /var/db/nscd if necessary.
Dynamically create new threads if necessary.
* nscd/connections.c (fd_ready): If no thread available for processing
the request, create a new one unless the limit is reached.
(start_threads): Check errors from pthread_create.
* nscd/nscd.h: Declare max_nthreads.
* nscd/nscd_conf.c: Parse max-nthreads entry.
* nscd/nscd.conf: Add max-threads entry.
* nscd/nscd_stat.c: Print current and maximum number of threads.
Implement paranoia mode.
* nscd/connections.c (nscd_init): Mark database and socket descriptors
as close on exec.
(restart): New function.
(restart_p): New function.
(nscd_run): Add missing descrement of nready in case readylist is
empty.
(main_loop_poll): Call restart_p and restart.
(main_loop_epoll): Likewise.
(begin_drop_privileges): Save original UID and GID.
* nscd/nscd.c: Define new variables paranoia, restart_time,
restart_interval, oldcwd, old_gid, old_uid.
(main): Disable paranoia mode if we are not forking.
(check_pid): When re-execing, the PID file contains the same PID as
the current process. Do not fail in this case.
* nscd/nscd.conf: Add paranoia and restart-interval entries.
* nscd/nscd.h: Define RESTART_INTERVAL. Declare new variables.
* nscd/nscd_conf.c: Parse paranoia and restart-internal configurations.
* nscd/nscd_stat.c: Print paranoia and restart-internal values.
* nscd/connections.c: Implement r/o sharing of nscd's cache with client
processes via shared memory.
* nscd/nscd-client.h: Likewise.
* nscd/nscd.h: Likewise.
* nscd/nscd_conf.c: Likewise.
* nscd/nscd_getgr_r.c: Likewise.
* nscd/nscd_getpw_r.c: Likewise.
* nscd/nscd_gethst_r.c: Likewise.
* nscd/nscd.conf: Add new config parameters.
* nscd/Makefile (aux): Add nscd_helper.
* nscd/nscd_helper.c: New file.
* nscd/mem.c (gc): Indicate beginning and end of the gc cycle.
* nscd/hstcache.c: Simplify a lot. We cache only the request itself,
no derived information.
* connections.c (nscd_init): Fix bug in testing size of the persistent.
* nis/Makefile (aux): Add nis_hash.
* nis/nis_hash.c: New file. Split out from nis_util.c.
* nis/nis_util.c: Move __nis_hash code in separate file.
* csu/tst-atomic.c: Improve atomic_increment_val test which would
not have found a ppc bug.
2004-08-26 Ulrich Drepper <drepper@redhat.com>
* nscd/cache.c: Major rewrite. The data is now optionally kept in
a mmaped memory region which is automatically mirrored on disk.
This implements persistent data storage. The Memory handled
needed to be completely revamped, it now uses a garbage collection
mechanism instead of malloc.
* nscd/connections.c: Likewise.
* nscd/nscd.c: Likewise.
* nscd/nscd.h: Likewise.
* nscd/nscd_conf.c: Likewise.
* nscd/nscd_stat.c: Likewise.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c:: Likewise.
* nscd/pwdcache.c:: Likewise.
* nscd/Makefile: Add rules to build mem.c.
* nscd/mem.c: New file.
* nscd/nscd.conf: Describe new configuration options.
* nscd/connections.c (client_queued): New variable.
(nscd_run): Revamp the loop. Don't call poll except for cleanup
threads. Keep track of the number of delays caused because of busy
worker threads.
* nscd/nscd.h: Declare client_queued.
* nscd/nscd_stat.c: Transmit and print client_queued information.
* nscd/cache.c (cache_search): Keep track of how many chain links
we searched and update table statistics.
(cache_add): Keep track of how many values are in the table.
(prune_cache): Likewise. Keep track of locking success.
Print messages about removed entries in separate pass.
* nscd/connections.c (handle_request): Don't print debug message here.
The caller will do it. Keep track of locking success.
(nscd_run): Print debug message. Also print PID of the client process.
* nscd/nscd.c (start_time): New variable.
(main): Remember start time.
* nscd/nscd.h: Declare start_time.
(struct database): Add more members for new statistics.
* nscd/nscd_stat.c: Add support for sending, receiving, and printing
of new statistics.
