Previously, if an entry was marked unusable for any reason, but had
not timed out yet, the assert would trigger.
One way to get into such state is if a data change is detected during
re-validation of an entry. This causes the entry to be marked as not
usable. If exits nscd soon after that, then the clock jumps
backwards, and nscd restarted, the cache re-validation run after
startup triggers the removed assert.
The change is more complicated than just the removal of the assert
because entries marked as not usable should be garbage-collected in
the second pass. To make this happen, it is necessary to update some
book-keeping data.
Reviewed-by: DJ Delorie <dj@redhat.com>
Replace atomic_increment and atomic_increment_val with atomic_fetch_add_relaxed.
One case in sem_post.c uses release semantics (see comment above it).
The others are simple counters and do not protect any shared data from
concurrent accesses.
Passes regress on AArch64.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
While nscd prunes a cache it becomes inconsistent temporarily, which is
visible to clients if that cache is shared. Bump the GC cycle counter so
that the clients notice the modification window.
Uniformly use atomic_fetch_add to modify the GC cycle counter.
atomic_compare_and_exchange_bool_rel and
catomic_compare_and_exchange_bool_rel are removed and replaced with the
new C11-like atomic_compare_exchange_weak_release. The concurrent code
in nscd/cache.c has not been reviewed yet, so this patch does not add
detailed comments.
* nscd/cache.c (cache_add): Use new C11-like atomic operation instead
of atomic_compare_and_exchange_bool_rel.
* nptl/pthread_mutex_unlock.c (__pthread_mutex_unlock_full): Likewise.
* include/atomic.h (atomic_compare_and_exchange_bool_rel,
catomic_compare_and_exchange_bool_rel): Remove.
* sysdeps/aarch64/atomic-machine.h
(atomic_compare_and_exchange_bool_rel): Likewise.
* sysdeps/alpha/atomic-machine.h
(atomic_compare_and_exchange_bool_rel): Likewise.
* sysdeps/arm/atomic-machine.h
(atomic_compare_and_exchange_bool_rel): Likewise.
* sysdeps/mips/atomic-machine.h
(atomic_compare_and_exchange_bool_rel): Likewise.
* sysdeps/tile/atomic-machine.h
(atomic_compare_and_exchange_bool_rel): Likewise.
In bug 14906 the user complains that the inotify support in nscd
is not sufficient when it comes to detecting changes in the
configurationfiles that should be watched for the various databases.
The current nscd implementation uses inotify to watch for changes in
the configuration files, but adds watches only for IN_DELETE_SELF and
IN_MODIFY. These watches are insufficient to cover even the most basic
uses by a system administrator. For example using emacs or vim to edit
a configuration file should trigger a reload but it might not if
the editors use move to atomically update the file. This atomic update
changes the inode and thus removes the notification on the file (as
inotify is based on inodes). Thus the inotify support in nscd for
configuration files is insufficient to account for the average use
cases of system administrators and users.
The inotify support is significantly enhanced and described here:
https://www.sourceware.org/ml/libc-alpha/2015-02/msg00504.html
Tested on x86_64 with and without inotify support.
nscd can clear caches when certain files change. The list of files
was hardcoded so far and worked for nss_files and nss_dns and those
modules which need no monitoring. nss_db, for instance, has its
own set of files to monitor. Now the NSS modules themselves can
request that certain files are monitored.
2009-02-13 Ulrich Drepper <drepper@redhat.com>
[BZ #5381]
* nscd/nscd.h: Remove definitions and declarations for mem_in_flight.
Change mempool_alloc prototype.
* nscd/mem.c (gc): Don't handle mem_in_flight.
(mempool_alloc): Third parameter now only indicates whether this is the
first call (to allocate data) or not. If it is, get db rdlock.
Release it on error. Don't handle mem_in_flight.
* nscd/aicache.c (addhstaiX): Mark he parameter as const.
Adjust third parameter of mempool_alloc calls.
Nothing to do here in case mempool_alloc fails.
Avoid local variable shadowing parameter. No need to get db rdlock
before calling cache_add.
* nscd/cache.c (cache_add): Adjust call to mempool_alloc. There is
no mem_in_flight array anymore.
* nscd/connections.c: Remove definition and handling of mem_in_flight.
* nscd/grpcache.c (cache_addgr): Adjust third parameter of
mempool_alloc calls. Mark he parameter as const. Nothing to do here
in case mempool_alloc fails. No need to get db rdlock before calling
cache_add.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/servicescache.c (cache_addserv): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise. Remove some debugging code.
fields.
* nscd/connections.c (inotify_fd): New variable.
(nscd_init): Try to open an inotify descriptor.
If successful, watch files for databases using inotify instead of
having prune threads stat the files.
(nscd_run_prune): Recognize clear_cache flag being set and call
prune_cache appropriately.
(main_loop_poll): Add inotify descriptor to wait set and handle the
reported changes.
(main_loop_epoll): Likewise.
* nscd/cache.c (prune_cache): Don't stat files for databases if
inotify is used.
* sysdeps/unix/sysv/linux/Makefile [subdir=nscd]
(CFLAGS-connections.c): Add -DHAVE_INOTIFY.
whether this is in response of a cache refill. Check alignment
of package data. Revamp waking of pruning thread.
(prune_cache): Small optimization.
* nscd/nscd.h: Adjust cache_add prototypes.
* nscd/aicache.c: Adjust cache_add calls.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicescache.c: Likewise.
* nscd/connections.c (restart): Really disable cache use before
exec attempt. If it fails, reenable cache.
(nscd_run_prune): Initialize wakeup_time. After wakeup, set wakeup
time to max to be able to notice concurrent cache additions. Unlock
prune_lock while performing gc. Afterwards compute wakeup time with
current wakeup_time value in mind.
2008-04-15 Ulrich Drepper <drepper@redhat.com>
[BZ #5209]
* sysdeps/unix/sysv/linux/times.c: New file.
[BZ #5381]
* nscd/nscd.h: Define enum in_flight, mem_in_flight, and
mem_in_flight_list variables. Add new parameter to mempool_alloc
prototype.
* nscd/mem.c (mempool_alloc): Take additional parameter. Initialize
appropriate mem_in_flight element.
(gc): Take allocations which have not yet been committed to the
database into account.
* nscd/cache.c (cache_add): Add new parameter to mempool_alloc call.
Reset mem_in_flight before returning.
* nscd/connections.c (nscd_run_worker): Initialize mem_in_flight and
cue it up in mem_in_flight_list.
* nscd/aicache.c: Adjust mempool_alloc call.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicescache.c: Likewise.
* nscd/Makefile (nscd-flags): Until ld is fixed, use -fpic instead
of -fpie.
* nscd/connections.c (handle_request): Provide better error message
in case SELinux forbids the service.
* version.h (VERSION): Bump to 2.8.90.
* nscd/mem.c (MAX_STACK_USE): Remove definition here.
(gc): Initialize stack_used based on allocation in prune_cache.
* nscd/cache.c (prune_cache): Use heap for mark array if necessary.
Clear array bfore use.
* nscd/aicache.c (addhstaiX): Update statistics counter in case
memory allocate failed.
when the lookup call failed.
* nscd/nscd.h (struct database_dyn): Rename prunelock to prune_lock.
Add prune_cond and wakeup_time.
(CACHE_PRUNE_INTERNAL): Define.
Update declarations of prune_cache and setup_thread.
* nscd/connections.c (dbs): Update initializers.
(CACHE_PRUNE_INTERNAL): Moved to nscd.h.
(nscd_init): Default number of threads is now 4.
(invalidate_cache): Take lock before calling prune_cache.
(handle_request): If SELinux forbids the request, say so.
(readylist_cond): Use static initializer.
(nscd_run_prune): New function. Used only by pruning threads.
(nscd_run_worder): Renamed from nscd_run. Remove support for pruning
here.
(fd_ready): Update nscd_run reference.
(start_threads): No need to initialize readylist_cond.
Start pruning threads separately.
* nscd/nscd_setup_thread.c: Change return value type to int and always
return 0.
* sysdeps/unix/sysv/linux/nscd_setup_thread.c: Change return value type
to int and return nonzero value if we can use the TID address hack.
* nscd/cache.c (cache_add): If next wakeup time of cleanup thread for
the database is later than the new entry's timeout, update the
wakeup time and wake the cleanup thread.
(prune_cache): Return seconds the next entry in the database is still
valid. Remove locking for pruning here.
* nscd/nscd.conf: Document default number of threads.
* nscd/nscd.c (parse_opt): Read response from INVALIDATE request
to make sure the database has been already invalidated.
* nscd/cache.c (prune_cache): Add fd argument. Write response to fd
after the cache has been invalidated. Use pthread_mutex_lock rather
than pthread_mutex_trylock if fd != -1.
* nscd/connections.c (invalidate_cache): Add fd argument, write
response to fd if not calling prune_cache, pass fd to prune_cache.
(handle_request): Adjust invalidate_cache caller.
(nscd_run): Pass -1 as fd to prune_cache.
* nscd/cache.c (prune_cache): Take prunelock before starting the
work. Just return in case it is already taken.
(inet6_option_alloc): Now a simple wrapper around option_alloc.
