The nscd/*cache.c files contain assert()s, writeall() and sendfileall() calls
that invalidly use together &dataset->resp and total where either dataset or
dataset->head.recsize should be used instead one of the components. In the
writeall() and sendfileall() cases, it is unlikely to matter in practice, but
the assertions can fail sometimes without a proper reason.
The commit 20e498bd removes the pthread_mutex_rdlock() calls, but not the
corresponding pthread_mutex_unlock() calls. Also, the database lock is never
unlocked in one branch of the mempool_alloc() if.
I think unreproducible random assert(dh->usable) crashes in prune_cache() were
caused by this. But an easy way to make nscd threads hang with the broken
locking was.
2009-02-13 Ulrich Drepper <drepper@redhat.com>
[BZ #5381]
* nscd/nscd.h: Remove definitions and declarations for mem_in_flight.
Change mempool_alloc prototype.
* nscd/mem.c (gc): Don't handle mem_in_flight.
(mempool_alloc): Third parameter now only indicates whether this is the
first call (to allocate data) or not. If it is, get db rdlock.
Release it on error. Don't handle mem_in_flight.
* nscd/aicache.c (addhstaiX): Mark he parameter as const.
Adjust third parameter of mempool_alloc calls.
Nothing to do here in case mempool_alloc fails.
Avoid local variable shadowing parameter. No need to get db rdlock
before calling cache_add.
* nscd/cache.c (cache_add): Adjust call to mempool_alloc. There is
no mem_in_flight array anymore.
* nscd/connections.c: Remove definition and handling of mem_in_flight.
* nscd/grpcache.c (cache_addgr): Adjust third parameter of
mempool_alloc calls. Mark he parameter as const. Nothing to do here
in case mempool_alloc fails. No need to get db rdlock before calling
cache_add.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/servicescache.c (cache_addserv): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise. Remove some debugging code.
whether this is in response of a cache refill. Check alignment
of package data. Revamp waking of pruning thread.
(prune_cache): Small optimization.
* nscd/nscd.h: Adjust cache_add prototypes.
* nscd/aicache.c: Adjust cache_add calls.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicescache.c: Likewise.
* nscd/connections.c (restart): Really disable cache use before
exec attempt. If it fails, reenable cache.
(nscd_run_prune): Initialize wakeup_time. After wakeup, set wakeup
time to max to be able to notice concurrent cache additions. Unlock
prune_lock while performing gc. Afterwards compute wakeup time with
current wakeup_time value in mind.
is the first use of the record, mark it as unusable.
* nscd/aicache.c: Don't touch the dataset after cache_add returns
reporting a failure.
* nscd/grpcache.c: Likewise
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicecache.c: Likewise.
prototypes.
* include/arpa/nameser_compat.h: Define T_UNSPEC.
* nis/Versions (libnss_nis): Export _nss_nis_gethostbyname4_r.
(libnss_nisplus): Export _nss_nisplus_gethostbyname4_r.
* nis/nss_nis/nis-hosts.c (LINE_PARSER): Change to also handle
af==AF_UNSPEC.
(_nss_nis_gethostbyname4_r): New function.
* nis/nss_nisplus/nisplus-hosts.c (_nss_nisplus_parse_hostent):
Change to also handle af==AF_UNSPEC.
(get_tablename): New function. Use it to avoid duplication.
(_nss_nisplus_gethostbyname4_r): New function.
* nscd/aicache.c (addhstaiX): Use gethostbyname4_r function is
available.
* nss/Versions (libnss_files): Export _nss_files_gethostbyname4_r.
* nss/nss.h: Define struct gaih_addrtuple.
* nss/nss_files/files-hosts.c (LINE_PARSER): Change to also handle
af==AF_UNSPEC.
(_nss_files_gethostbyname4_r): New function.
* resolv/Versions (libnss_dns): Export _nss_dns_gethostbyname4_r.
* resolv/gethnmaddr.c: Adjust __libc_res_nsearch and __libc_res_nquery
calls.
* resolv/res_query.c (__libc_res_nquery): Take two additional
parameters for second answer buffer. Handle type=T_UNSPEC to mean
look up IPv4 and IPv6.
Change all callers.
* resolv/res_send.c (__libc_res_nsend): Take five aditional parameters
for an additional query and answer buffer. Pass to send_vc and
send_dg.
(send_vc): Send possibly two requests and receive two answers.
(send_dg): Likewise.
* resolv/nss_dns/dns-host.c: Adjust calls to __libc_res_nsearch and
__libc_res_nquery.
(_nss_dns_gethostbyname4_r): New function.
(gaih_getanswer_slice): Likewise.
(gaih_getanswer): Likewise.
* resolv/nss_dns/dns-canon.c (_nss_dns_getcanonname_r): Adjust
__libc_res_nquery call.
* resolv/nss_dns/dns-network.c (_nss_dns_getnetbyaddr_r): Likewise.
(_nss_dns_getnetbyname_r): Adjust __libc_res_nsearch call.
* sysdeps/posix/getaddrinfo.c: Use gethostbyname4_r function is
available.
2008-04-15 Ulrich Drepper <drepper@redhat.com>
[BZ #5209]
* sysdeps/unix/sysv/linux/times.c: New file.
[BZ #5381]
* nscd/nscd.h: Define enum in_flight, mem_in_flight, and
mem_in_flight_list variables. Add new parameter to mempool_alloc
prototype.
* nscd/mem.c (mempool_alloc): Take additional parameter. Initialize
appropriate mem_in_flight element.
(gc): Take allocations which have not yet been committed to the
database into account.
* nscd/cache.c (cache_add): Add new parameter to mempool_alloc call.
Reset mem_in_flight before returning.
* nscd/connections.c (nscd_run_worker): Initialize mem_in_flight and
cue it up in mem_in_flight_list.
* nscd/aicache.c: Adjust mempool_alloc call.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
* nscd/servicescache.c: Likewise.
* nscd/Makefile (nscd-flags): Until ld is fixed, use -fpic instead
of -fpie.
* nscd/connections.c (handle_request): Provide better error message
in case SELinux forbids the service.
* version.h (VERSION): Bump to 2.8.90.
* nscd/mem.c (MAX_STACK_USE): Remove definition here.
(gc): Initialize stack_used based on allocation in prune_cache.
* nscd/cache.c (prune_cache): Use heap for mark array if necessary.
Clear array bfore use.
* nscd/aicache.c (addhstaiX): Update statistics counter in case
memory allocate failed.
when the lookup call failed.
* nscd/nscd.h (struct database_dyn): Rename prunelock to prune_lock.
Add prune_cond and wakeup_time.
(CACHE_PRUNE_INTERNAL): Define.
Update declarations of prune_cache and setup_thread.
* nscd/connections.c (dbs): Update initializers.
(CACHE_PRUNE_INTERNAL): Moved to nscd.h.
(nscd_init): Default number of threads is now 4.
(invalidate_cache): Take lock before calling prune_cache.
(handle_request): If SELinux forbids the request, say so.
(readylist_cond): Use static initializer.
(nscd_run_prune): New function. Used only by pruning threads.
(nscd_run_worder): Renamed from nscd_run. Remove support for pruning
here.
(fd_ready): Update nscd_run reference.
(start_threads): No need to initialize readylist_cond.
Start pruning threads separately.
* nscd/nscd_setup_thread.c: Change return value type to int and always
return 0.
* sysdeps/unix/sysv/linux/nscd_setup_thread.c: Change return value type
to int and return nonzero value if we can use the TID address hack.
* nscd/cache.c (cache_add): If next wakeup time of cleanup thread for
the database is later than the new entry's timeout, update the
wakeup time and wake the cleanup thread.
(prune_cache): Return seconds the next entry in the database is still
valid. Remove locking for pruning here.
* nscd/nscd.conf: Document default number of threads.
gethstbynm3_r.
* nscd/gethstbynm2_r.c: Remove.
* nscd/gethstbynm3_r.c: New file.
* nscd/aicache.c (addhstaiX): Use __gethostbyaddr2_r instead of
__gethostbyaddr_r.
* nscd/gethstbyad_r.c: Generate __gethostbyaddr2_r function. Define
__gethostbyaddr_r compatibility wrapper.
* nscd/hstcache.c (cache_addhst): Add ttl parameter. Use it when
determining timeout of entry.
(lookup): Take new parameter and pass it to __gethostbyname3_r and
__gethostbyaddr2_r.
(addhstbyX): Pass reference to variable for TTL to lookup and
cache_addhst.
* nss/Versions [glibc] (GLIBC_PRIVATE): Export __nss_passwd_lookup2,
__nss_group_lookup2, __nss_hosts_lookup2, __nss_services_lookup2,
and __nss_next2. Remove __nss_services_lookup.
* nss/XXX-lookup.c: Name function now *_lookup2. Add new parameter.
Add compat wrapper.
* nss/getXXbyYY_r.c: Changes to call new *_lookup2 functions and
__nss_next2.
* nss/getXXent_r.c: Likewise.
* nss/getnssent_r.c: Likewise.
* nss/nsswitch.c (__nss_lookup): Add new parameter. If first function
does not exist in module, try the optional second name.
(__nss_next2): New function.
(__nss_next): Now wrapper around __nss_next2.
* nss/nsswitch.h: Adjust __nss_lookup prototype.
Declare __nss_next2.
Adjust definition of db_lookup_function type.
* nss/service-lookup.c: Define NO_COMPAT.
* include/netdb.h: Declare __gethostbyaddr2_r and __gethostbyname3_r.
* inet/ether_hton.c: Use __nss_next2 instead of __nss_next.
* inet/ether_ntoh.c: Likewise.
* sunrpc/netname.c: Likewise.
* sunrpc/publickey.c: Likewise.
* inet/getnetgrent.c: Likewise. Adjust calls to __nss_lookup.
* inet/gethstbyad_r.c (DB_LOOKUP_FCT): Change to __nss_hosts_lookup2.
* inet/gethstbynm2_r.c (DB_LOOKUP_FCT): Likewise.
* inet/gethstbynm_r.c (DB_LOOKUP_FCT): Likewise.
* inet/gethstent_r.c (DB_LOOKUP_FCT): Likewise.
* nscd/aicache.c (addhstaiX): Fix default TTL handling.
* inet/getnetgrent.c (setup): Encrypt static pointer.
* ncsd/aicache.c: Don't use sendfile for records on the stack.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/pwdcache.c: Likewise.
Patch by dmueller@suse.com.
(struct shmid_ds): Use it for shm_nattch field.
2005-11-18 Jakub Jelinek <jakub@redhat.com>
* sysdeps/unix/sysv/linux/futimesat.c (futimesat): If FILE is NULL,
set access and modification times of the file referenced by FD.
* sysdeps/generic/futimesat.c (futimesat): Don't return EINVAL if
FILE is NULL. Don't check FD if FILE is absolute path.
2005-11-19 Ulrich Drepper <drepper@redhat.com>
* nscd/nscd_gethst_r.c (nscd_gethst_r): Avoid unnecesary read call
if there are no aliases.
* sysdeps/unix/sysv/linux/Makefile (CFLAGS-connections.c,
CFLAGS-pwdcache.c, CFLAGS-grpcache.c, CFLAGS-hstcache.c,
CFLAGS-aicache.c, CFLAGS-initgrcache.c): Add -DHAVE_SENDFILE.
* sysdeps/unix/sysv/linux/kernel-features.h (__ASSUME_SENDFILE):
Define.
* nscd/pwdcache.c [HAVE_SENDFILE]: Include <sys/sendfile.h> and
<kernel-features.h>.
[HAVE_SENDFILE] (cache_addpw): Use sendfile to transmit positive
result.
* nscd/grpcache.c: Likewise.
* nscd/hstcache.c: Likewise.
* nscd/aicache.c: Likewise.
* nscd/initgrcache.c: Likewise.
* nscd/connectionc.c: Likewise.
(dbs): Initialize max_db_size fields.
(nscd_init): When mapping the database, use max_db_size as the
mapping size even if it is bigger than the file size.
* nscd/mem.c (mempool_alloc): When resizing the file make sure the
limit in max_db_size is not exceeded. Don't use mremap, just
posix_fallocate is enough (according to Linus). Use posix_fallocate
correctly.
* nscd/nscd.conf: Add max-db-size parameters.
* nscd/nscd.h (struct database_dyn): Add max_db_size field.
Define DEFAULT_MAX_DB_SIZE and DEFAULT_DATASIZE_PER_BUCKET.
Temporarily define TEMP_FAILURE_RETRY_VAL here.
* nscd/nscd_conf.c (nscd_parse_file): Parse max-db-size parameter
and add sanity checks for it.
* nscd/aicache.c (addhstaiX): Use send with MSG_NOSIGNAL not write to
send reply.
* nscd/connection.c (writeall): Likewise.
(handle_request): Likewise.
* nscd/grpcache.c (cache_addgr): Likewise.
* nscd/hstcache.c (cache_addhst): Likewise.
* nscd/initgrcache.c (addinitgroupsX): Likewise.
* nscd/nscd.c (parse_opt): Likewise.
* nscd/nscd_stat.c (send_stats): Likewise.
(receive_print_stats): Likewise.
* nscd/pwdcache.c (cache_addpw): Likewise.
* nscd/aicache.c: Prefer using gethostbyname3_r NSS callback to also
get ttl and canonical name. Use these two values.
* resolv/Versions: Export _nss_dns_gethostbyname3_r from libnss_dns.
* resolv/nss_dns/dns-host.c (getanswer_r): Take two new parameters.
If nonnull fill with TTL and pointer to canonical name respectively.
(_nss_dns_gethostbyaddr_r): Pass NULL in new parameters of getanswer_r.
(_nss_dns_gethostbyname2_r): Just wrapper around
_nss_dns_gethostbyname3_r.
(_nss_dns_gethostbyname3_r): Renamed from _nss_dns_gethostbyname2_r.
Take two new parameters which as passed to getanswer_r.
