I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
__nss_database_lookup2's extra arguments were left unused in the
nsswitch reloading patch set; this broke compat (default config
ignored) and shadow files (secondary name ignored) which relies on
these fallbacks.
This patch adds in the previous behavior by correcting the
initialization of the database list to reflect the fallbacks. This
means that the nss_database_lookup2 interface no longer needs to be
passed the fallback info, so API and callers were adjusted.
Since all callers needed to be edited anyway, the calls were changed
from __nss_database_lookup2 to the faster __nss_database_get. This
was an intended optimization which was deferred during the initial
lookup changes to avoid touching so many files.
The test case verifies that compat targets work (passwd) and that the
default configuration works (group). Tested on x86-64.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
Some internal functions need to know if a database has a nonzero
list of actions; success getting the database does not guarantee
that. Add checks for such as needed.
Skip the ":" in each nsswitch.conf line so as not to add a dummy
action libnss_:.so
See also https://bugzilla.redhat.com/show_bug.cgi?id=1906066
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
this means that *always* libnsl is only built as shared library for
backward compatibility and the NSS modules libnss_nis and libnss_nisplus
are not built at all, libnsl's headers aren't installed.
This compatibility is kept only for architectures and ABIs that have
been added in or before version 2.28.
Replacement implementations based on TIRPC, which additionally support
IPv6, are available from <https://github.com/thkukuk/>.
This change does not affect libnss_compat which does not depended
on libnsl since 2.27 and thus can be used without NIS.
libnsl code depends on Sun RPC, e.g. on --enable-obsolete-rpc (installed
libnsl headers use installed Sun RPC headers), which will be removed in
the following commit.
The function uses the internal service_user type, so it is not
really usable from the outside of glibc. Rename the function
to __nss_database_lookup2 for internal use, and change
__nss_database_lookup to always indicate failure to the caller.
__nss_next already was a compatibility symbol. The new
implementation always fails and no longer calls __nss_next2.
unscd, the alternative nscd implementation, does not use
__nss_database_lookup, so it is not affected by this change.
Continuing the fixes for linknamespace and localplt test failures with
-Os that arise from functions not being inlined in that case, this
patch fixes such failures for feof_unlocked.
The usual approach is followed of adding __feof_unlocked (inlined when
feof_unlocked is), making calls use it when required for namespace
reasons, and using libc_hidden_proto / libc_hidden_weak for the
feof_unlocked weak alias when only localplt but not namespace issues
are involved. In the case of getaddrinfo.c, use of __feof_unlocked
needs to be conditional since that code is also used in nscd (where
__feof_unlocked is not available).
Tested for x86_64 (both without -Os to make sure that case continues
to work, and with -Os to make sure all the relevant linknamespace and
localplt test failures are resolved). Because of other such failures
that remain after this patch, neither of the bugs can yet be closed.
[BZ #15105]
[BZ #19463]
* libio/feof_u.c (feof_unlocked): Rename to __feof_unlocked and
define as weak alias of __feof_unlocked. Use libc_hidden_weak.
* include/stdio.h (feof_unlocked): Use libc_hidden_proto.
(__feof_unlocked): New declaration, and inline function if
[__USE_EXTERN_INLINES].
* iconv/gconv_conf.c (read_conf_file): Call __feof_unlocked
instead of feof_unlocked.
* intl/localealias.c [_LIBC] (FEOF): Likewise.
* nss/nsswitch.c (nss_parse_file): Likewise.
* sysdeps/unix/sysv/linux/readonly-area.c (__readonly_area):
Likewise.
* time/getdate.c (__getdate_r): Likewise.
* sysdeps/posix/getaddrinfo.c [IS_IN (libc)] (feof_unlocked):
Define as macro to call __feof_unlocked.
Mark internal nss symbols with attribute_hidden to allow direct access
within libc.so and libc.a without using GOT nor PLT.
Tested on x86-64 with and without --disable-nscd.
[BZ #18822]
* grp/initgroups.c (__nss_group_database): Removed.
(__nss_initgroups_database): Likewise.
* nscd/gai.c (__nss_hosts_database): Likewise.
* nss/XXX-lookup.c (DATABASE_NAME_SYMBOL): Likewise.
* posix/tst-rfc3484-2.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484-3.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484.c (__nss_hosts_database): Likewise.
* sysdeps/posix/getaddrinfo.c (__nss_hosts_database): Likewise.
* nss/getXXent.c (INTERNAL (REENTRANT_GETNAME)): Add
attribute_hidden.
* nss/nsswitch.c (__nss_database_custom): Define only if
USE_NSCD is defined.
(__nss_configure_lookup): Use __nss_database_custom only if
USE_NSCD is defined.
* nss/nsswitch.h (__nss_database_custom): Declare only if
USE_NSCD is defined. Add attribute_hidden.
(__nss_setent): Add attribute_hidden.
(__nss_endent): Likewise.
(__nss_getent_r): Likewise.
(__nss_getent): Likewise.
(DEFINE_DATABASE): Declare __nss_##arg##_database.
https://sourceware.org/glibc/wiki/Proposals/GroupMerging
== Justification ==
It is common today for users to rely on centrally-managed user stores for
handling their user accounts. However, much software existing today does
not have an innate understanding of such accounts. Instead, they commonly
rely on membership in known groups for managing access-control (for
example the "wheel" group on Fedora and RHEL systems or the "adm" group
on Debian-derived systems). In the present incarnation of nsswitch, the
only way to have such groups managed by a remote user store such as
FreeIPA or Active Directory would be to manually remove the groups from
/etc/group on the clients so that nsswitch would then move past nss_files
and into the SSSD, nss-ldap or other remote user database.
== Solution ==
With this patch, a new action is introduced for nsswitch:
NSS_ACTION_MERGE. To take advantage of it, one will add [SUCCESS=merge]
between two database entries in the nsswitch.conf file. When a group is
located in the first of the two group entries, processing will continue
on to the next one. If the group is also found in the next entry (and the
group name and GID are an exact match), the member list of the second
entry will be added to the group object to be returned.
== Implementation ==
After each DL_LOOKUP_FN() returns, the next action is checked. If the
function returned NSS_STATUS_SUCCESS and the next action is
NSS_ACTION_MERGE, a copy of the result buffer is saved for the next pass
through the loop. If on this next pass through the loop the database
returns another instance of a group matching both the group name and GID,
the member list is added to the previous list and it is returned as a
single object. If the following database does not contain the same group,
then the original is copied back into the destination buffer.
This patch implements merge functionality only for the group database.
For other databases, there is a default implementation that will return
the EINVAL errno if a merge is requested. The merge functionality can be
implemented for other databases at a later time if such is needed. Each
database must provide a unique implementation of the deep-copy and merge
functions.
If [SUCCESS=merge] is present in nsswitch.conf for a glibc version that
does not support it, glibc will process results up until that operation,
at which time it will return results if it has found them or else will
simply return an error. In practical terms, this ends up behaving like
the remainder of the nsswitch.conf line does not exist.
== Iterators ==
This feature does not modify the iterator functionality from its current
behavior. If getgrnam() or getgrgid() is called, glibc will iterate
through all entries in the `group` line in nsswitch.conf and display the
list of members without attempting to merge them. This is consistent with
the behavior of nss_files where if two separate lines are specified for
the same group in /etc/groups, getgrnam()/getgrgid() will display both.
Clients are already expected to handle this gracefully.
== No Premature Optimizations ==
The following is a list of places that might be eligible for
optimization, but were not overengineered for this initial contribution:
* Any situation where a merge may occur will result in one malloc() of
the same size as the input buffer.
* Any situation where a merge does occur will result in a second
malloc() to hold the list of pointers to member name strings.
* The list of members is simply concatenated together and is not tested
for uniqueness (which is identical to the behavior for nss_files,
which will simply return identical values if they both exist on the
line in the file. This could potentially be optimized to reduce space
usage in the buffer, but it is both complex and computationally
expensive to do so.
== Testing ==
I performed testing by running the getent utility against my newly-built
glibc and configuring /etc/nsswitch.conf with the following entry:
group: group: files [SUCCESS=merge] sss
In /etc/group I included the line:
wheel❌10:sgallagh
I then configured my local SSSD using the id_provider=local to respond
with:
wheel:*:10:localuser,localuser2
I then ran `getent group wheel` against the newly-built glibc in
multiple situations and received the expected output as described
above:
* When SSSD was running.
* When SSSD was configured in nsswitch.conf but the daemon was not
running.
* When SSSD was configured in nsswitch.conf but nss_sss.so.2 was not
installed on the system.
* When the order of 'sss' and 'files' was reversed.
* All of the above with the [SUCCESS=merge] removed (to ensure no
regressions).
* All of the above with `getent group 10`.
* All of the above with `getent group` with and without
`enumerate=true` set in SSSD.
* All of the above with and without nscd enabled on the system.
nscd can clear caches when certain files change. The list of files
was hardcoded so far and worked for nss_files and nss_dns and those
modules which need no monitoring. nss_db, for instance, has its
own set of files to monitor. Now the NSS modules themselves can
request that certain files are monitored.
gethstbynm3_r.
* nscd/gethstbynm2_r.c: Remove.
* nscd/gethstbynm3_r.c: New file.
* nscd/aicache.c (addhstaiX): Use __gethostbyaddr2_r instead of
__gethostbyaddr_r.
* nscd/gethstbyad_r.c: Generate __gethostbyaddr2_r function. Define
__gethostbyaddr_r compatibility wrapper.
* nscd/hstcache.c (cache_addhst): Add ttl parameter. Use it when
determining timeout of entry.
(lookup): Take new parameter and pass it to __gethostbyname3_r and
__gethostbyaddr2_r.
(addhstbyX): Pass reference to variable for TTL to lookup and
cache_addhst.
* nss/Versions [glibc] (GLIBC_PRIVATE): Export __nss_passwd_lookup2,
__nss_group_lookup2, __nss_hosts_lookup2, __nss_services_lookup2,
and __nss_next2. Remove __nss_services_lookup.
* nss/XXX-lookup.c: Name function now *_lookup2. Add new parameter.
Add compat wrapper.
* nss/getXXbyYY_r.c: Changes to call new *_lookup2 functions and
__nss_next2.
* nss/getXXent_r.c: Likewise.
* nss/getnssent_r.c: Likewise.
* nss/nsswitch.c (__nss_lookup): Add new parameter. If first function
does not exist in module, try the optional second name.
(__nss_next2): New function.
(__nss_next): Now wrapper around __nss_next2.
* nss/nsswitch.h: Adjust __nss_lookup prototype.
Declare __nss_next2.
Adjust definition of db_lookup_function type.
* nss/service-lookup.c: Define NO_COMPAT.
* include/netdb.h: Declare __gethostbyaddr2_r and __gethostbyname3_r.
* inet/ether_hton.c: Use __nss_next2 instead of __nss_next.
* inet/ether_ntoh.c: Likewise.
* sunrpc/netname.c: Likewise.
* sunrpc/publickey.c: Likewise.
* inet/getnetgrent.c: Likewise. Adjust calls to __nss_lookup.
* inet/gethstbyad_r.c (DB_LOOKUP_FCT): Change to __nss_hosts_lookup2.
* inet/gethstbynm2_r.c (DB_LOOKUP_FCT): Likewise.
* inet/gethstbynm_r.c (DB_LOOKUP_FCT): Likewise.
* inet/gethstent_r.c (DB_LOOKUP_FCT): Likewise.
* nscd/aicache.c (addhstaiX): Fix default TTL handling.
* inet/getnetgrent.c (setup): Encrypt static pointer.
void **.
* nss/nsswitch.h (service_user): Use void * type for KNOWN field.
* nss/nss_files/files-hosts.c (LINE_PARSER): Cast host_addr to
char * to avoid warning.
* nis/nss_nis/nis-hosts.c (LINE_PARSER): Likewise.
* timezone/Makefile (CFLAGS-zdump.c): Add -fwrapv.
* locale/programs/ld-ctype.c (ctype_finish, set_class_defaults,
allocate_arrays): Cast second argument to charmap_find_symbol
to char * to avoid warnings.
* locale/programs/repertoire.c (repertoire_new_char): Change
from_nr, to_nr and cnt to unsigned long, adjust printf format
string.
* locale/programs/ld-collate.c (insert_value, handle_ellipsis):
Cast second argument to new_element to char * to avoid warnings.
* locale/weightwc.h (findidx): Cast &extra[-i] to const int32_t *.
* intl/gettextP.h (struct loaded_domain): Change plural to const
struct expression *.
* intl/plural-eval.c (plural_eval): Change first argument to
const struct expression *.
* intl/plural-exp.c (EXTRACT_PLURAL_EXPRESSION): Change first
argument to const struct expression **.
* intl/plural-exp.h (EXTRACT_PLURAL_EXPRESSION, plural_eval): Adjust
prototypes.
* intl/loadmsgcat (_nl_unload_domain): Cast away const
in call to __gettext_free_exp.
* posix/fnmatch.c (fnmatch): Rearrange code to avoid maybe
unitialized wstring/wpattern var warnings.
* posix/runtests.c (struct a_test): Make data field const char *.
* stdio-common/tst-sprintf2.c (main): Don't declere u, v and buf
vars if not LDBL_MANT_DIG >= 106.
* stdio-common/Makefile (CFLAGS-vfwprintf.c): Add -Wno-unitialized.
* stdio-common/vfprintf.c (vfprintf): Cast first arugment to
__find_specmb to avoid warning.
* rt/tst-mqueue1.c (do_one_test): Add casts to avoid warnings.
* debug/test-strcpy_chk.c (do_tests, do_random_tests): Add casts
to avoid warnings.
* sysdeps/ieee754/ldbl-96/s_roundl.c (huge): Add L suffix to
initializer.
* sysdeps/unix/clock_gettime.c (clock_gettime): Only define
tv var when it will be actually used.
* sunrpc/rpc_cmsg.c (xdr_callmsg): Cast IXDR_PUT_* to void
to avoid warnings.
was allocated here. [Coverity CID 219]
* posix/getconf.c (print_all): Free confstr data after printing.
[Coverity CID 218]
* sysdeps/posix/getaddrinfo.c (gaih_inet): Free canon string if
list allocation fails. [Coverity CID 215]
* nss/nsswitch.c (__nss_configure_lookup): Fix loop end condition.
[Coverity CID 213]
* argp/argp-help.c (hol_entry_cmp): Don't call canon_doc_option if
string is NULL. [Coverity CID 212]
* argp/Makefile: Add rules to build and run bug-argp1.
* argp/bug-argp1.c: New file.
* io/ftw.c (ftw_dir): Use __rawmemchr instead of strchr to find
end of string.
* stdlib/canonicalize.c (__realpath): Likewise.
* locale/programs/ld-time.c (time_finish): Don't dereference NULL
pointer. [Coverity CID 206]
* elf/dl-dst.h (DL_DST_REQUIRED): Be prepared for missing link map
in statically linked code.
* elf/dl-load.c (_dl_dst_substitute): When replacing ORIGIN in
statically built code, be prepared to have no link map.
[Coverity CID 205]
* argp/argp-help.c (fill_in_uparams): Handle STATE==NULL in
dgettext calls. [Coverity CID 204]
* argp/argp-help.c (struct uparams): Remove valid member. Change
the one user.
(uparam_names): Reduce size. Avoid relative relocations.
Moved to read-only segment.
(fill_in_uparams): Update for new layout.
* sysdeps/unix/sysv/linux/ifaddrs.c (getifaddrs): Parameter can be
assumed to always be != NULL. [Coverity CID 202]
* argp/argp-help.c (hol_entry_help): Remove some dead code
[Coverity CID 200].
* nis/nss_nis/nis-service.c (_nss_nis_getservbyport_r): Optimize
away a few more unconditional yperr2nss calls.
(_nss_nis_getservbyname_r): Likewise.
