They are both used by __libc_freeres to free all library malloc
allocated resources to help tooling like mtrace or valgrind with
memory leak tracking.
The current scheme uses assembly markers and linker script entries
to consolidate the free routine function pointers in the RELRO segment
and to be freed buffers in BSS.
This patch changes it to use specific free functions for
libc_freeres_ptrs buffers and call the function pointer array directly
with call_function_static_weak.
It allows the removal of both the internal macros and the linker
script sections.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
C2x adds binary integer constants starting with 0b or 0B, and supports
those constants in strtol-family functions when the base passed is 0
or 2. Implement that strtol support for glibc.
As discussed at
<https://sourceware.org/pipermail/libc-alpha/2020-December/120414.html>,
this is incompatible with previous C standard versions, in that such
an input string starting with 0b or 0B was previously required to be
parsed as 0 (with the rest of the string unprocessed). Thus, as
proposed there, this patch adds 20 new __isoc23_* functions with
appropriate header redirection support. This patch does *not* do
anything about scanf %i (which will need 12 new functions per long
double variant, so 12, 24 or 36 depending on the glibc configuration),
instead leaving that for a future patch. The function names would
remain as __isoc23_* even if C2x ends up published in 2024 rather than
2023.
Making this change leads to the question of what should happen to
internal uses of these functions in glibc and its tests. The header
redirection (which applies for _GNU_SOURCE or any other feature test
macros enabling C2x features) has the effect of redirecting internal
uses but without those uses then ending up at a hidden alias (see the
comment in include/stdio.h about interaction with libc_hidden_proto).
It seems desirable for the default for internal uses to be the same
versions used by normal code using _GNU_SOURCE, so rather than doing
anything to disable that redirection, similar macro definitions to
those in include/stdio.h are added to the include/ headers for the new
functions.
Given that the default for uses in glibc is for the redirections to
apply, the next question is whether the C2x semantics are correct for
all those uses. Uses with the base fixed to 10, 16 or any other value
other than 0 or 2 can be ignored. I think this leaves the following
internal uses to consider (an important consideration for review of
this patch will be both whether this list is complete and whether my
conclusions on all entries in it are correct):
benchtests/bench-malloc-simple.c
benchtests/bench-string.h
elf/sotruss-lib.c
math/libm-test-support.c
nptl/perf.c
nscd/nscd_conf.c
nss/nss_files/files-parse.c
posix/tst-fnmatch.c
posix/wordexp.c
resolv/inet_addr.c
rt/tst-mqueue7.c
soft-fp/testit.c
stdlib/fmtmsg.c
support/support_test_main.c
support/test-container.c
sysdeps/pthread/tst-mutex10.c
I think all of these places are OK with the new semantics, except for
resolv/inet_addr.c, where the POSIX semantics of inet_addr do not
allow for binary constants; thus, I changed that file (to use
__strtoul_internal, whose semantics are unchanged) and added a test
for this case. In the case of posix/wordexp.c I think accepting
binary constants is OK since POSIX explicitly allows additional forms
of shell arithmetic expressions, and in stdlib/fmtmsg.c SEV_LEVEL is
not in POSIX so again I think accepting binary constants is OK.
Functions such as __strtol_internal, which are only exported for
compatibility with old binaries from when those were used in inline
functions in headers, have unchanged semantics; the __*_l_internal
versions (purely internal to libc and not exported) have a new
argument to specify whether to accept binary constants.
As well as for the standard functions, the header redirection also
applies to the *_l versions (GNU extensions), and to legacy functions
such as strtoq, to avoid confusing inconsistency (the *q functions
redirect to __isoc23_*ll rather than needing their own __isoc23_*
entry points). For the functions that are only declared with
_GNU_SOURCE, this means the old versions are no longer available for
normal user programs at all. An internal __GLIBC_USE_C2X_STRTOL macro
is used to control the redirections in the headers, and cases in glibc
that wish to avoid the redirections - the function implementations
themselves and the tests of the old versions of the GNU functions -
then undefine and redefine that macro to allow the old versions to be
accessed. (There would of course be greater complexity should we wish
to make any of the old versions into compat symbols / avoid them being
defined at all for new glibc ABIs.)
strtol_l.c has some similarity to strtol.c in gnulib, but has already
diverged some way (and isn't listed at all at
https://sourceware.org/glibc/wiki/SharedSourceFiles unlike strtoll.c
and strtoul.c); I haven't made any attempts at gnulib compatibility in
the changes to that file.
I note incidentally that inttypes.h and wchar.h are missing the
__nonnull present on declarations of this family of functions in
stdlib.h; I didn't make any changes in that regard for the new
declarations added.
Almost all uses of rawmemchr find the end of a string. Since most targets use
a generic implementation, replacing it with strchr is better since that is
optimized by compilers into strlen (s) + s. Also fix the generic rawmemchr
implementation to use a cast to unsigned char in the if statement.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Linux 6.1 adds a define IPPROTO_L2TP to its include/uapi/linux/in.h
(not strictly a new constant, since it's moved from
include/uapi/linux/l2tp.h). Add this constant to glibc's
netinet/in.h.
Tested for x86_64.
This patch is split out of
<https://sourceware.org/pipermail/libc-alpha/2022-December/144122.html>.
atoi has undefined behavior on out-of-range input, which makes it
problematic to use anywhere in glibc that might be processing input
out-of-range for atoi but not specified to produce undefined behavior
for the function calling atoi. Change some uses of atoi to call
strtol instead; this avoids the undefined behavior, though there is no
guarantee that the overflow handling of strtol is really right in
those places either. This also serves to avoid localplt test failures
given an installed header redirection for strtol (which means that the
call from the inline atoi implementation doesn't end up at a hidden
alias from libc_hidden_proto).
Certainly, the use of atoi is questionable in argp-help.c (shared with
gnulib, so shouldn't depend on glibc implementation details, and
processing user-provided input), and maybe also in argp-parse.c (I'm
not sure what that code in argp-parse.c is meant to be used for). I
also changed inet/rexec.c and resolv/res_init.c similarly to use
strtol to avoid such localplt failures, although given those files (in
those versions) are only used in glibc it's not problematic for them
to rely on the specific behavior of glibc's atoi on out-of-range input
(in the absence of compiler optimizations based on the undefined
behavior) in the same way it's problematic for gnulib code to do so.
There may be other uses of atoi (or atol or atoll), in any of glibc's
installed code, for which it would also be appropriate to avoid the
undefined behavior on out-of-range input; this patch only fixes the
specific cases needed to avoid localplt failures.
Tested for x86_64.
clang emits an warning when a double alias redirection is used, to warn
the the original symbol will be used even when weak definition is
overridden. However, this is a common pattern for weak_alias, where
multiple alias are set to same symbol.
Reviewed-by: Fangrui Song <maskray@google.com>
In the future, this will result in a compilation failure if the
macros are unexpectedly undefined (due to header inclusion ordering
or header inclusion missing altogether).
Assembler sources are more difficult to convert. In many cases,
they are hand-optimized for the mangling and no-mangling variants,
which is why they are not converted.
sysdeps/s390/s390-32/__longjmp.c and sysdeps/s390/s390-64/__longjmp.c
are special: These are C sources, but most of the implementation is
in assembler, so the PTR_DEMANGLE macro has to be undefined in some
cases, to match the assembler style.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This allows us to define a generic no-op version of PTR_MANGLE and
PTR_DEMANGLE. In the future, we can use PTR_MANGLE and PTR_DEMANGLE
unconditionally in C sources, avoiding an unintended loss of hardening
due to missing include files or unlucky header inclusion ordering.
In i386 and x86_64, we can avoid a <tls.h> dependency in the C
code by using the computed constant from <tcb-offsets.h>. <sysdep.h>
no longer includes these definitions, so there is no cyclic dependency
anymore when computing the <tcb-offsets.h> constants.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
I.e. from sysdeps/unix/sysv/linux/bits/in.h to netinet/in.h
It is following both the BSD and Linux definitions.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
The 'not_first' is accessed on nrl_domainname() in a non atomically
way, although it is only updated after the lock is taken.
This patch fix the double-checked locking by using acquire-release
atomic operation instead of plain load and by moving the 'not_first'
store only after 'domain' is actually set.
Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Linux 5.13 adds an INADDR_DUMMY definition; add a corresponding
definition to glibc's netinet/in.h. (This isn't strictly a new kernel
interface, rather a value defined in RFC 7600.)
Tested for x86_64.
Debugging interfaces: p_*, fp_*, and sym_* could conceivably be
used to produce debug out, but these functions have not been
updated to parse more resource records, so they are not very useful
today. Likewise for ns_sprintrr and ns_sprintrrf. ns_format_ttl and
ns_parse_ttl are related to these.
Internal implementation details: res_isourserver is probably only
useful in the implementation of a stub resolver, and so is
res_nameinquery.
Unclear semantics and bad performance: ns_samedomain, ns_subdomain,
ns_makecanon, ns_samename do textual converions & copies instead of
checking equivalence of the wire format.
inet_neta cannot handle IPv6 addresses.
res_hostalias has been superseded by getaddrinfo with AI_CANONNAME.
hostalias is not thread-safe.
Some functions have int as size arguments instead of size_t, so they
do not follow current coding practices. However, dn_expand and
b64_ntop are somewhat widely used (to name just two examples), so
deprecating them seems problematic.
Reviewed-by: Carlos O'Donell <carlos@systemhalted.org>
RFC 8335 defines the network utility PROBE, which builds off of the
capabilities of Ping to query more detailed interface information from
networking nodes.
The definitions included in this patchset have been accepted into the
linux net-next branch and will be included in Linux 5.13. This
patchset adds the same definitions to the glibc for use in the
iputils package.
The relevant commits for the Linux definitions can be found here:
e542d29ca8750f4fc2a1
These changes have been tested by running the glibc tests on x86_64
Signed-off-by: Andreas Roeseler <andreas.a.roeseler@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
For the legacy ABI with supports 32-bit time_t it calls the 64-bit
time directly, since the LFS symbols calls the 64-bit time_t ones
internally.
Checked on i686-linux-gnu and x86_64-linux-gnu.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Some symbols have explicit versioned_symbol or compat_symbol markers
in the sources, but no corresponding entry in the Versions files.
This presently works because the local: * directive is only applied
to the base version.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This change continues the improvements to compile-time out of bounds
checking by decorating more APIs with either attribute access, or by
explicitly providing the array bound in APIs such as tmpnam() that
expect arrays of some minimum size as arguments. (The latter feature
is new in GCC 11.)
The only effects of the attribute and/or the array bound is to check
and diagnose calls to the functions that fail to provide a sufficient
number of elements, and the definitions of the functions that access
elements outside the specified bounds. (There is no interplay with
_FORTIFY_SOURCE here yet.)
Tested with GCC 7 through 11 on x86_64-linux.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
On GNU/Hurd we not only need $(common-objpfx) in LD_LIBRARY_PATH when loading
dynamic objects, but also $(common-objpfx)/mach and $(common-objpfx)/hurd. This
adds an ld-library-path variable to be used as LD_LIBRARY_PATH basis in
Makefiles, and a sysdep-ld-library-path variable for sysdeps to add some
more paths, here mach/ and hurd/.
It replaces the internal usage of __{f,l}xstat{at}{64} with the
__{f,l}stat{at}{64}. It should not change the generate code since
sys/stat.h explicit defines redirections to internal calls back to
xstat* symbols.
Checked with a build for all affected ABIs. I also check on
x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Restore <rpc/netdb.h> as an installed header. Delete the dummy header
resolv/rpc/netdb.h because inet is not an optional glibc component
(so its <rpc/netdb.h> is always available).
Fixes commit acb527929d ("Move
non-deprecated RPC-related functions from sunrpc to inet") in
combination with commit 5500cdba40
("Remove --enable-obsolete-rpc configure flag").
This includes bindresvport and the NSS-related RPC functions. This will
simplify the removal of the sunrpc functionality because these functions
no longer have to be treated specially.
The core problem here is that the filter array elements are unsigned
but the computed constants are signed. This both causes a
signededness conversion at the &= step and may cause undefined
behavior if the MSB is being modified. This patch uses unsigned
constants to avoid both cases. - DJ
This change brings 64 bit time support to inet deadline related code for
architectures with __WORDSIZE == 32 && __TIMESIZE != 64.
It is also safe to replace struct timespec with struct __timespec64 in
deadline related structures as:
- The __deadline_to_ms () returns the number of miliseconds to deadline to
be used with __poll (and hence it is a relative value).
- To calculate the deadline from timeval (which will be converted latter)
the uintmax_t type is used (unsinged long long int).
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Since gettimeofday will shortly be implemented in terms of
clock_gettime on all platforms, internal code should use clock_gettime
directly; in addition to removing a layer of indirection, this will
allow us to remove the PLT-bypass gunk for gettimeofday. (We can't
quite do that yet, but it'll be coming later in this patch series.)
In many cases, the changed code does fewer conversions.
The changed code always assumes __clock_gettime (CLOCK_REALTIME)
cannot fail. Most of the call sites were assuming gettimeofday could
not fail, but a few places were checking for errors. POSIX says
clock_gettime can only fail if the clock constant is invalid or
unsupported, and CLOCK_REALTIME is the one and only clock constant
that's required to be supported. For consistency I grepped the entire
source tree for any other places that checked for errors from
__clock_gettime (CLOCK_REALTIME), found one, and changed it too.
(For the record, POSIX also says gettimeofday can never fail.)
(It would be nice if we could declare that GNU systems will always
support CLOCK_MONOTONIC as well as CLOCK_REALTIME; there are several
places where we are using CLOCK_REALTIME where _MONOTONIC would be
more appropriate, and/or trying to use _MONOTONIC and then falling
back to _REALTIME. But the Hurd doesn't support CLOCK_MONOTONIC yet,
and it looks like adding it would involve substantial changes to
gnumach's internals and API. Oh well.)
A few Hurd-specific files were changed to use __host_get_time instead
of __clock_gettime, as this seemed tidier. We also assume this cannot
fail. Skimming the code in gnumach leads me to believe the only way
it could fail is if __mach_host_self also failed, and our
Hurd-specific code consistently assumes that can't happen, so I'm
going with that.
With the exception of support/support_test_main.c, test cases are not
modified, mainly because I didn't want to have to figure out which
test cases were testing gettimeofday specifically.
The definition of GETTIME in sysdeps/generic/memusage.h had a typo and
was not reading tv_sec at all. I fixed this. It appears nobody has been
generating malloc traces on a machine that doesn't have a superseding
definition.
There are a whole bunch of places where the code could be simplified
by factoring out timespec subtraction and/or comparison logic, but I
want to keep this patch as mechanical as possible.
Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64le-linux-gnu,
powerpc64-linux-gnu, powerpc-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Reviewed-by: Lukasz Majewski <lukma@denx.de>
The total_deadline variable inside the clntudp_call() function inside
sunrpc/clnt_udp.c can cause uninitalised variable warnings when building
with GCC 8.3 or 9.2 on a platform with a 64-bit tv_nsec on a 32-bit
architecture. To fix the warning let's use the DIAG_* macros to hide the
warning.
A GCC bug case has also been submitted:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91691
2019-09-24 Alistair Francis <alistair.francis@wdc.com>
* inet/net-internal.h: Fix uninitalised clntudp_call() variable.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This is a major rewrite of the description of 'crypt', 'getentropy',
and 'getrandom'.
A few highlights of the content changes:
- Throughout the manual, public headers, and user-visible messages,
I replaced the term "password" with "passphrase", the term
"password database" with "user database", and the term
"encrypt(ion)" with "(one-way) hashing" whenever it was applied to
passphrases. I didn't bother making this change in internal code
or tests. The use of the term "password" in ruserpass.c survives,
because that refers to a keyword in netrc files, but it is adjusted
to make this clearer.
There is a note in crypt.texi explaining that they were
traditionally called passwords but single words are not good enough
anymore, and a note in users.texi explaining that actual passphrase
hashes are found in a "shadow" database nowadays.
- There is a new short introduction to the "Cryptographic Functions"
section, explaining how we do not intend to be a general-purpose
cryptography library, and cautioning that there _are_, or have
been, legal restrictions on the use of cryptography in many
countries, without getting into any kind of detail that we can't
promise to keep up to date.
- I added more detail about what a "one-way function" is, and why
they are used to obscure passphrases for storage. I removed the
paragraph saying that systems not connected to a network need no
user authentication, because that's a pretty rare situation
nowadays. (It still says "sometimes it is necessary" to
authenticate the user, though.)
- I added documentation for all of the hash functions that glibc
actually supports, but not for the additional hash functions
supported by libxcrypt. If we're going to keep this manual section
around after the transition is more advanced, it would probably
make sense to add them then.
- There is much more detailed discussion of how to generate a salt,
and the failure behavior for crypt is documented. (Returning an
invalid hash on failure is what libxcrypt does; Solar Designer's
notes say that this was done "for compatibility with old programs
that assume crypt can never fail".)
- As far as I can tell, the header 'crypt.h' is entirely a GNU
invention, and never existed on any other Unix lineage. The
function 'crypt', however, was in Issue 1 of the SVID and is now
in the XSI component of POSIX. I tried to make all of the
@standards annotations consistent with this, but I'm not sure I got
them perfectly right.
- The genpass.c example has been improved to use getentropy instead
of the current time to generate the salt, and to use a SHA-256 hash
instead of MD5. It uses more random bytes than is strictly
necessary because I didn't want to complicate the code with proper
base64 encoding.
- The testpass.c example has three hardwired hashes now, to
demonstrate that different one-way functions produce different
hashes for the same input. It also demonstrates how DES hashing
only pays attention to the first eight characters of the input.
- There is new text explaining in more detail how a CSPRNG differs
from a regular random number generator, and how
getentropy/getrandom are not exactly a CSPRNG. I tried not to make
specific falsifiable claims here. I also tried to make the
blocking/cancellation/error behavior of both getentropy and
getrandom clearer.
