They are both used by __libc_freeres to free all library malloc
allocated resources to help tooling like mtrace or valgrind with
memory leak tracking.
The current scheme uses assembly markers and linker script entries
to consolidate the free routine function pointers in the RELRO segment
and to be freed buffers in BSS.
This patch changes it to use specific free functions for
libc_freeres_ptrs buffers and call the function pointer array directly
with call_function_static_weak.
It allows the removal of both the internal macros and the linker
script sections.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
C2x adds binary integer constants starting with 0b or 0B, and supports
those constants for the %i scanf format (in addition to the %b format,
which isn't yet implemented for scanf in glibc). Implement that scanf
support for glibc.
As with the strtol support, this is incompatible with previous C
standard versions, in that such an input string starting with 0b or 0B
was previously required to be parsed as 0 (with the rest of the input
potentially matching subsequent parts of the scanf format string).
Thus this patch adds 12 new __isoc23_* functions per long double
format (12, 24 or 36 depending on how many long double formats the
glibc configuration supports), with appropriate header redirection
support (generally very closely following that for the __isoc99_*
scanf functions - note that __GLIBC_USE (DEPRECATED_SCANF) takes
precedence over __GLIBC_USE (C2X_STRTOL), so the case of GNU
extensions to C89 continues to get old-style GNU %a and does not get
this new feature). The function names would remain as __isoc23_* even
if C2x ends up published in 2024 rather than 2023.
When scanf %b support is added, I think it will be appropriate for all
versions of scanf to follow C2x rules for inputs to the %b format
(given that there are no compatibility concerns for a new format).
Tested for x86_64 (full glibc testsuite). The first version was also
tested for powerpc (32-bit) and powerpc64le (stdio-common/ and wcsmbs/
tests), and with build-many-glibcs.py.
Almost all uses of rawmemchr find the end of a string. Since most targets use
a generic implementation, replacing it with strchr is better since that is
optimized by compilers into strlen (s) + s. Also fix the generic rawmemchr
implementation to use a cast to unsigned char in the if statement.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The __printf_buffer_flush_dprintf function needs to record that
the buffer has been written before reusing it. Without this
accounting, dprintf always returns zero.
Fixes commit 8ece45e4f5
("libio: Convert __vdprintf_internal to buffers").
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
This shows up as an assertion failure when sprintf is called with
a specifier like "%.8g" and libquadmath is linked in:
Fatal glibc error: printf_buffer_as_file.c:31
(__printf_buffer_as_file_commit): assertion failed:
file->stream._IO_write_ptr <= file->next->write_end
Fix this by detecting pointer wraparound in __vsprintf_internal
and saturate the addition to the end of the address space instead.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Always null-terminate the buffer and set E2BIG if the buffer is too
small. This fixes bug 27857.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The internal buffer size is set to 2048 bytes. This is less than
the original BUFSIZ value used by buffered_vfprintf before
the conversion, but it hopefully covers all cases where write
boundaries matter.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The buffer resizing algorithm is slightly different. The initial
buffer is on the stack, and small buffers are directly allocated
on the heap using the exact required size. The overhead of the
additional copy is compensated by the lowered setup cost for buffers
compared to libio streams.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
vfprintf is entangled with vfwprintf (of course), __printf_fp,
__printf_fphex, __vstrfmon_l_internal, and the strfrom family of
functions. The latter use the internal snprintf functionality,
so vsnprintf is converted as well.
The simples conversion is __printf_fphex, followed by
__vstrfmon_l_internal and __printf_fp, and finally
__vfprintf_internal and __vfwprintf_internal. __vsnprintf_internal
and strfrom* are mostly consuming the new interfaces, so they
are comparatively simple.
__printf_fp is a public symbol, so the FILE *-based interface
had to preserved.
The __printf_fp rewrite does not change the actual binary-to-decimal
conversion algorithm, and digits are still not emitted directly to
the target buffer. However, the staging buffer now uses bytes
instead of wide characters, and one buffer copy is eliminated.
The changes are at least performance-neutral in my testing.
Floating point printing and snprintf improved measurably, so that
this Lua script
for i=1,5000000 do
print(i, i * math.pi)
end
runs about 5% faster for me. To preserve fprintf performance for
a simple "%d" format, this commit has some logic changes under
LABEL (unsigned_number) to avoid additional function calls. There
are certainly some very easy performance improvements here: binary,
octal and hexadecimal formatting can easily avoid the temporary work
buffer (the number of digits can be computed ahead-of-time using one
of the __builtin_clz* built-ins). Decimal formatting can use a
specialized version of _itoa_word for base 10.
The existing (inconsistent) width handling between strfmon and printf
is preserved here. __print_fp_buffer_1 would have to use
__translated_number_width to achieve ISO conformance for printf.
Test expectations in libio/tst-vtables-common.c are adjusted because
the internal staging buffer merges all virtual function calls into
one.
In general, stack buffer usage is greatly reduced, particularly for
unbuffered input streams. __printf_fp can still use a large buffer
in binary128 mode for %g, though.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
clang emits an warning when a double alias redirection is used, to warn
the the original symbol will be used even when weak definition is
overridden. However, this is a common pattern for weak_alias, where
multiple alias are set to same symbol.
Reviewed-by: Fangrui Song <maskray@google.com>
In the future, this will result in a compilation failure if the
macros are unexpectedly undefined (due to header inclusion ordering
or header inclusion missing altogether).
Assembler sources are more difficult to convert. In many cases,
they are hand-optimized for the mangling and no-mangling variants,
which is why they are not converted.
sysdeps/s390/s390-32/__longjmp.c and sysdeps/s390/s390-64/__longjmp.c
are special: These are C sources, but most of the implementation is
in assembler, so the PTR_DEMANGLE macro has to be undefined in some
cases, to match the assembler style.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This allows us to define a generic no-op version of PTR_MANGLE and
PTR_DEMANGLE. In the future, we can use PTR_MANGLE and PTR_DEMANGLE
unconditionally in C sources, avoiding an unintended loss of hardening
due to missing include files or unlucky header inclusion ordering.
In i386 and x86_64, we can avoid a <tls.h> dependency in the C
code by using the computed constant from <tcb-offsets.h>. <sysdep.h>
no longer includes these definitions, so there is no cyclic dependency
anymore when computing the <tcb-offsets.h> constants.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The Z modifier is a nonstandard synonymn for z (that predates z
itself) and compiler might issue an warning for in invalid
conversion specifier.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Compilers may not be able to apply asm redirections to functions after
these functions are used for the first time, e.g. clang 13.
Fix [BZ #27087] by applying all long double-related asm redirections
before using functions in bits/stdio.h.
However, as these asm redirections depend on the declarations provided
by libio/bits/stdio2.h, this header was split in 2:
- libio/bits/stdio2-decl.h contains all function declarations;
- libio/bits/stdio2.h remains with the remaining contents, including
redirections.
This also adds the access attribute to __vsnprintf_chk that was missing.
Tested with build-many-glibcs.py.
Reviewed-by: Paul E. Murphy <murphyp@linux.ibm.com>
In general, _IO_str_overflow returns the character passed as an argument
on success. However, if flush-only operation is requested by passing
EOF, returning EOF looks like an error, and the caller cannot tell
whether the operation was successful or not.
_IO_wstr_overflow had the same bug regarding WEOF.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The _IO_wfile_overflow does not check if the write pointer for wide
data is valid before access, different than _IO_file_overflow. This
leads to crash on some cases, as described by bug 28828.
The minimal sequence to produce the crash was:
#include <stdio.h>
#include <wchar.h>
int main (int ac, char **av)
{
setvbuf (stdout, NULL, _IOLBF, 0);
fgetwc (stdin);
fputwc (10, stdout); /*CRASH HERE!*/
return 0;
}
The "fgetwc(stdin);" is necessary since it triggers the bug by setting
the flag _IO_CURRENTLY_PUTTING on stdout indirectly (file wfileops.c,
function _IO_wfile_underflow, line 213).
Signed-off-by: Jose Bollo <jobol@nonadev.net>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
In _FORTIFY_SOURCE=3, the size expression may be non-constant,
resulting in branches in the inline functions remaining intact and
causing a tiny overhead. Clang (and in future, gcc) make sure that
the -1 case is always safe, i.e. any comparison of the generated
expression with (size_t)-1 is always false so that bit is taken care
of. The rest is avoidable since we want the _chk variant whenever we
have a size expression and it's not -1.
Rework the conditionals in a uniform way to clearly indicate two
conditions at compile time:
- Either the size is unknown (-1) or we know at compile time that the
operation length is less than the object size. We can call the
original function in this case. It could be that either the length,
object size or both are non-constant, but the compiler, through
range analysis, is able to fold the *comparison* to a constant.
- The size and length are known and the compiler can see at compile
time that operation length > object size. This is valid grounds for
a warning at compile time, followed by emitting the _chk variant.
For everything else, emit the _chk variant.
This simplifies most of the fortified function implementations and at
the same time, ensures that only one call from _chk or the regular
function is emitted.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
In the context of a function definition, the size hints imply that the
size of an object pointed to by one parameter is another parameter.
This doesn't make sense for the fortified versions of the functions
since that's the bit it's trying to validate.
This is harmless with __builtin_object_size since it has fairly simple
semantics when it comes to objects passed as function parameters.
With __builtin_dynamic_object_size we could (as my patchset for gcc[1]
already does) use the access attribute to determine the object size in
the general case but it misleads the fortified functions.
Basically the problem occurs when access attributes are present on
regular functions that have inline fortified definitions to generate
_chk variants; the attributes get inherited by these definitions,
causing problems when analyzing them. For example with poll(fds, nfds,
timeout), nfds is hinted using the __attr_access as being the size of
fds.
Now, when analyzing the inline function definition in bits/poll2.h, the
compiler sees that nfds is the size of fds and tries to use that
information in the function body. In _FORTIFY_SOURCE=3 case, where the
object size could be a non-constant expression, this information results
in the conclusion that nfds is the size of fds, which defeats the
purpose of the implementation because we're trying to check here if nfds
does indeed represent the size of fds. Hence for this case, it is best
to not have the access attribute.
With the attributes gone, the expression evaluation should get delayed
until the function is actually inlined into its destinations.
Disable the access attribute for fortified function inline functions
when building at _FORTIFY_SOURCE=3 to make this work better. The
access attributes remain for the _chk variants since they can be used
by the compiler to warn when the caller is passing invalid arguments.
[1] https://gcc.gnu.org/pipermail/gcc-patches/2021-October/581125.html
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The test expects stdin to be a file which is not the case when running
tests over ssh where stdin is piped in.
The test fails with:
error: xlseek.c:27: lseek64 (0, 0, 1): Illegal seek
Update the test to create a temporary file and use that to perform the
test.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
C2X adds a macro _PRINTF_NAN_LEN_MAX to <stdio.h>, giving the maximum
length of printf output for a NaN. glibc never includes an
n-char-sequence in its printf output for NaNs, so the correct value
for glibc is 4 ("-nan" or "-NAN"); define the macro accordingly.
This patch makes the macro definition conditional on __GLIBC_USE
(ISOC2X), as is generally done with features from new standard
versions. The name is in the implementation namespace for older
standards, so it would also be possible to define it unconditionally.
Tested for x86_64.
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so. With this, the hooks now no
longer have any effect on the core library.
libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.
Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.
The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.
Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
For the legacy ABI with supports 32-bit time_t it calls the 64-bit
time directly, since the LFS symbols calls the 64-bit time_t ones
internally.
Checked on i686-linux-gnu and x86_64-linux-gnu.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Some symbols have explicit versioned_symbol or compat_symbol markers
in the sources, but no corresponding entry in the Versions files.
This presently works because the local: * directive is only applied
to the base version.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The symbol has never been exported, so no compatibility symbol is
needed. Removing this file prevents ld from creation an exported
symbol in case GLIBC_2_0 expands to a symbol version which
does not have a local: *; directive in the symbol version map file.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
To help detect common kinds of memory (and other resource) management
bugs, GCC 11 adds support for the detection of mismatched calls to
allocation and deallocation functions. At each call site to a known
deallocation function GCC checks the set of allocation functions
the former can be paired with and, if the two don't match, issues
a -Wmismatched-dealloc warning (something similar happens in C++
for mismatched calls to new and delete). GCC also uses the same
mechanism to detect attempts to deallocate objects not allocated
by any allocation function (or pointers past the first byte into
allocated objects) by -Wfree-nonheap-object.
This support is enabled for built-in functions like malloc and free.
To extend it beyond those, GCC extends attribute malloc to designate
a deallocation function to which pointers returned from the allocation
function may be passed to deallocate the allocated objects. Another,
optional argument designates the positional argument to which
the pointer must be passed.
This change is the first step in enabling this extended support for
Glibc.
This change continues the improvements to compile-time out of bounds
checking by decorating more APIs with either attribute access, or by
explicitly providing the array bound in APIs such as tmpnam() that
expect arrays of some minimum size as arguments. (The latter feature
is new in GCC 11.)
The only effects of the attribute and/or the array bound is to check
and diagnose calls to the functions that fail to provide a sufficient
number of elements, and the definitions of the functions that access
elements outside the specified bounds. (There is no interplay with
_FORTIFY_SOURCE here yet.)
Tested with GCC 7 through 11 on x86_64-linux.
No new symbol version is required because there was a forwarder.
The symbol has been moved using scripts/move-symbol-to-libc.py.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
So that text_set_element/data_set_element/bss_set_element defined
variables will be retained by the linker.
Note: 'used' and 'retain' are orthogonal: 'used' makes sure the variable
will not be optimized out; 'retain' prevents section garbage collection
if the linker support SHF_GNU_RETAIN.
GNU ld 2.37 and LLD 13 will support -z start-stop-gc which allow C
identifier name sections to be GCed even if there are live
__start_/__stop_ references.
Without the change, there are some static linking problems, e.g.
_IO_cleanup (libio/genops.c) may be discarded by ld --gc-sections, so
stdout is not flushed on exit.
Note: GCC may warning 'retain' attribute ignored while __has_attribute(retain)
is 1 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99587).
Reviewed-by: H.J. Lu <hjl.tools@gmail.com>
About a decade ago, I accidentally wrote the GPLv3 license text on the
test case when the rest of glibc source is LGPL v2.1 or later. As
original author of the test (and there are no other legally
significant changes to the test) I propose to update the license text
to be consistent with the project.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
An experimental build of GCC 11 with an enhanced -Warray-bounds
reports a bug in IO_wdefault_doallocate where the function forms
an invalid past-the-end pointer to an allocated wchar_t buffer
by failingf to consider the scaling by sizeof (wchar_t).
The fix path below corrects this problem. It keeps the buffer
size the same as opposed to increasing it according to what other
code like it does.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
It replaces the internal usage of __{f,l}xstat{at}{64} with the
__{f,l}stat{at}{64}. It should not change the generate code since
sys/stat.h explicit defines redirections to internal calls back to
xstat* symbols.
Checked with a build for all affected ABIs. I also check on
x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Lukasz Majewski <lukma@denx.de>
If build with optimization, stdio.h and sys/stat.h are defining some inlining
functions. This leads to test fails if glibc is build with the following
commands. (Note that the conformtests usually builds without optimization or
other CFLAGS):
<glibc>/configure CC="gcc -O3" --prefix=/usr
make
make subdirs=conform check
- FAIL: conform/XPG4/stdio.h/conform
- FAIL: conform/XPG42/stdio.h/conform
out-files:
...
PASSCOMBINED: Availability of variable optopt
PASSCOMBINED: Type of variable optopt
Namespace violation: "getc_unlocked"
Namespace violation: "getchar_unlocked"
Namespace violation: "putc_unlocked"
Namespace violation: "putchar_unlocked"
FAIL: Namespace of <stdio.h>
----------------------------------------------------------------------------
Total number of tests : 168
Number of failed tests : 1
Number of xfailed tests : 0
Number of skipped tests : 0
- FAIL: conform/POSIX2008/sys/stat.h/conform
out-file:
...
PASSCOMBINED: Availability of function utimensat
PASSCOMBINED: Type of function utimensat
Namespace violation: "mknodat"
FAIL: Namespace of <sys/stat.h>
----------------------------------------------------------------------------
Total number of tests : 97
Number of failed tests : 1
Number of xfailed tests : 0
Number of skipped tests : 0
For getc_unlocked, getchar_unlocked, putc_unlocked, putchar_unlocked in stdio.h,
those are defined "# ifdef __USE_POSIX" instead of "#ifdef __USE_POSIX199506"
for the non-inlining declaration. See also
"Bug 20014 - stdio.h namespace for pre-threads POSIX"
(https://sourceware.org/bugzilla/show_bug.cgi?id=20014).
For mknodat in sys/stat.h, those are defined "# ifdef __USE_ATFILE" instead of
the additional guard "# if defined __USE_MISC || defined __USE_XOPEN_EXTENDED".
__nss_readline supersedes it. This reverts part of commit
3f5e3f5d06 ("libio: Implement
internal function __libc_readline_unlocked"). The internal
aliases __fseeko64 and __ftello64 are preserved because
they are needed by __nss_readline as well.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The symbol is deprecated by strerror since its usage imposes some issues
such as copy relocations.
Its internal name is also changed to _sys_errlist_internal to avoid
static linking usage. The compat code is also refactored by removing
the over enginered errlist-compat.c generation from manual entried and
extra comment token in linker script file. It disantangle the code
generation from manual and simplify both Linux and Hurd compat code.
The definitions from errlist.c are moved to errlist.h and a new test
is added to avoid a new errno entry without an associated one in manual.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also run a check-abi
on all affected platforms.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
