They are both used by __libc_freeres to free all library malloc
allocated resources to help tooling like mtrace or valgrind with
memory leak tracking.
The current scheme uses assembly markers and linker script entries
to consolidate the free routine function pointers in the RELRO segment
and to be freed buffers in BSS.
This patch changes it to use specific free functions for
libc_freeres_ptrs buffers and call the function pointer array directly
with call_function_static_weak.
It allows the removal of both the internal macros and the linker
script sections.
Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
git commit 0849eed45d ("malloc: Move MORECORE fallback mmap to
sysmalloc_mmap_fallback") moved a block of code from sysmalloc to a
new helper function sysmalloc_mmap_fallback(), but 'pagesize' is used
for the 'minsize' argument and 'MMAP_AS_MORECORE_SIZE' for the
'pagesize' argument.
Fixes: 0849eed45d ("malloc: Move MORECORE fallback mmap to sysmalloc_mmap_fallback")
Signed-off-by: Robert Morell <rmorell@nvidia.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* malloc/malloc.c (_int_malloc): remove redundant check of
unsorted bin corruption
With commit "b90ddd08f6dd688e651df9ee89ca3a69ff88cd0c"
(malloc: Additional checks for unsorted bin integrity),
same check of (bck->fd != victim) is added before checking of unsorted
chunk corruption, which was added in "bdc3009b8ff0effdbbfb05eb6b10966753cbf9b8"
(Added check before removing from unsorted list).
..
3773 if (__glibc_unlikely (bck->fd != victim)
3774 || __glibc_unlikely (victim->fd != unsorted_chunks (av)))
3775 malloc_printerr ("malloc(): unsorted double linked list corrupted");
..
..
3815 /* remove from unsorted list */
3816 if (__glibc_unlikely (bck->fd != victim))
3817 malloc_printerr ("malloc(): corrupted unsorted chunks 3");
3818 unsorted_chunks (av)->bk = bck;
..
So this extra check can be removed.
Signed-off-by: Maninder Singh <maninder1.s@samsung.com>
Signed-off-by: Ayush Mittal <ayush.m@samsung.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
I've updated copyright dates in glibc for 2023. This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.
This patch is analogous to commit
a3708cf6b0.
atoi has undefined behavior on out-of-range input, which makes it
problematic to use anywhere in glibc that might be processing input
out-of-range for atoi but not specified to produce undefined behavior
for the function calling atoi. In conjunction with the C2x strtol
changes, use of atoi in libc can also result in localplt test failures
because the redirection for strtol does not interact properly with the
libc_hidden_proto call for __isoc23_strtol for the call in the inline
atoi implementation.
In malloc/arena.c, this issue shows up for atoi calls that are only
compiled for --disable-tunables (thus with the
x86_64-linux-gnu-minimal configuration of build-many-glibcs.py, for
example). Change those atoi calls to use strtol directly, as in the
previous such changes.
Tested for x86_64 (--disable-tunables).
If there is enough space in the chunk to satisfy the new size, return
the old pointer as is, thus avoiding any locks or reallocations. The
only real place this has a benefit is in large chunks that tend to get
satisfied with mmap, since there is a large enough spare size (up to a
page) for it to matter. For allocations on heap, the extra size is
typically barely a few bytes (up to 15) and it's unlikely that it would
make much difference in performance.
Also added a smoke test to ensure that the old pointer is returned
unchanged if the new size to realloc is within usable size of the old
pointer.
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: DJ Delorie <dj@redhat.com>
Turns out scratch_buffer_dupfree internal API was unused since
commit ef0700004b
stdlib: Simplify buffer management in canonicalize
And the related test in malloc/tst-scratch_buffer had issues
so it's better to remove it completely.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
MAX_FAST_SIZE is 160 at most, so a uint8_t is sufficient. This makes
it harder to use memory corruption, by overwriting global_max_fast
with a large value, to fundamentally alter malloc behavior.
Reviewed-by: DJ Delorie <dj@redhat.com>
Rename atomic_exchange_rel/acq to use atomic_exchange_release/acquire
since these map to the standard C11 atomic builtins.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This is used to detect errors early. The read of the oldsize is
not protected by any lock, so check this value to avoid causing
bigger mistakes.
Reviewed-by: DJ Delorie <dj@redhat.com>
Address space for heap segments is reserved in a mmap call with
MAP_ANONYMOUS | MAP_PRIVATE and protection flags PROT_NONE. This
reservation does not count against the RSS limit of the process or
system. Backing memory is allocated using mprotect in alloc_new_heap
and grow_heap, and at this point, the allocator expects the kernel
to provide memory (subject to memory overcommit).
The SIGSEGV that might generate due to MAP_NORESERVE (according to
the mmap manual page) does not seem to occur in practice, it's always
SIGKILL from the OOM killer. Even if there is a way that SIGSEGV
could be generated, it is confusing to applications that this only
happens for secondary heaps, not for large mmap-based allocations,
and not for the main arena.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Redirect internal assertion failures to __libc_assert_fail, based on
based on __libc_message, which writes directly to STDERR_FILENO
and calls abort. Also disable message translation and reword the
error message slightly (adjusting stdlib/tst-bz20544 accordingly).
As a result of these changes, malloc no longer needs its own
redefinition of __assert_fail.
__libc_assert_fail needs to be stubbed out during rtld dependency
analysis because the rtld rebuilds turn __libc_assert_fail into
__assert_fail, which is unconditionally provided by elf/dl-minimal.c.
This change is not possible for the public assert macro and its
__assert_fail function because POSIX requires that the diagnostic
is written to stderr.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Since commit ec2c1fcefb ("malloc:
Abort on heap corruption, without a backtrace [BZ #21754]"),
__libc_message always terminates the process. Since commit
a289ea09ea ("Do not print backtraces
on fatal glibc errors"), the backtrace facility has been removed.
Therefore, remove enum __libc_message_action and the action
argument of __libc_message, and mark __libc_message as _No_return.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Cancellation currently cannot happen at this point because dlopen
as used by the unwind link always performs additional allocations
for libgcc_s.so.1, even if it has been loaded already as a dependency
of the main executable. But it seems prudent not to rely on this
quirk.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The implementation is based on scalar Chacha20 with per-thread cache.
It uses getrandom or /dev/urandom as fallback to get the initial entropy,
and reseeds the internal state on every 16MB of consumed buffer.
To improve performance and lower memory consumption the per-thread cache
is allocated lazily on first arc4random functions call, and if the
memory allocation fails getentropy or /dev/urandom is used as fallback.
The cache is also cleared on thread exit iff it was initialized (so if
arc4random is not called it is not touched).
Although it is lock-free, arc4random is still not async-signal-safe
(the per thread state is not updated atomically).
The ChaCha20 implementation is based on RFC8439 [1], omitting the final
XOR of the keystream with the plaintext because the plaintext is a
stream of zeros. This strategy is similar to what OpenBSD arc4random
does.
The arc4random_uniform is based on previous work by Florian Weimer,
where the algorithm is based on Jérémie Lumbroso paper Optimal Discrete
Uniform Generation from Coin Flips, and Applications (2013) [2], who
credits Donald E. Knuth and Andrew C. Yao, The complexity of nonuniform
random number generation (1976), for solving the general case.
The main advantage of this method is the that the unit of randomness is not
the uniform random variable (uint32_t), but a random bit. It optimizes the
internal buffer sampling by initially consuming a 32-bit random variable
and then sampling byte per byte. Depending of the upper bound requested,
it might lead to better CPU utilization.
Checked on x86_64-linux-gnu, aarch64-linux, and powerpc64le-linux-gnu.
Co-authored-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Yann Droneaud <ydroneaud@opteya.com>
[1] https://datatracker.ietf.org/doc/html/rfc8439
[2] https://arxiv.org/pdf/1304.1916.pdf
It is prudent not to run too much code after detecting heap
corruption, and __fxprintf is really complex. The line number
and file name do not carry much information, so it is not included
in the error message. (__libc_message only supports %s formatting.)
The function name and assertion should provide some context.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
In-band signaling avoids an uninitialized variable warning when
building with -Og and GCC 12.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
This addresses more (correct) use-after-free warnings reported by
GCC 12 on some targets.
Fixes commit c094c232eb ("Avoid
-Wuse-after-free in tests [BZ #26779].").
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
1. Also generate .d dependency files for $(tests-container) and
$(tests-printers).
2. elf: Add tst-auditmod17.os to extra-test-objs.
3. iconv: Add tst-gconv-init-failure-mod.os to extra-test-objs.
4. malloc: Rename extra-tests-objs to extra-test-objs.
5. linux: Add tst-sysconf-iov_max-uapi.o to extra-test-objs.
6. x86_64: Add tst-x86_64mod-1.o, tst-platformmod-2.o, test-libmvec.o,
test-libmvec-avx.o, test-libmvec-avx2.o and test-libmvec-avx512f.o to
extra-test-objs.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The fix for BZ#22716 replacde LD_TRACE_LOADED_OBJECTS with
LD_TRACE_PRELINKING so mtrace could record executable address
position.
To provide the same information, LD_TRACE_LOADED_OBJECTS is
extended where a value or '2' also prints the executable address
as well. It avoid adding another loader environment variable
to be used solely for mtrace. The vDSO will be printed as
a default library (with '=>' pointing the same name), which is
ok since both mtrace and ldd already handles it.
The mtrace script is changed to also parse the new format. To
correctly support PIE and non-PIE executables, both the default
mtrace address and the one calculated as used (it fixes mtrace
for non-PIE exectuable as for BZ#22716 for PIE).
Checked on x86_64-linux-gnu.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The test leaks bits from the freed pointer via the return value
in ret, and the compiler correctly identifies this issue.
We switch the test to use TEST_VERIFY and terminate the test
if any of the pointers return an unexpected alignment.
This fixes another -Wuse-after-free error when compiling glibc
with gcc 12.
Tested on x86_64 and i686 without regression.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
I've updated copyright dates in glibc for 2022. This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files. As well as the usual annual
updates, mainly dates in --version output (minus csu/version.c which
previously had to be handled manually but is now successfully updated
by update-copyrights), there is a small change to the copyright notice
in NEWS which should let NEWS get updated automatically next year.
Please remember to include 2022 in the dates for any new files added
in future (which means updating any existing uncommitted patches you
have that add new files to use the new copyright dates in them).
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
And use machine-sp.h instead. The Linux implementation is based on
already provided CURRENT_STACK_FRAME (used on nptl code) and
STACK_GROWS_UPWARD is replaced with _STACK_GROWS_UP.
The current limit on MALLOC_MMAP_THRESHOLD is either 1 Mbyte (for
32-bit apps) or 32 Mbytes (for 64-bit apps). This value was set by a
patch dated 2006 (15 years ago). Attempts to set the threshold higher
are currently ignored.
The default behavior is appropriate for many highly parallel
applications where many processes or threads are sharing RAM. In other
situations where the number of active processes or threads closely
matches the number of cores, a much higher limit may be desired by the
application designer. By today's standards on personal computers and
small servers, 2 Gbytes of RAM per core is commonly available. On
larger systems 4 Gbytes or more of RAM is sometimes available.
Instead of raising the limit to match current needs, this patch
proposes to remove the limit of the tunable, leaving the decision up
to the user of a tunable to judge the best value for their needs.
This patch does not change any of the defaults for malloc tunables,
retaining the current behavior of the dynamic malloc mmap threshold.
bugzilla 27801 - Remove upper limit on tunable MALLOC_MMAP_THRESHOLD
Reviewed-by: DJ Delorie <dj@redhat.com>
malloc/
malloc.c changed do_set_mmap_threshold to remove test
for HEAP_MAX_SIZE.
This patch adds support huge page support on main arena allocation,
enable with tunable glibc.malloc.hugetlb=2. The patch essentially
disable the __glibc_morecore() sbrk() call (similar when memory
tag does when sbrk() call does not support it) and fallback to
default page size if the memory allocation fails.
Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
It is enabled as default for glibc.malloc.hugetlb set to 2 or higher.
It also uses a non configurable minimum value and maximum value,
currently set respectively to 1 and 4 selected huge page size.
The arena allocation with huge pages does not use MAP_NORESERVE. As
indicate by kernel internal documentation [1], the flag might trigger
a SIGBUS on soft page faults if at memory access there is no left
pages in the pool.
On systems without a reserved huge pages pool, is just stress the
mmap(MAP_HUGETLB) allocation failure. To improve test coverage it is
required to create a pool with some allocated pages.
Checked on x86_64-linux-gnu with no reserved pages, 10 reserved pages
(which trigger mmap(MAP_HUGETBL) failures) and with 256 reserved pages
(which does not trigger mmap(MAP_HUGETLB) failures).
[1] https://www.kernel.org/doc/html/v4.18/vm/hugetlbfs_reserv.html#resv-map-modifications
Reviewed-by: DJ Delorie <dj@redhat.com>
With the morecore hook removed, there is not easy way to provide huge
pages support on with glibc allocator without resorting to transparent
huge pages. And some users and programs do prefer to use the huge pages
directly instead of THP for multiple reasons: no splitting, re-merging
by the VM, no TLB shootdowns for running processes, fast allocation
from the reserve pool, no competition with the rest of the processes
unlike THP, no swapping all, etc.
This patch extends the 'glibc.malloc.hugetlb' tunable: the value
'2' means to use huge pages directly with the system default size,
while a positive value means and specific page size that is matched
against the supported ones by the system.
Currently only memory allocated on sysmalloc() is handled, the arenas
still uses the default system page size.
To test is a new rule is added tests-malloc-hugetlb2, which run the
addes tests with the required GLIBC_TUNABLE setting. On systems without
a reserved huge pages pool, is just stress the mmap(MAP_HUGETLB)
allocation failure. To improve test coverage it is required to create
a pool with some allocated pages.
Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
To increase effectiveness with Transparent Huge Page with madvise, the
large page size is use instead page size for sbrk increment for the
main arena.
Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
Linux Transparent Huge Pages (THP) current supports three different
states: 'never', 'madvise', and 'always'. The 'never' is
self-explanatory and 'always' will enable THP for all anonymous
pages. However, 'madvise' is still the default for some system and
for such case THP will be only used if the memory range is explicity
advertise by the program through a madvise(MADV_HUGEPAGE) call.
To enable it a new tunable is provided, 'glibc.malloc.hugetlb',
where setting to a value diffent than 0 enables the madvise call.
This patch issues the madvise(MADV_HUGEPAGE) call after a successful
mmap() call at sysmalloc() with sizes larger than the default huge
page size. The madvise() call is disable is system does not support
THP or if it has the mode set to "never" and on Linux only support
one page size for THP, even if the architecture supports multiple
sizes.
To test is a new rule is added tests-malloc-hugetlb1, which run the
addes tests with the required GLIBC_TUNABLE setting.
Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
The change 1e5a5866cb ("Remove malloc hooks [BZ #23328]") has broken
ports that are using GLIBC_2_35, like the new OpenRISC port I am working
on.
The libc_malloc_debug.so library used to bring in the debug
infrastructure is currently essentially empty for GLIBC_2_35 ports like
mine causing mtrace tests to fail:
cat sysdeps/unix/sysv/linux/or1k/shlib-versions
DEFAULT GLIBC_2.35
ld=ld-linux-or1k.so.1
FAIL: posix/bug-glob2-mem
FAIL: posix/bug-regex14-mem
FAIL: posix/bug-regex2-mem
FAIL: posix/bug-regex21-mem
FAIL: posix/bug-regex31-mem
FAIL: posix/bug-regex36-mem
FAIL: malloc/tst-mtrace.
The issue seems to be with the ifdefs in malloc/malloc-debug.c. The
ifdefs are currently essentially exluding all symbols for ports > 2.35.
Removing the top level SHLIB_COMPAT ifdef allows things to just work.
Fixes: 1e5a5866cb ("Remove malloc hooks [BZ #23328]")
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
My build-many-glibcs.py bot shows -Waggressive-loop-optimizations
errors building the glibc testsuite for 32-bit architectures with GCC
mainline, which seem to have appeared between GCC commits
4abc0c196b10251dc80d0743ba9e8ab3e56c61ed and
d8edfadfc7a9795b65177a50ce44fd348858e844:
In function 'dynarray_long_noscratch_resize',
inlined from 'test_long_overflow' at tst-dynarray.c:489:5,
inlined from 'do_test' at tst-dynarray.c:571:3:
../malloc/dynarray-skeleton.c:391:36: error: iteration 1073741823 invokes undefined behavior [-Werror=aggressive-loop-optimizations]
391 | DYNARRAY_ELEMENT_INIT (&list->u.dynarray_header.array[i]);
tst-dynarray.c:39:37: note: in definition of macro 'DYNARRAY_ELEMENT_INIT'
39 | #define DYNARRAY_ELEMENT_INIT(e) (*(e) = 23)
| ^
In file included from tst-dynarray.c:42:
../malloc/dynarray-skeleton.c:389:37: note: within this loop
389 | for (size_t i = old_size; i < size; ++i)
| ~~^~~~~~
In function 'dynarray_long_resize',
inlined from 'test_long_overflow' at tst-dynarray.c:479:5,
inlined from 'do_test' at tst-dynarray.c:571:3:
../malloc/dynarray-skeleton.c:391:36: error: iteration 1073741823 invokes undefined behavior [-Werror=aggressive-loop-optimizations]
391 | DYNARRAY_ELEMENT_INIT (&list->u.dynarray_header.array[i]);
tst-dynarray.c:27:37: note: in definition of macro 'DYNARRAY_ELEMENT_INIT'
27 | #define DYNARRAY_ELEMENT_INIT(e) (*(e) = 17)
| ^
In file included from tst-dynarray.c:28:
../malloc/dynarray-skeleton.c:389:37: note: within this loop
389 | for (size_t i = old_size; i < size; ++i)
| ~~^~~~~~
I don't know what GCC change made these errors appear, or why they
only appear for 32-bit architectures. However, the warnings appear to
be both true (that iteration would indeed involve undefined behavior
if executed) and useless in this particular case (that iteration is
never executed, because the allocation size overflows and so the
allocation fails - but the check for allocation size overflow is in a
separate source file and so can't be seen by the compiler when
compiling this test). So use the DIAG_* macros to disable
-Waggressive-loop-optimizations around the calls in question to
dynarray_long_resize and dynarray_long_noscratch_resize in this test.
Tested with build-many-glibcs.py (GCC mainline) for arm-linux-gnueabi,
where it restores a clean testsuite build.
Hoist the NULL check for malloc_usable_size into its entry points in
malloc-debug and malloc and assume non-NULL in all callees. This fixes
BZ #28506
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Richard W.M. Jones <rjones@redhat.com>
GCC 4.9.0 added the alloc_align attribute to say that a function
argument specifies the alignment of the returned pointer. Clang supports
the attribute too. Using the attribute can allow a compiler to generate
better code if it knows the returned pointer has a minimum alignment.
See https://gcc.gnu.org/PR60092 for more details.
GCC implicitly knows the semantics of aligned_alloc and posix_memalign,
but not the obsolete memalign. As a result, GCC generates worse code
when memalign is used, compared to aligned_alloc. Clang knows about
aligned_alloc and memalign, but not posix_memalign.
This change adds a new __attribute_alloc_align__ macro to <sys/cdefs.h>
and then uses it on memalign (where it helps GCC) and aligned_alloc
(where GCC and Clang already know the semantics, but it doesn't hurt)
and xposix_memalign. It can't be used on posix_memalign because that
doesn't return a pointer (the allocated pointer is returned via a void**
parameter instead).
Unlike the alloc_size attribute, alloc_align only allows a single
argument. That means the new __attribute_alloc_align__ macro doesn't
really need to be used with double parentheses to protect a comma
between its arguments. For consistency with __attribute_alloc_size__
this patch defines it the same way, so that double parentheses are
required.
Signed-off-by: Jonathan Wakely <jwakely@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
