This is an internal function meant to return the number of avaliable
processor where the process can scheduled, different than the
__get_nprocs which returns a the system available online CPU.
The Linux implementation currently only calls __get_nprocs(), which
in tuns calls sched_getaffinity.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Record only the relative address of the caller in mtrace file. Use
LD_TRACE_PRELINKING to get the executable as well as binary vs
executable load offsets so that we may compute a base to add to the
relative address in the mtrace file. This allows us to get a valid
address to pass to addr2line in all cases.
Fixes BZ #22716.
Co-authored-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Andreas Schwab <schwab@linux-m68k.org>
Reviewed-by: DJ Delorie <dj@redhat.com>
mcheck and malloc-check no longer work with static binaries, so drop
those tests.
Reported-by: Samuel Thibault <samuel.thibault@gnu.org>
Tested-by: Samuel Thibault <samuel.thibault@gnu.org>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Remove unused code and declare __libc_mallopt when !IS_IN (libc) to
allow the debug hook to build with --disable-tunables.
Also, run tst-ifunc-isa-2* tests only when tunables are enabled since
the result depends on it.
Tested on x86_64.
Reported-by: Matheus Castanho <msc@linux.ibm.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Interpose malloc_usable_size to return the correct mcheck value for
malloc_usable_size.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Make malloc hooks symbols compat-only so that new applications cannot
link against them and remove the declarations from the API. Also
remove the unused malloc-hooks.h.
Finally, mark all symbols in libc_malloc_debug.so as compat so that
the library cannot be linked against.
Add a note about the deprecation in NEWS.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
These deprecated functions are only safe to call from
__malloc_initialize_hook and as a result, are not useful in the
general case. Move the implementations to libc_malloc_debug so that
existing binaries that need it will now have to preload the debug DSO
to work correctly.
This also allows simplification of the core malloc implementation by
dropping all the undumping support code that was added to make
malloc_set_state work.
One known breakage is that of ancient emacs binaries that depend on
this. They will now crash when running with this libc. With
LD_BIND_NOW=1, it will terminate immediately because of not being able
to find malloc_set_state but with lazy binding it will crash in
unpredictable ways. It will need a preloaded libc_malloc_debug.so so
that its initialization hook is executed to allow its malloc
implementation to work properly.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
The malloc-check debugging feature is tightly integrated into glibc
malloc, so thanks to an idea from Florian Weimer, much of the malloc
implementation has been moved into libc_malloc_debug.so to support
malloc-check. Due to this, glibc malloc and malloc-check can no
longer work together; they use altogether different (but identical)
structures for heap management. This should not make a difference
though since the malloc check hook is not disabled anywhere.
malloc_set_state does, but it does so early enough that it shouldn't
cause any problems.
The malloc check tunable is now in the debug DSO and has no effect
when the DSO is not preloaded.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Wean mtrace away from the malloc hooks and move them into the debug
DSO. Split the API away from the implementation so that we can add
the API to libc.so as well as libc_malloc_debug.so, with the libc
implementations being empty.
Update localplt data since memalign no longer has any callers after
this change.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Now that mcheck no longer needs to check __malloc_initialized (and no
other third party hook can since the symbol is not exported), make the
variable boolean and static so that it is used strictly within malloc.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Split the mcheck implementation into the debugging hooks and API so
that the API can be replicated in libc and libc_malloc_debug.so. The
libc APIs always result in failure.
The mcheck implementation has also been moved entirely into
libc_malloc_debug.so and with it, all of the hook initialization code
can now be moved into the debug library. Now the initialization can
be done independently of libc internals.
With this patch, libc_malloc_debug.so can no longer be used with older
libcs, which is not its goal anyway. tst-vfork3 breaks due to this
since it spawns shell scripts, which in turn execute using the system
glibc. Move the test to tests-container so that only the built glibc
is used.
This move also fixes bugs in the mcheck version of memalign and
realloc, thus allowing removal of the tests from tests-mcheck
exclusion list.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so. With this, the hooks now no
longer have any effect on the core library.
libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.
Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.
The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.
Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Make the __morecore and __default_morecore symbols compat-only and
remove their declarations from the API. Also, include morecore.c
directly into malloc.c; this should ideally get merged into malloc in
a future cleanup.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Remove __after_morecore_hook from the API and finalize the symbol so
that it can no longer be used in new applications. Old applications
using __after_morecore_hook will find that their hook is no longer
called.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Targets with base versions of 2.24 or later won't have
__malloc_initialize_hook because of which the tests will essentially
be the same as the regular malloc tests. Avoid running them instead
and save time.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
There is a 1 in 16 chance of a corruption escaping safe-linking and to
guard against spurious failures, tst-safe-linking runs each subtest 10
times to ensure that the chance is reduced to 1 in 2^40. However, in
the 1 in 16 chance that a corruption does escape safe linking, it
could well be caught by other sanity checks we do in malloc, which
then results in spurious test failures like below:
test test_fastbin_consolidate failed with a different error
expected: malloc_consolidate(): unaligned fastbin chunk detected
actual: malloc_consolidate(): invalid chunk size
This failure is seen more frequently on i686; I was able to reproduce
it in about 5 min of running it in a loop.
Guard against such failures by recording them and retrying the test.
Also, do not fail the test if we happened to get defeated by the 1 in
2^40 odds if in at least one of the instances it was detected by other
checks.
Finally, bolster the odds to 2^64 by running 16 times instead of 10.
The test still has a chance of failure so it is still flaky in theory.
However in practice if we see a failure here then it's more likely
that there's a bug than it being an issue with the test. Add more
printfs and also dump them to stdout so that in the event the test
actually fails, we will have some data to try and understand why it
may have failed.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
1. Align struct hdr to MALLOC_ALIGNMENT bytes so that malloc hooks in
libmcheck align memory to MALLOC_ALIGNMENT bytes.
2. Remove tst-mallocalign1 from tests-exclude-mcheck for i386 and x32.
3. Add tst-pvalloc-fortify and tst-reallocarray to tests-exclude-mcheck
since they use malloc_usable_size (see BZ #22057).
This fixed BZ #28068.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
As a result, is not necessary to specify __attribute__ ((nocommon))
on individual definitions.
GCC 10 defaults to -fno-common on all architectures except ARC,
but this change is compatible with older GCC versions and ARC, too.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
1. Add sysdeps/generic/malloc-size.h to define size related macros for
malloc.
2. Move x86_64/tst-mallocalign1.c to malloc and replace ALIGN_MASK with
MALLOC_ALIGN_MASK.
3. Add tst-mallocalign1 to tests-exclude-mcheck for i386 and x32 since
mcheck doesn't honor MALLOC_ALIGNMENT.
_int_realloc is correctly declared at the top to be static, but
incorrectly defined without the static keyword. Fix that. The
generated binaries have identical code.
The tcache allocator layer uses the tcache pointer as a key to
identify a block that may be freed twice. Since this is in the
application data area, an attacker exploiting a use-after-free could
potentially get access to the entire tcache structure through this
key. A detailed write-up was provided by Awarau here:
https://awaraucom.wordpress.com/2020/07/19/house-of-io-remastered/
Replace this static pointer use for key checking with one that is
generated at malloc initialization. The first attempt is through
getrandom with a fallback to random_bits(), which is a simple
pseudo-random number generator based on the clock. The fallback ought
to be sufficient since the goal of the randomness is only to make the
key arbitrary enough that it is very unlikely to collide with user
data.
Co-authored-by: Eyal Itkin <eyalit@checkpoint.com>
It's tst-realloc, not tst-posix-realloc. Verified this time to ensure
that the total number of tests reduced by 1.
Reported-by: Stefan Liebler <stli@linux.ibm.com>
The realloc (NULL, 0) test in tst-realloc fails with gcc 7.x but
passes with newer gcc. This is because a newer gcc transforms the
realloc call to malloc (0), thus masking the bug in mcheck.
Disable the test with mcheck for now. The malloc removal patchset
will fix this and then remove this test from the exclusion list.
Reported-by: Stefan Liebler <stli@linux.ibm.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Separate the malloc check implementation from the malloc hooks. They
still use the hooks but are now maintained in a separate file.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The variable and function pair appear to provide a way for users to
set conditional breakpoints in mtrace when a specific address is
returned by the allocator. This can be achieved by using conditional
breakpoints in gdb so it is redundant. There is no documentation of
this interface in the manual either, so it appears to have been a hack
that got added to debug malloc. Deprecate these symbols and do not
call tr_break anymore.
Reviewed-by: DJ Delorie <dj@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Dependencies on hooks.c and arena.c get auto-computed when generating
malloc.o{,s}.d so there is no need to add them manually.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Andreas Schwab <schwab@linux-m68k.org>
After commit 1e26d35193 ("malloc: Fix
tcache leak after thread destruction [BZ #22111]"),
tcache_shutting_down is still not early enough. When we detach a
thread with no tcache allocated, tcache_shutting_down would still be
false.
Reviewed-by: DJ Delorie <dj@redhat.com>
Like malloc-check, add generic rules to run all tests in malloc by
linking with libmcheck.a so as to provide coverage for mcheck().
Currently the following 12 tests fail:
FAIL: malloc/tst-malloc-backtrace-mcheck
FAIL: malloc/tst-malloc-fork-deadlock-mcheck
FAIL: malloc/tst-malloc-stats-cancellation-mcheck
FAIL: malloc/tst-malloc-tcache-leak-mcheck
FAIL: malloc/tst-malloc-thread-exit-mcheck
FAIL: malloc/tst-malloc-thread-fail-mcheck
FAIL: malloc/tst-malloc-usable-static-mcheck
FAIL: malloc/tst-malloc-usable-static-tunables-mcheck
FAIL: malloc/tst-malloc-usable-tunables-mcheck
FAIL: malloc/tst-malloc_info-mcheck
FAIL: malloc/tst-memalign-mcheck
FAIL: malloc/tst-posix_memalign-mcheck
and they have been added to tests-exclude-mcheck for now to keep
status quo. At least the last two can be attributed to bugs in
mcheck() but I haven't fixed them here since they should be fixed by
removing malloc hooks. Others need to be triaged to check if they're
due to mcheck bugs or due to actual bugs.
Reviewed-by: DJ Delorie <dj@redhat.com>
Austin Group issue 62 [1] dropped the async-signal-safe requirement
for fork and provided a async-signal-safe _Fork replacement that
does not run the atfork handlers. It will be included in the next
POSIX standard.
It allow to close a long standing issue to make fork AS-safe (BZ#4737).
As indicated on the bug, besides the internal lock for the atfork
handlers itself; there is no guarantee that the handlers itself will
not introduce more AS-safe issues.
The idea is synchronize fork with the required internal locks to allow
children in multithread processes to use mostly of standard function
(even though POSIX states only AS-safe function should be used). On
signal handles, _Fork should be used intead and only AS-safe functions
should be used.
For testing, the new tst-_Fork only check basic usage. I also added
a new tst-mallocfork3 which uses the same strategy to check for
deadlock of tst-mallocfork2 but using threads instead of subprocesses
(and it does deadlock if it replaces _Fork with fork).
[1] https://austingroupbugs.net/view.php?id=62
MALLOC_CHECK_ and mcheck() are two different malloc checking features.
tst-mcheck does not check mcheck(), instead it checks MALLOC_CHECK_,
so rename the file to avoid confusion.
This commit removes the ELF constructor and internal variables from
dlfcn/dlfcn.c. The file now serves the same purpose as
nptl/libpthread-compat.c, so it is renamed to dlfcn/libdl-compat.c.
The use of libdl-shared-only-routines ensures that libdl.a is empty.
This commit adjusts the test suite not to use $(libdl). The libdl.so
symbolic link is no longer installed.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Since test uses 160 multiple for malloc size, we should also use 160 multiple
for total variable instead of 16, then comparison is meaningful. So fix it.
Also change the ">" to ">=" so that the test is technically valid.
Reviewed-by: DJ Delorie <dj@redhat.com>
When MALLOC_CHECK_ is non-zero, the realloc hook missed to set errno to
ENOMEM when called with too big size. Run the test tst-malloc-too-large
also with MALLOC_CHECK_=3 to catch that.
To help detect common kinds of memory (and other resource) management
bugs, GCC 11 adds support for the detection of mismatched calls to
allocation and deallocation functions. At each call site to a known
deallocation function GCC checks the set of allocation functions
the former can be paired with and, if the two don't match, issues
a -Wmismatched-dealloc warning (something similar happens in C++
for mismatched calls to new and delete). GCC also uses the same
mechanism to detect attempts to deallocate objects not allocated
by any allocation function (or pointers past the first byte into
allocated objects) by -Wfree-nonheap-object.
This support is enabled for built-in functions like malloc and free.
To extend it beyond those, GCC extends attribute malloc to designate
a deallocation function to which pointers returned from the allocation
function may be passed to deallocate the allocated objects. Another,
optional argument designates the positional argument to which
the pointer must be passed.
This change is the first step in enabling this extended support for
Glibc.
(FYI, this is a repost of
https://sourceware.org/pipermail/libc-alpha/2019-July/105035.html now
that FSF papers have been signed and confirmed on FSF side).
This trivial patch attemps to fix BZ 24106. Basically the bash locally
used when building glibc on the host shall not leak on the installed
glibc, as the system where it is installed might be different and use
another bash location.
So I have looked for all occurences of @BASH@ or $(BASH) in installed
files, and replaced it by /bin/bash. This was suggested by Florian
Weimer in the bug report.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
This replaces the FREE_P macro with the __nptl_stack_in_use inline
function. stack_list_del is renamed to __nptl_stack_list_del,
stack_list_add to __nptl_stack_list_add, __deallocate_stack to
__nptl_deallocate_stack, free_stacks to __nptl_free_stacks.
It is convenient to move __libpthread_freeres into libc at the
same time. This removes the temporary __default_pthread_attr_freeres
export and restores full freeres coverage for __default_pthread_attr.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Calling free directly may end up freeing a pointer allocated by the
dynamic loader using malloc from libc.so in the base namespace using
the allocator from libc.so in a secondary namespace, which results in
crashes.
This commit redirects the free call through GLRO and the dynamic
linker, to reach the correct namespace. It also cleans up the dlerror
handling along the way, so that pthread_setspecific is no longer
needed (which avoids triggering bug 24774).
