Commit Graph

147 Commits

Author SHA1 Message Date
Paul Eggert
dff8da6b3e Update copyright dates with scripts/update-copyrights 2024-01-01 10:53:40 -08:00
Adhemerval Zanella
bc6d79f4ae malloc: Improve MAP_HUGETLB with glibc.malloc.hugetlb=2
Even for explicit large page support, allocation might use mmap without
the hugepage bit set if the requested size is smaller than
mmap_threshold.  For this case where mmap is issued, MAP_HUGETLB is set
iff the allocation size is larger than the used large page.

To force such allocations to use large pages, also tune the mmap_threhold
(if it is not explicit set by a tunable).  This forces allocation to
follow the sbrk path, which will fall back to mmap (which will try large
pages before galling back to default mmap).

Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
Tested-by: Zhangfei Gao <zhangfei.gao@linaro.org>
2023-11-29 09:30:04 -03:00
Adhemerval Zanella
472894d2cf malloc: Use __get_nprocs on arena_get2 (BZ 30945)
This restore the 2.33 semantic for arena_get2.  It was changed by
11a02b035b to avoid arena_get2 call malloc (back when __get_nproc
was refactored to use an scratch_buffer - 903bc7dcc2).  The
__get_nproc was refactored over then and now it also avoid to call
malloc.

The 11a02b035b did not take in consideration any performance
implication, which should have been discussed properly.  The
__get_nprocs_sched is still used as a fallback mechanism if procfs
and sysfs is not acessible.

Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-22 09:39:29 -03:00
Adhemerval Zanella
fee9e40a8d malloc: Decorate malloc maps
Add anonymous mmap annotations on loader malloc, malloc when it
allocates memory with mmap, and on malloc arena.  The /proc/self/maps
will now print:

   [anon: glibc: malloc arena]
   [anon: glibc: malloc]
   [anon: glibc: loader malloc]

On arena allocation, glibc annotates only the read/write mapping.

Checked on x86_64-linux-gnu and aarch64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-07 10:27:20 -03:00
Paul Pluzhnikov
7f0d9e61f4 Fix all the remaining misspellings -- BZ 25337 2023-06-02 01:39:48 +00:00
Adhemerval Zanella Netto
33237fe83d Remove --enable-tunables configure option
And make always supported.  The configure option was added on glibc 2.25
and some features require it (such as hwcap mask, huge pages support, and
lock elisition tuning).  It also simplifies the build permutations.

Changes from v1:
 * Remove glibc.rtld.dynamic_sort changes, it is orthogonal and needs
   more discussion.
 * Cleanup more code.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-03-29 14:33:06 -03:00
Joseph Myers
6d7e8eda9b Update copyright dates with scripts/update-copyrights 2023-01-06 21:14:39 +00:00
Joseph Myers
c923cd8c49 Avoid use of atoi in malloc
This patch is analogous to commit
a3708cf6b0.

atoi has undefined behavior on out-of-range input, which makes it
problematic to use anywhere in glibc that might be processing input
out-of-range for atoi but not specified to produce undefined behavior
for the function calling atoi.  In conjunction with the C2x strtol
changes, use of atoi in libc can also result in localplt test failures
because the redirection for strtol does not interact properly with the
libc_hidden_proto call for __isoc23_strtol for the call in the inline
atoi implementation.

In malloc/arena.c, this issue shows up for atoi calls that are only
compiled for --disable-tunables (thus with the
x86_64-linux-gnu-minimal configuration of build-many-glibcs.py, for
example).  Change those atoi calls to use strtol directly, as in the
previous such changes.

Tested for x86_64 (--disable-tunables).
2022-12-22 19:37:09 +00:00
Carlos Eduardo Seo
d08dec4ea7 malloc: Use uintptr_t for pointer alignment
Avoid integer casts that assume unsigned long can represent pointers.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-11-01 09:48:22 +00:00
Florian Weimer
85860ad6ea malloc: Do not use MAP_NORESERVE to allocate heap segments
Address space for heap segments is reserved in a mmap call with
MAP_ANONYMOUS | MAP_PRIVATE and protection flags PROT_NONE.  This
reservation does not count against the RSS limit of the process or
system.  Backing memory is allocated using mprotect in alloc_new_heap
and grow_heap, and at this point, the allocator expects the kernel
to provide memory (subject to memory overcommit).

The SIGSEGV that might generate due to MAP_NORESERVE (according to
the mmap manual page) does not seem to occur in practice, it's always
SIGKILL from the OOM killer.  Even if there is a way that SIGSEGV
could be generated, it is confusing to applications that this only
happens for secondary heaps, not for large mmap-based allocations,
and not for the main arena.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2022-08-15 16:45:40 +02:00
Florian Weimer
032712621f Remove spurious references to _dl_open_hook
_dl_open_hook was removed in commit 466c1ea15f
("dlfcn: Rework static dlopen hooks").

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-08-01 15:50:05 +02:00
Paul Eggert
581c785bf3 Update copyright dates with scripts/update-copyrights
I used these shell commands:

../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")

and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.

I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah.  I don't
know why I run into these diagnostics whereas others evidently do not.

remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
2022-01-01 11:40:24 -08:00
Adhemerval Zanella
0f982c1827 malloc: Enable huge page support on main arena
This patch adds support huge page support on main arena allocation,
enable with tunable glibc.malloc.hugetlb=2.  The patch essentially
disable the __glibc_morecore() sbrk() call (similar when memory
tag does when sbrk() call does not support it) and fallback to
default page size if the memory allocation fails.

Checked on x86_64-linux-gnu.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-12-15 17:35:39 -03:00
Adhemerval Zanella
c1beb51d08 malloc: Add Huge Page support to arenas
It is enabled as default for glibc.malloc.hugetlb set to 2 or higher.
It also uses a non configurable minimum value and maximum value,
currently set respectively to 1 and 4 selected huge page size.

The arena allocation with huge pages does not use MAP_NORESERVE.  As
indicate by kernel internal documentation [1], the flag might trigger
a SIGBUS on soft page faults if at memory access there is no left
pages in the pool.

On systems without a reserved huge pages pool, is just stress the
mmap(MAP_HUGETLB) allocation failure.  To improve test coverage it is
required to create a pool with some allocated pages.

Checked on x86_64-linux-gnu with no reserved pages, 10 reserved pages
(which trigger mmap(MAP_HUGETBL) failures) and with 256 reserved pages
(which does not trigger mmap(MAP_HUGETLB) failures).

[1] https://www.kernel.org/doc/html/v4.18/vm/hugetlbfs_reserv.html#resv-map-modifications

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-12-15 17:35:39 -03:00
Adhemerval Zanella
98d5fcb8d0 malloc: Add Huge Page support for mmap
With the morecore hook removed, there is not easy way to provide huge
pages support on with glibc allocator without resorting to transparent
huge pages.  And some users and programs do prefer to use the huge pages
directly instead of THP for multiple reasons: no splitting, re-merging
by the VM, no TLB shootdowns for running processes, fast allocation
from the reserve pool, no competition with the rest of the processes
unlike THP, no swapping all, etc.

This patch extends the 'glibc.malloc.hugetlb' tunable: the value
'2' means to use huge pages directly with the system default size,
while a positive value means and specific page size that is matched
against the supported ones by the system.

Currently only memory allocated on sysmalloc() is handled, the arenas
still uses the default system page size.

To test is a new rule is added tests-malloc-hugetlb2, which run the
addes tests with the required GLIBC_TUNABLE setting.  On systems without
a reserved huge pages pool, is just stress the mmap(MAP_HUGETLB)
allocation failure.  To improve test coverage it is required to create
a pool with some allocated pages.

Checked on x86_64-linux-gnu.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-12-15 17:35:38 -03:00
Adhemerval Zanella
5f6d8d97c6 malloc: Add madvise support for Transparent Huge Pages
Linux Transparent Huge Pages (THP) current supports three different
states: 'never', 'madvise', and 'always'.  The 'never' is
self-explanatory and 'always' will enable THP for all anonymous
pages.  However, 'madvise' is still the default for some system and
for such case THP will be only used if the memory range is explicity
advertise by the program through a madvise(MADV_HUGEPAGE) call.

To enable it a new tunable is provided, 'glibc.malloc.hugetlb',
where setting to a value diffent than 0 enables the madvise call.

This patch issues the madvise(MADV_HUGEPAGE) call after a successful
mmap() call at sysmalloc() with sizes larger than the default huge
page size.  The madvise() call is disable is system does not support
THP or if it has the mode set to "never" and on Linux only support
one page size for THP, even if the architecture supports multiple
sizes.

To test is a new rule is added tests-malloc-hugetlb1, which run the
addes tests with the required GLIBC_TUNABLE setting.

Checked on x86_64-linux-gnu.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-12-15 17:35:14 -03:00
Adhemerval Zanella
11a02b035b misc: Add __get_nprocs_sched
This is an internal function meant to return the number of avaliable
processor where the process can scheduled, different than the
__get_nprocs which returns a the system available online CPU.

The Linux implementation currently only calls __get_nprocs(), which
in tuns calls sched_getaffinity.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2021-09-27 09:13:06 -03:00
Siddhesh Poyarekar
30891f35fa Remove "Contributed by" lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date.  Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.

Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions.  These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.

The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively.  These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:

https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dc
https://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-09-03 22:06:44 +05:30
Siddhesh Poyarekar
5b8d271571 Fix build and tests with --disable-tunables
Remove unused code and declare __libc_mallopt when !IS_IN (libc) to
allow the debug hook to build with --disable-tunables.

Also, run tst-ifunc-isa-2* tests only when tunables are enabled since
the result depends on it.

Tested on x86_64.

Reported-by: Matheus Castanho <msc@linux.ibm.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-07-23 13:57:56 +05:30
Siddhesh Poyarekar
b5bd5bfe88 glibc.malloc.check: Wean away from malloc hooks
The malloc-check debugging feature is tightly integrated into glibc
malloc, so thanks to an idea from Florian Weimer, much of the malloc
implementation has been moved into libc_malloc_debug.so to support
malloc-check.  Due to this, glibc malloc and malloc-check can no
longer work together; they use altogether different (but identical)
structures for heap management.  This should not make a difference
though since the malloc check hook is not disabled anywhere.
malloc_set_state does, but it does so early enough that it shouldn't
cause any problems.

The malloc check tunable is now in the debug DSO and has no effect
when the DSO is not preloaded.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:38:08 +05:30
Siddhesh Poyarekar
cc35896ea3 Simplify __malloc_initialized
Now that mcheck no longer needs to check __malloc_initialized (and no
other third party hook can since the symbol is not exported), make the
variable boolean and static so that it is used strictly within malloc.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:38:04 +05:30
Siddhesh Poyarekar
2d2d9f2b48 Move malloc hooks into a compat DSO
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so.  With this, the hooks now no
longer have any effect on the core library.

libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.

Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.

The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.

Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:37:59 +05:30
Siddhesh Poyarekar
55a4dd3930 Remove __morecore and __default_morecore
Make the __morecore and __default_morecore symbols compat-only and
remove their declarations from the API.  Also, include morecore.c
directly into malloc.c; this should ideally get merged into malloc in
a future cleanup.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:37:57 +05:30
Siddhesh Poyarekar
fc859c3048 Harden tcache double-free check
The tcache allocator layer uses the tcache pointer as a key to
identify a block that may be freed twice.  Since this is in the
application data area, an attacker exploiting a use-after-free could
potentially get access to the entire tcache structure through this
key.  A detailed write-up was provided by Awarau here:

https://awaraucom.wordpress.com/2020/07/19/house-of-io-remastered/

Replace this static pointer use for key checking with one that is
generated at malloc initialization.  The first attempt is through
getrandom with a fallback to random_bits(), which is a simple
pseudo-random number generator based on the clock.  The fallback ought
to be sufficient since the goal of the randomness is only to make the
key arbitrary enough that it is very unlikely to collide with user
data.

Co-authored-by: Eyal Itkin <eyalit@checkpoint.com>
2021-07-08 01:39:38 +05:30
H.J. Lu
310be3cc09 malloc: Make tunable callback functions static
Since malloc tunable callback functions are only used within the same
file, we should make them static.
2021-05-07 11:11:46 -07:00
Szabolcs Nagy
63a20eb03c malloc: Use branches instead of mtag_granule_mask
The branches may be better optimized since mtag_enabled is widely used.

Granule size larger than a chunk header is not supported since then we
cannot have both the chunk header and user area granule aligned.  To
fix that for targets with large granule, the chunk layout has to change.

So code that attempted to handle the granule mask generally was changed.
This simplified CHUNK_AVAILABLE_SIZE and the logic in malloc_usable_size.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-03-26 11:03:06 +00:00
Szabolcs Nagy
42bac88a21 malloc: Use global flag instead of function pointer dispatch for mtag
A flag check can be faster than function pointers because of how
branch prediction and speculation works and it can also remove a layer
of indirection when there is a mismatch between the malloc internal
tag_* api and __libc_mtag_* target hooks.

Memory tagging wrapper functions are moved to malloc.c from arena.c and
the logic now checks mmap_enabled.  The definition of tag_new_usable is
moved after chunk related definitions.

This refactoring also allows using mtag_enabled checks instead of
USE_MTAG ifdefs when memory tagging support only changes code logic
when memory tagging is enabled at runtime. Note: an "if (false)" code
block is optimized away even at -O0 by gcc.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-03-26 11:03:06 +00:00
Szabolcs Nagy
0c719cf42c malloc: Refactor TAG_ macros to avoid indirection
This does not change behaviour, just removes one layer of indirection
in the internal memory tagging logic.

Use tag_ and mtag_ prefixes instead of __tag_ and __mtag_ since these
are all symbols with internal linkage, private to malloc.c, so there
is no user namespace pollution issue.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-03-26 11:03:06 +00:00
Szabolcs Nagy
91e5c439d3 malloc: Simplify __mtag_tag_new_usable
The chunk cannot be a dumped one here.  The only non-obvious cases
are free and realloc which may be called on a dumped area chunk,
but in both cases it can be verified that tagging is already
avoided for dumped area chunks.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-03-26 11:03:06 +00:00
Paul Eggert
2b778ceb40 Update copyright dates with scripts/update-copyrights
I used these shell commands:

../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")

and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
2021-01-02 12:17:34 -08:00
Richard Earnshaw
3784dfc098 malloc: Basic support for memory tagging in the malloc() family
This patch adds the basic support for memory tagging.

Various flavours are supported, particularly being able to turn on
tagged memory at run-time: this allows the same code to be used on
systems where memory tagging support is not present without neededing
a separate build of glibc.  Also, depending on whether the kernel
supports it, the code will use mmap for the default arena if morecore
does not, or cannot support tagged memory (on AArch64 it is not
available).

All the hooks use function pointers to allow this to work without
needing ifuncs.

Reviewed-by: DJ Delorie <dj@redhat.com>
2020-12-21 15:25:25 +00:00
Florian Weimer
29a4db291b malloc: Use __libc_initial to detect an inner libc
The secondary/non-primary/inner libc (loaded via dlmopen, LD_AUDIT,
static dlopen) must not use sbrk to allocate member because that would
interfere with allocations in the outer libc.  On Linux, this does not
matter because sbrk itself was changed to fail in secondary libcs.

 _dl_addr occasionally shows up in profiles, but had to be used before
because __libc_multiple_libs was unreliable.  So this change achieves
a slight reduction in startup time.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2020-12-16 15:13:40 +01:00
liqingqing
961d12d947 malloc debug: fix compile error when enable macro MALLOC_DEBUG > 1
malloc debug: fix compile error when enable macro MALLOC_DEBUG > 1.

this is because commit e9c4fe93b3 has change the struct malloc_chunk's member "size" to "mchunk_size".

the reproduction is like that:
setp1: modify related Makefile.
vim ../glibc/malloc/Makefile
CPPFLAGS-malloc.o += -DMALLOC_DEBUG=2

step2: ../configure --prefix=/usr
       make -j32

this will cause the compile error:
/home/liqingqing/glibc_upstream/buildglibc/malloc/malloc.o
In file included from malloc.c:1899:0:
arena.c: In function 'dump_heap':
arena.c:422:58: error: 'struct malloc_chunk' has no member named 'size'
       fprintf (stderr, "chunk %p size %10lx", p, (long) p->size);
                                                          ^~
arena.c:428:17: error: 'struct malloc_chunk' has no member named 'size'
       else if (p->size == (0 | PREV_INUSE))

Reviewed-by: DJ Delorie <dj@redhat.com>
2020-10-30 14:49:08 -04:00
Joseph Myers
d614a75396 Update copyright dates with scripts/update-copyrights. 2020-01-01 00:14:33 +00:00
Paul Eggert
5a82c74822 Prefer https to http for gnu.org and fsf.org URLs
Also, change sources.redhat.com to sourceware.org.
This patch was automatically generated by running the following shell
script, which uses GNU sed, and which avoids modifying files imported
from upstream:

sed -ri '
  s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g
  s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g
' \
  $(find $(git ls-files) -prune -type f \
      ! -name '*.po' \
      ! -name 'ChangeLog*' \
      ! -path COPYING ! -path COPYING.LIB \
      ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \
      ! -path manual/texinfo.tex ! -path scripts/config.guess \
      ! -path scripts/config.sub ! -path scripts/install-sh \
      ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \
      ! -path INSTALL ! -path  locale/programs/charmap-kw.h \
      ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \
      ! '(' -name configure \
            -execdir test -f configure.ac -o -f configure.in ';' ')' \
      ! '(' -name preconfigure \
            -execdir test -f preconfigure.ac ';' ')' \
      -print)

and then by running 'make dist-prepare' to regenerate files built
from the altered files, and then executing the following to cleanup:

  chmod a+x sysdeps/unix/sysv/linux/riscv/configure
  # Omit irrelevant whitespace and comment-only changes,
  # perhaps from a slightly-different Autoconf version.
  git checkout -f \
    sysdeps/csky/configure \
    sysdeps/hppa/configure \
    sysdeps/riscv/configure \
    sysdeps/unix/sysv/linux/csky/configure
  # Omit changes that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines
  git checkout -f \
    sysdeps/powerpc/powerpc64/ppc-mcount.S \
    sysdeps/unix/sysv/linux/s390/s390-64/syscall.S
  # Omit change that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline
  git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S
2019-09-07 02:43:31 -07:00
DJ Delorie
c48d92b430 Add glibc.malloc.mxfast tunable
* elf/dl-tunables.list: Add glibc.malloc.mxfast.
* manual/tunables.texi: Document it.
* malloc/malloc.c (do_set_mxfast): New.
(__libc_mallopt): Call it.
* malloc/arena.c: Add mxfast tunable.
* malloc/tst-mxfast.c: New.
* malloc/Makefile: Add it.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2019-08-09 14:04:03 -04:00
H.J. Lu
2d398aa272 Remove do_set_mallopt_check prototype
Remove do_set_mallopt_check prototype since it is unused.

	* malloc/arena.c (do_set_mallopt_check): Removed.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
2019-04-23 13:12:25 -07:00
Joseph Myers
04277e02d7 Update copyright dates with scripts/update-copyrights.
* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.
2019-01-01 00:11:28 +00:00
Florian Weimer
1ecba1fafc malloc: Convert the unlink macro to the unlink_chunk function
This commit is in preparation of turning the macro into a proper
function.  The output arguments of the macro were in fact unused.

Also clean up uses of __builtin_expect.
2018-11-12 14:35:06 +01:00
Florian Weimer
124e025864 Run thread shutdown functions in an explicit order
This removes the __libc_thread_subfreeres hook in favor of explict
calls.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-06-26 15:27:12 +02:00
Joseph Myers
688903eb3e Update copyright dates with scripts/update-copyrights.
* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.
2018-01-01 00:32:25 +00:00
Florian Weimer
0a947e061d malloc: Call tcache destructor in arena_thread_freeres
It does not make sense to register separate cleanup functions for arena
and tcache since they're always going to be called together.  Call the
tcache cleanup function from within arena_thread_freeres since it at
least makes the order of those cleanups clear in the code.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2017-11-23 14:47:31 +01:00
Wilco Dijkstra
3381be5cde Improve malloc initialization sequence
The current malloc initialization is quite convoluted. Instead of
sometimes calling malloc_consolidate from ptmalloc_init, call
malloc_init_state early so that the main_arena is always initialized.
The special initialization can now be removed from malloc_consolidate.
This also fixes BZ #22159.

Check all calls to malloc_consolidate and remove calls that are
redundant initialization after ptmalloc_init, like in int_mallinfo
and __libc_mallopt (but keep the latter as consolidation is required for
set_max_fast).  Update comments to improve clarity.

Remove impossible initialization check from _int_malloc, fix assert
in do_check_malloc_state to ensure arena->top != 0.  Fix the obvious bugs
in do_check_free_chunk and do_check_remalloced_chunk to enable single
threaded malloc debugging (do_check_malloc_state is not thread safe!).

	[BZ #22159]
	* malloc/arena.c (ptmalloc_init): Call malloc_init_state.
	* malloc/malloc.c (do_check_free_chunk): Fix build bug.
	(do_check_remalloced_chunk): Fix build bug.
	(do_check_malloc_state): Add assert that checks arena->top.
	(malloc_consolidate): Remove initialization.
	(int_mallinfo): Remove call to malloc_consolidate.
	(__libc_mallopt): Clarify why malloc_consolidate is needed.
2017-10-17 18:55:16 +01:00
Florian Weimer
0c71122c0c malloc: Remove the internal_function attribute 2017-08-31 15:59:06 +02:00
Florian Weimer
a9da0bb266 malloc: Remove corrupt arena flag
This is no longer needed because we now abort immediately
once heap corruption is detected.
2017-08-30 20:08:34 +02:00
Florian Weimer
ac3ed168d0 malloc: Remove check_action variable [BZ #21754]
Clean up calls to malloc_printerr and trim its argument list.

This also removes a few bits of work done before calling
malloc_printerr (such as unlocking operations).

The tunable/environment variable still enables the lightweight
additional malloc checking, but mallopt (M_CHECK_ACTION)
no longer has any effect.
2017-08-30 20:08:34 +02:00
DJ Delorie
d5c3fafc43 Add per-thread cache to malloc
* config.make.in: Enable experimental malloc option.
* configure.ac: Likewise.
* configure: Regenerate.
* manual/install.texi: Document it.
* INSTALL: Regenerate.
* malloc/Makefile: Likewise.
* malloc/malloc.c: Add per-thread cache (tcache).
(tcache_put): New.
(tcache_get): New.
(tcache_thread_freeres): New.
(tcache_init): New.
(__libc_malloc): Use cached chunks if available.
(__libc_free): Initialize tcache if needed.
(__libc_realloc): Likewise.
(__libc_calloc): Likewise.
(_int_malloc): Prefill tcache when appropriate.
(_int_free): Likewise.
(do_set_tcache_max): New.
(do_set_tcache_count): New.
(do_set_tcache_unsorted_limit): New.
* manual/probes.texi: Document new probes.
* malloc/arena.c: Add new tcache tunables.
* elf/dl-tunables.list: Likewise.
* manual/tunables.texi: Document them.
* NEWS: Mention the per-thread cache.
2017-07-06 13:37:30 -04:00
Siddhesh Poyarekar
44330b6d32 tunables: Clean up hooks to get and set tunables
The TUNABLE_SET_VALUE and family of macros (and my later attempt to
add a TUNABLE_GET) never quite went together very well because the
overall interface was not clearly defined.  This patch is an attempt
to do just that.

This patch consolidates the API to two simple sets of macros,
TUNABLE_GET* and TUNABLE_SET*.  If TUNABLE_NAMESPACE is defined,
TUNABLE_GET takes just the tunable name, type and a (optionally NULL)
callback function to get the value of the tunable.  The callback
function, if non-NULL, is called if the tunable was externally set
(i.e. via GLIBC_TUNABLES or any future mechanism).  For example:

    val = TUNABLE_GET (check, int32_t, check_callback)

returns the value of the glibc.malloc.check tunable (assuming
TUNABLE_NAMESPACE is set to malloc) as an int32_t into VAL after
calling check_callback.

Likewise, TUNABLE_SET can be used to set the value of the tunable,
although this is currently possible only in the dynamic linker before
it relocates itself.  For example:

  TUNABLE_SET (check, int32_t, 2)

will set glibc.malloc.check to 2.  Of course, this is not possible
since we set (or read) glibc.malloc.check long after it is relocated.

To access or set a tunable outside of TUNABLE_NAMESPACE, use the
TUNABLE_GET_FULL and TUNABLE_SET_FULL macros, which have the following
prototype:

  TUNABLE_GET_FULL (glibc, tune, hwcap_mask, uint64_t, NULL)
  TUNABLE_SET_FULL (glibc, tune, hwcap_mask, uint64_t, 0xffff)

In future the tunable list may get split into mutable and immutable
tunables where mutable tunables can be modified by the library and
userspace after relocation as well and TUNABLE_SET will be more useful
than it currently is.  However whenever we actually do that split, we
will have to ensure that the mutable tunables are protected with
locks.

	* elf/Versions (__tunable_set_val): Rename to __tunable_get_val.
	* elf/dl-tunables.c: Likewise.
	(do_tunable_update_val): New function.
	(__tunable_set_val): New function.
	(__tunable_get_val): Call CB only if the tunable was externally
	initialized.
	(tunables_strtoul): Replace strval with initialized.
	* elf/dl-tunables.h (strval): Replace with a bool initialized.
	(TUNABLE_ENUM_NAME, TUNABLE_ENUM_NAME1): Adjust names to
	prevent collision.
	(__tunable_set_val): New function.
	(TUNABLE_GET, TUNABLE_GET_FULL): New macros.
	(TUNABLE_SET, TUNABLE_SET_FULL): Likewise.
	(TUNABLE_SET_VAL): Remove.
	(TUNABLE_SET_VAL_WITH_CALLBACK): Likewise.
	* README.tunables: Document the new macros.
	* malloc/arena.c (ptmalloc_init): Adjust.
2017-06-07 11:11:36 +05:30
Siddhesh Poyarekar
8cbc826c37 Fix getting tunable values on big-endian (BZ #21109)
The code to set value passed a tunable_val_t, which when cast to
int32_t on big-endian gives the wrong value.  Instead, use
tunable_val_t.numval instead, which can then be safely cast into
int32_t.
2017-02-08 14:17:17 +05:30
Joseph Myers
bfff8b1bec Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00