AuroraMiddleware/glibc
1
0
Fork 0
mirror of https://sourceware.org/git/glibc.git synced 2024-11-22 13:00:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
709fbd3ec3
glibc/manual/tunables.texi

737 lines
31 KiB
Plaintext
Raw Normal View History

User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
@node Tunables
@c @node Tunables, , Internal Probes, Top
@c %MENU% Tunable switches to alter libc internal behavior
@chapter Tunables
@cindex tunables
@dfn{Tunables} are a feature in @theglibc{} that allows application authors and
distribution maintainers to alter the runtime library behavior to match
their workload. These are implemented as a set of switches that may be
modified in different ways. The current default method to do this is via
the @env{GLIBC_TUNABLES} environment variable by setting it to a string
of colon-separated @var{name}=@var{value} pairs. For example, the following
Improve documentation for malloc etc. (BZ#27719) Cover key corner cases (e.g., whether errno is set) that are well settled in glibc, fix some examples to avoid integer overflow, and update some other dated examples (code needed for K&R C, e.g.). * manual/charset.texi (Non-reentrant String Conversion): * manual/filesys.texi (Symbolic Links): * manual/memory.texi (Allocating Cleared Space): * manual/socket.texi (Host Names): * manual/string.texi (Concatenating Strings): * manual/users.texi (Setting Groups): Use reallocarray instead of realloc, to avoid integer overflow issues. * manual/filesys.texi (Scanning Directory Content): * manual/memory.texi (The GNU Allocator, Hooks for Malloc): * manual/tunables.texi: Use code font for 'malloc' instead of roman font. (Symbolic Links): Don't assume readlink return value fits in 'int'. * manual/memory.texi (Memory Allocation and C, Basic Allocation) (Malloc Examples, Alloca Example): * manual/stdio.texi (Formatted Output Functions): * manual/string.texi (Concatenating Strings, Collation Functions): Omit pointer casts that are needed only in ancient K&R C. * manual/memory.texi (Basic Allocation): Say that malloc sets errno on failure. Say "convert" rather than "cast", since casts are no longer needed. * manual/memory.texi (Basic Allocation): * manual/string.texi (Concatenating Strings): In examples, use C99 declarations after statements for brevity. * manual/memory.texi (Malloc Examples): Add portability notes for malloc (0), errno setting, and PTRDIFF_MAX. (Changing Block Size): Say that realloc (p, 0) acts like (p ? (free (p), NULL) : malloc (0)). Add xreallocarray example, since other examples can use it. Add portability notes for realloc (0, 0), realloc (p, 0), PTRDIFF_MAX, and improve notes for reallocating to the same size. (Allocating Cleared Space): Reword now-confusing discussion about replacement, and xref "Replacing malloc". * manual/stdio.texi (Formatted Output Functions): Don't assume message size fits in 'int'. * manual/string.texi (Concatenating Strings): Fix undefined behavior involving arithmetic on a freed pointer.
2021-04-12 02:06:00 +00:00
example enables @code{malloc} checking and sets the @code{malloc}
trim threshold to 128
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
bytes:
@example
GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3
export GLIBC_TUNABLES
@end example
Tunables are not part of the @glibcadj{} stable ABI, and they are
subject to change or removal across releases. Additionally, the method to
modify tunable values may change between releases and across distributions.
It is possible to implement multiple `frontends' for the tunables allowing
distributions to choose their preferred method at build time.
Finally, the set of tunables available may vary between distributions as
the tunables feature allows distributions to add their own tunables under
their own namespace.
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
Passing @option{--list-tunables} to the dynamic loader to print all
tunables with minimum and maximum values:
@example
$ /lib64/ld-linux-x86-64.so.2 --list-tunables
glibc.rtld.nns: 0x4 (min: 0x1, max: 0x10)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.elision.skip_lock_after_retries: 3 (min: 0, max: 2147483647)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.malloc.trim_threshold: 0x0 (min: 0x0, max: 0xffffffffffffffff)
glibc.malloc.perturb: 0 (min: 0, max: 255)
glibc.cpu.x86_shared_cache_size: 0x100000 (min: 0x0, max: 0xffffffffffffffff)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.pthread.rseq: 1 (min: 0, max: 1)
glibc.cpu.prefer_map_32bit_exec: 0 (min: 0, max: 1)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.mem.tagging: 0 (min: 0, max: 255)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.elision.tries: 3 (min: 0, max: 2147483647)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.elision.enable: 0 (min: 0, max: 1)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.malloc.hugetlb: 0x0 (min: 0x0, max: 0xffffffffffffffff)
glibc.cpu.x86_rep_movsb_threshold: 0x2000 (min: 0x100, max: 0xffffffffffffffff)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.malloc.mxfast: 0x0 (min: 0x0, max: 0xffffffffffffffff)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.rtld.dynamic_sort: 2 (min: 1, max: 2)
glibc.elision.skip_lock_busy: 3 (min: 0, max: 2147483647)
glibc.malloc.top_pad: 0x20000 (min: 0x0, max: 0xffffffffffffffff)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.cpu.x86_rep_stosb_threshold: 0x800 (min: 0x1, max: 0xffffffffffffffff)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.cpu.x86_non_temporal_threshold: 0xc0000 (min: 0x4040, max: 0xfffffffffffffff)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.cpu.x86_shstk:
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.pthread.stack_cache_size: 0x2800000 (min: 0x0, max: 0xffffffffffffffff)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.cpu.hwcap_mask: 0x6 (min: 0x0, max: 0xffffffffffffffff)
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.malloc.mmap_max: 0 (min: 0, max: 2147483647)
glibc.elision.skip_trylock_internal_abort: 3 (min: 0, max: 2147483647)
elf: Add ELF_DYNAMIC_AFTER_RELOC to rewrite PLT Add ELF_DYNAMIC_AFTER_RELOC to allow target specific processing after relocation. For x86-64, add #define DT_X86_64_PLT (DT_LOPROC + 0) #define DT_X86_64_PLTSZ (DT_LOPROC + 1) #define DT_X86_64_PLTENT (DT_LOPROC + 3) 1. DT_X86_64_PLT: The address of the procedure linkage table. 2. DT_X86_64_PLTSZ: The total size, in bytes, of the procedure linkage table. 3. DT_X86_64_PLTENT: The size, in bytes, of a procedure linkage table entry. With the r_addend field of the R_X86_64_JUMP_SLOT relocation set to the memory offset of the indirect branch instruction. Define ELF_DYNAMIC_AFTER_RELOC for x86-64 to rewrite the PLT section with direct branch after relocation when the lazy binding is disabled. PLT rewrite is disabled by default since SELinux may disallow modifying code pages and ld.so can't detect it in all cases. Use $ export GLIBC_TUNABLES=glibc.cpu.plt_rewrite=1 to enable PLT rewrite with 32-bit direct jump at run-time or $ export GLIBC_TUNABLES=glibc.cpu.plt_rewrite=2 to enable PLT rewrite with 32-bit direct jump and on APX processors with 64-bit absolute jump at run-time. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
2024-01-05 04:19:39 +00:00
glibc.cpu.plt_rewrite: 0 (min: 0, max: 2)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.malloc.tcache_unsorted_limit: 0x0 (min: 0x0, max: 0xffffffffffffffff)
glibc.cpu.x86_ibt:
glibc.cpu.hwcaps:
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
glibc.elision.skip_lock_internal_abort: 3 (min: 0, max: 2147483647)
ld.so: Add --list-tunables to print tunable values Pass --list-tunables to ld.so to print tunables with min and max values. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2020-07-12 13:04:53 +00:00
glibc.malloc.arena_max: 0x0 (min: 0x1, max: 0xffffffffffffffff)
glibc.malloc.mmap_threshold: 0x0 (min: 0x0, max: 0xffffffffffffffff)
glibc.cpu.x86_data_cache_size: 0x8000 (min: 0x0, max: 0xffffffffffffffff)
glibc.malloc.tcache_count: 0x0 (min: 0x0, max: 0xffffffffffffffff)
glibc.malloc.arena_test: 0x0 (min: 0x1, max: 0xffffffffffffffff)
glibc.pthread.mutex_spin_count: 100 (min: 0, max: 32767)
glibc.rtld.optional_static_tls: 0x200 (min: 0x0, max: 0xffffffffffffffff)
glibc.malloc.tcache_max: 0x0 (min: 0x0, max: 0xffffffffffffffff)
glibc.malloc.check: 0 (min: 0, max: 3)
@end example
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
@menu
* Tunable names:: The structure of a tunable name
* Memory Allocation Tunables:: Tunables in the memory allocation subsystem
rtld: Add rtld.nns tunable for the number of supported namespaces TLS_STATIC_SURPLUS is 1664 bytes currently which is not enough to support DL_NNS (== 16) number of dynamic link namespaces, if we assume 192 bytes of TLS are reserved for libc use and 144 bytes are reserved for other system libraries that use IE TLS. A new tunable is introduced to control the number of supported namespaces and to adjust the surplus static TLS size as follows: surplus_tls = 192 * (rtld.nns-1) + 144 * rtld.nns + 512 The default is rtld.nns == 4 and then the surplus TLS size is the same as before, so the behaviour is unchanged by default. If an application creates more namespaces than the rtld.nns setting allows, then it is not guaranteed to work, but the limit is not checked. So existing usage will continue to work, but in the future if an application creates more than 4 dynamic link namespaces then the tunable will need to be set. In this patch DL_NNS is a fixed value and provides a maximum to the rtld.nns setting. Static linking used fixed 2048 bytes surplus TLS, this is changed so the same contract is used as for dynamic linking. With static linking DL_NNS == 1 so rtld.nns tunable is forced to 1, so by default the surplus TLS is reduced to 144 + 512 = 656 bytes. This change is not expected to cause problems. Tested on aarch64-linux-gnu and x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-06-09 08:57:28 +00:00
* Dynamic Linking Tunables:: Tunables in the dynamic linking subsystem
Add elision tunables This patch adds several new tunables to control the behavior of elision on supported platforms[1]. Since elision now depends on tunables, we should always *compile* with elision enabled, and leave the code disabled, but available for runtime selection. This gives us *much* better compile-time testing of the existing code to avoid bit-rot[2]. Tested on ppc, ppc64, ppc64le, s390x and x86_64. [1] This part of the patch was initially proposed by Paul Murphy but was "staled" because the framework have changed since the patch was originally proposed: https://patchwork.sourceware.org/patch/10342/ [2] This part of the patch was inititally proposed as a RFC by Carlos O'Donnell. Make sense to me integrate this on the patch: https://sourceware.org/ml/libc-alpha/2017-05/msg00335.html * elf/dl-tunables.list: Add elision parameters. * manual/tunables.texi: Add entries about elision tunable. * sysdeps/unix/sysv/linux/powerpc/elision-conf.c: Add callback functions to dynamically enable/disable elision. Add multiple callbacks functions to set elision parameters. Deleted __libc_enable_secure check. * sysdeps/unix/sysv/linux/s390/elision-conf.c: Likewise. * sysdeps/unix/sysv/linux/x86/elision-conf.c: Likewise. * configure: Regenerated. * configure.ac: Option enable_lock_elision was deleted. * config.h.in: ENABLE_LOCK_ELISION flag was deleted. * config.make.in: Remove references to enable_lock_elision. * manual/install.texi: Elision configure option was removed. * INSTALL: Regenerated to remove enable_lock_elision. * nptl/Makefile: Disable elision so it can verify error case for destroying a mutex. * sysdeps/powerpc/nptl/elide.h: Cleanup ENABLE_LOCK_ELISION check. Deleted macros for the case when ENABLE_LOCK_ELISION was not defined. * sysdeps/s390/configure: Regenerated. * sysdeps/s390/configure.ac: Remove references to enable_lock_elision.. * nptl/tst-mutex8.c: Deleted all #ifndef ENABLE_LOCK_ELISION from the test. * sysdeps/powerpc/powerpc32/sysdep.h: Deleted all ENABLE_LOCK_ELISION checks. * sysdeps/powerpc/powerpc64/sysdep.h: Likewise. * sysdeps/powerpc/sysdep.h: Likewise. * sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise. * sysdeps/unix/sysv/linux/powerpc/force-elision.h: Likewise. * sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise. * sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise. * sysdeps/unix/sysv/linux/s390/lowlevellock.h: Likewise. * sysdeps/unix/sysv/linux/s390/Makefile: Remove references to enable-lock-elision. Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
2017-12-05 16:24:14 +00:00
* Elision Tunables:: Tunables in elision subsystem
Mutex: Add pthread mutex tunables This patch does not have any functionality change, we only provide a spin count tunes for pthread adaptive spin mutex. The tunable glibc.pthread.mutex_spin_count tunes can be used by system administrator to squeeze system performance according to different hardware capabilities and workload characteristics. The maximum value of spin count is limited to 32767 to avoid the overflow of mutex->__data.__spins variable with the possible type of short in pthread_mutex_lock (). The default value of spin count is set to 100 with the reference to the previous number of times of spinning via trylock. This value would be architecture-specific and can be tuned with kinds of benchmarks to fit most cases in future. I would extend my appreciation sincerely to H.J.Lu for his help to refine this patch series. * manual/tunables.texi (POSIX Thread Tunables): New node. * nptl/Makefile (libpthread-routines): Add pthread_mutex_conf. * nptl/nptl-init.c: Include pthread_mutex_conf.h (__pthread_initialize_minimal_internal) [HAVE_TUNABLES]: Call __pthread_tunables_init. * nptl/pthreadP.h (MAX_ADAPTIVE_COUNT): Remove. (max_adaptive_count): Define. * nptl/pthread_mutex_conf.c: New file. * nptl/pthread_mutex_conf.h: New file. * sysdeps/generic/adaptive_spin_count.h: New file. * sysdeps/nptl/dl-tunables.list: New file. * nptl/pthread_mutex_lock.c (__pthread_mutex_lock): Use max_adaptive_count () not MAX_ADAPTIVE_COUNT. * nptl/pthread_mutex_timedlock.c (__pthrad_mutex_timedlock): Likewise. Suggested-by: Andi Kleen <andi.kleen@intel.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com> Signed-off-by: Kemi.wang <kemi.wang@intel.com>
2018-12-01 15:03:33 +00:00
* POSIX Thread Tunables:: Tunables in the POSIX thread subsystem
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
* Hardware Capability Tunables:: Tunables that modify the hardware
capabilities seen by @theglibc{}
elf: Add a tunable to control use of tagged memory Add a new glibc tunable: mem.tagging. This is a decimal constant in the range 0-255 but used as a bit-field. Bit 0 enables use of tagged memory in the malloc family of functions. Bit 1 enables precise faulting of tag failure on platforms where this can be controlled. Other bits are currently unused, but if set will cause memory tag checking for the current process to be enabled in the kernel.
2020-12-21 15:03:03 +00:00
* Memory Related Tunables:: Tunables that control the use of memory by
@theglibc{}.
gmon: improve mcount overflow handling [BZ# 27576] When mcount overflows, no gmon.out file is generated, but no message is printed to the user, leaving the user with no idea why, and thinking maybe there is some bug - which is how BZ 27576 ended up being logged. Print a message to stderr in this case so the user knows what is going on. As a comment in sys/gmon.h acknowledges, the hardcoded MAXARCS value is too small for some large applications, including the test case in that BZ. Rather than increase it, add tunables to enable MINARCS and MAXARCS to be overridden at runtime (glibc.gmon.minarcs and glibc.gmon.maxarcs). So if a user gets the mcount overflow error, they can try increasing maxarcs (they might need to increase minarcs too if the heuristic is wrong in their case.) Note setting minarcs/maxarcs too large can cause monstartup to fail with an out of memory error. If you set them large enough, it can cause an integer overflow in calculating the buffer size. I haven't done anything to defend against that - it would not generally be a security vulnerability, since these tunables will be ignored in suid/sgid programs (due to the SXID_ERASE default), and if you can set GLIBC_TUNABLES in the environment of a process, you can take it over anyway (LD_PRELOAD, LD_LIBRARY_PATH, etc). I thought about modifying the code of monstartup to defend against integer overflows, but doing so is complicated, and I realise the existing code is susceptible to them even prior to this change (e.g. try passing a pathologically large highpc argument to monstartup), so I decided just to leave that possibility in-place. Add a test case which demonstrates mcount overflow and the tunables. Document the new tunables in the manual. Signed-off-by: Simon Kissane <skissane@gmail.com> Reviewed-by: DJ Delorie <dj@redhat.com>
2023-02-11 09:12:13 +00:00
* gmon Tunables:: Tunables that control the gmon profiler, used in
conjunction with gprof
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
@end menu
@node Tunable names
@section Tunable names
@cindex Tunable names
@cindex Tunable namespaces
A tunable name is split into three components, a top namespace, a tunable
namespace and the tunable name. The top namespace for tunables implemented in
@theglibc{} is @code{glibc}. Distributions that choose to add custom tunables
in their maintained versions of @theglibc{} may choose to do so under their own
top namespace.
The tunable namespace is a logical grouping of tunables in a single
module. This currently holds no special significance, although that may
change in the future.
The tunable name is the actual name of the tunable. It is possible that
different tunable namespaces may have tunables within them that have the
same name, likewise for top namespaces. Hence, we only support
identification of tunables by their full name, i.e. with the top
namespace, tunable namespace and tunable name, separated by periods.
@node Memory Allocation Tunables
@section Memory Allocation Tunables
@cindex memory allocation tunables
@cindex malloc tunables
@cindex tunables, malloc
@deftp {Tunable namespace} glibc.malloc
Memory allocation behavior can be modified by setting any of the
following tunables in the @code{malloc} namespace:
@end deftp
@deftp Tunable glibc.malloc.check
This tunable supersedes the @env{MALLOC_CHECK_} environment variable and is
Move malloc hooks into a compat DSO Remove all malloc hook uses from core malloc functions and move it into a new library libc_malloc_debug.so. With this, the hooks now no longer have any effect on the core library. libc_malloc_debug.so is a malloc interposer that needs to be preloaded to get hooks functionality back so that the debugging features that depend on the hooks, i.e. malloc-check, mcheck and mtrace work again. Without the preloaded DSO these debugging features will be nops. These features will be ported away from hooks in subsequent patches. Similarly, legacy applications that need hooks functionality need to preload libc_malloc_debug.so. The symbols exported by libc_malloc_debug.so are maintained at exactly the same version as libc.so. Finally, static binaries will no longer be able to use malloc debugging features since they cannot preload the debugging DSO. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 13:07:59 +00:00
identical in features. This tunable has no effect by default and needs the
manual: Drop the .so suffix in libc_malloc_debug description All references to libraries in the manual are without the .so prefix, so do the same for libc_malloc_debug. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-07-27 02:24:46 +00:00
debug library @file{libc_malloc_debug} to be preloaded using the
Move malloc hooks into a compat DSO Remove all malloc hook uses from core malloc functions and move it into a new library libc_malloc_debug.so. With this, the hooks now no longer have any effect on the core library. libc_malloc_debug.so is a malloc interposer that needs to be preloaded to get hooks functionality back so that the debugging features that depend on the hooks, i.e. malloc-check, mcheck and mtrace work again. Without the preloaded DSO these debugging features will be nops. These features will be ported away from hooks in subsequent patches. Similarly, legacy applications that need hooks functionality need to preload libc_malloc_debug.so. The symbols exported by libc_malloc_debug.so are maintained at exactly the same version as libc.so. Finally, static binaries will no longer be able to use malloc debugging features since they cannot preload the debugging DSO. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 13:07:59 +00:00
@code{LD_PRELOAD} environment variable.
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
glibc.malloc.check: Fix nit in documentation The tunable will not work with *any* non-zero tunable value since its list of allowed values is 0-3. Fix the documentation to reflect that. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2021-07-07 01:32:13 +00:00
Setting this tunable to a non-zero value less than 4 enables a special (less
Improve documentation for malloc etc. (BZ#27719) Cover key corner cases (e.g., whether errno is set) that are well settled in glibc, fix some examples to avoid integer overflow, and update some other dated examples (code needed for K&R C, e.g.). * manual/charset.texi (Non-reentrant String Conversion): * manual/filesys.texi (Symbolic Links): * manual/memory.texi (Allocating Cleared Space): * manual/socket.texi (Host Names): * manual/string.texi (Concatenating Strings): * manual/users.texi (Setting Groups): Use reallocarray instead of realloc, to avoid integer overflow issues. * manual/filesys.texi (Scanning Directory Content): * manual/memory.texi (The GNU Allocator, Hooks for Malloc): * manual/tunables.texi: Use code font for 'malloc' instead of roman font. (Symbolic Links): Don't assume readlink return value fits in 'int'. * manual/memory.texi (Memory Allocation and C, Basic Allocation) (Malloc Examples, Alloca Example): * manual/stdio.texi (Formatted Output Functions): * manual/string.texi (Concatenating Strings, Collation Functions): Omit pointer casts that are needed only in ancient K&R C. * manual/memory.texi (Basic Allocation): Say that malloc sets errno on failure. Say "convert" rather than "cast", since casts are no longer needed. * manual/memory.texi (Basic Allocation): * manual/string.texi (Concatenating Strings): In examples, use C99 declarations after statements for brevity. * manual/memory.texi (Malloc Examples): Add portability notes for malloc (0), errno setting, and PTRDIFF_MAX. (Changing Block Size): Say that realloc (p, 0) acts like (p ? (free (p), NULL) : malloc (0)). Add xreallocarray example, since other examples can use it. Add portability notes for realloc (0, 0), realloc (p, 0), PTRDIFF_MAX, and improve notes for reallocating to the same size. (Allocating Cleared Space): Reword now-confusing discussion about replacement, and xref "Replacing malloc". * manual/stdio.texi (Formatted Output Functions): Don't assume message size fits in 'int'. * manual/string.texi (Concatenating Strings): Fix undefined behavior involving arithmetic on a freed pointer.
2021-04-12 02:06:00 +00:00
efficient) memory allocator for the @code{malloc} family of functions that is
malloc: Abort on heap corruption, without a backtrace [BZ #21754] The stack trace printing caused deadlocks and has been itself been targeted by code execution exploits.
2017-08-30 14:39:41 +00:00
designed to be tolerant against simple errors such as double calls of
free with the same argument, or overruns of a single byte (off-by-one
bugs). Not all such errors can be protected against, however, and memory
leaks can result. Any detected heap corruption results in immediate
termination of the process.
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
Like @env{MALLOC_CHECK_}, @code{glibc.malloc.check} has a problem in that it
diverges from normal program behavior by writing to @code{stderr}, which could
by exploited in SUID and SGID binaries. Therefore, @code{glibc.malloc.check}
elf: Remove /etc/suid-debug support Since malloc debug support moved to a different library (libc_malloc_debug.so), the glibc.malloc.check requires preloading the debug library to enable it. It means that suid-debug support has not been working since 2.34. To restore its support, it would require to add additional information and parsing to where to find libc_malloc_debug.so. It is one thing less that might change AT_SECURE binaries' behavior due to environment configurations. Checked on x86_64-linux-gnu. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-11-06 20:25:34 +00:00
is disabled by default for SUID and SGID binaries.
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
@end deftp
@deftp Tunable glibc.malloc.top_pad
This tunable supersedes the @env{MALLOC_TOP_PAD_} environment variable and is
identical in features.
This tunable determines the amount of extra memory in bytes to obtain from the
system when any of the arenas need to be extended. It also specifies the
number of bytes to retain when shrinking any of the arenas. This provides the
necessary hysteresis in heap size such that excessive amounts of system calls
can be avoided.
malloc: Correct the documentation of the top_pad default DEFAULT_TOP_PAD is defined as 131072 in sysdeps/generic/malloc-machine.h.
2022-08-04 08:24:47 +00:00
The default value of this tunable is @samp{131072} (128 KB).
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
@end deftp
@deftp Tunable glibc.malloc.perturb
This tunable supersedes the @env{MALLOC_PERTURB_} environment variable and is
identical in features.
If set to a non-zero value, memory blocks are initialized with values depending
on some low order bits of this tunable when they are allocated (except when
Improve documentation for malloc etc. (BZ#27719) Cover key corner cases (e.g., whether errno is set) that are well settled in glibc, fix some examples to avoid integer overflow, and update some other dated examples (code needed for K&R C, e.g.). * manual/charset.texi (Non-reentrant String Conversion): * manual/filesys.texi (Symbolic Links): * manual/memory.texi (Allocating Cleared Space): * manual/socket.texi (Host Names): * manual/string.texi (Concatenating Strings): * manual/users.texi (Setting Groups): Use reallocarray instead of realloc, to avoid integer overflow issues. * manual/filesys.texi (Scanning Directory Content): * manual/memory.texi (The GNU Allocator, Hooks for Malloc): * manual/tunables.texi: Use code font for 'malloc' instead of roman font. (Symbolic Links): Don't assume readlink return value fits in 'int'. * manual/memory.texi (Memory Allocation and C, Basic Allocation) (Malloc Examples, Alloca Example): * manual/stdio.texi (Formatted Output Functions): * manual/string.texi (Concatenating Strings, Collation Functions): Omit pointer casts that are needed only in ancient K&R C. * manual/memory.texi (Basic Allocation): Say that malloc sets errno on failure. Say "convert" rather than "cast", since casts are no longer needed. * manual/memory.texi (Basic Allocation): * manual/string.texi (Concatenating Strings): In examples, use C99 declarations after statements for brevity. * manual/memory.texi (Malloc Examples): Add portability notes for malloc (0), errno setting, and PTRDIFF_MAX. (Changing Block Size): Say that realloc (p, 0) acts like (p ? (free (p), NULL) : malloc (0)). Add xreallocarray example, since other examples can use it. Add portability notes for realloc (0, 0), realloc (p, 0), PTRDIFF_MAX, and improve notes for reallocating to the same size. (Allocating Cleared Space): Reword now-confusing discussion about replacement, and xref "Replacing malloc". * manual/stdio.texi (Formatted Output Functions): Don't assume message size fits in 'int'. * manual/string.texi (Concatenating Strings): Fix undefined behavior involving arithmetic on a freed pointer.
2021-04-12 02:06:00 +00:00
allocated by @code{calloc}) and freed. This can be used to debug the use of
User manual documentation for tunables Create a new node for tunables documentation and add notes for the malloc tunables. * manual/tunables.texi: New chapter. * manual/Makefile (chapters): Add it. * manual/probes.texi (@node): Point to the Tunables chapter.
2016-12-31 18:04:42 +00:00
uninitialized or freed heap memory. Note that this option does not guarantee
that the freed block will have any specific values. It only guarantees that the
content the block had before it was freed will be overwritten.
The default value of this tunable is @samp{0}.
@end deftp
@deftp Tunable glibc.malloc.mmap_threshold
This tunable supersedes the @env{MALLOC_MMAP_THRESHOLD_} environment variable
and is identical in features.
When this tunable is set, all chunks larger than this value in bytes are
allocated outside the normal heap, using the @code{mmap} system call. This way
it is guaranteed that the memory for these chunks can be returned to the system
on @code{free}. Note that requests smaller than this threshold might still be
allocated via @code{mmap}.
If this tunable is not set, the default value is set to @samp{131072} bytes and
the threshold is adjusted dynamically to suit the allocation patterns of the
program. If the tunable is set, the dynamic adjustment is disabled and the
value is set as static.
@end deftp
@deftp Tunable glibc.malloc.trim_threshold
This tunable supersedes the @env{MALLOC_TRIM_THRESHOLD_} environment variable
and is identical in features.
The value of this tunable is the minimum size (in bytes) of the top-most,
releasable chunk in an arena that will trigger a system call in order to return
memory to the system from that arena.
If this tunable is not set, the default value is set as 128 KB and the
threshold is adjusted dynamically to suit the allocation patterns of the
program. If the tunable is set, the dynamic adjustment is disabled and the
value is set as static.
@end deftp
@deftp Tunable glibc.malloc.mmap_max
This tunable supersedes the @env{MALLOC_MMAP_MAX_} environment variable and is
identical in features.
The value of this tunable is maximum number of chunks to allocate with
@code{mmap}. Setting this to zero disables all use of @code{mmap}.
The default value of this tunable is @samp{65536}.
@end deftp
@deftp Tunable glibc.malloc.arena_test
This tunable supersedes the @env{MALLOC_ARENA_TEST} environment variable and is
identical in features.
The @code{glibc.malloc.arena_test} tunable specifies the number of arenas that
can be created before the test on the limit to the number of arenas is
conducted. The value is ignored if @code{glibc.malloc.arena_max} is set.
The default value of this tunable is 2 for 32-bit systems and 8 for 64-bit
systems.
@end deftp
@deftp Tunable glibc.malloc.arena_max
This tunable supersedes the @env{MALLOC_ARENA_MAX} environment variable and is
identical in features.
This tunable sets the number of arenas to use in a process regardless of the
number of cores in the system.
The default value of this tunable is @code{0}, meaning that the limit on the
number of arenas is determined by the number of CPU cores online. For 32-bit
systems the limit is twice the number of cores online and on 64-bit systems, it
is 8 times the number of cores online.
@end deftp
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
Add per-thread cache to malloc * config.make.in: Enable experimental malloc option. * configure.ac: Likewise. * configure: Regenerate. * manual/install.texi: Document it. * INSTALL: Regenerate. * malloc/Makefile: Likewise. * malloc/malloc.c: Add per-thread cache (tcache). (tcache_put): New. (tcache_get): New. (tcache_thread_freeres): New. (tcache_init): New. (__libc_malloc): Use cached chunks if available. (__libc_free): Initialize tcache if needed. (__libc_realloc): Likewise. (__libc_calloc): Likewise. (_int_malloc): Prefill tcache when appropriate. (_int_free): Likewise. (do_set_tcache_max): New. (do_set_tcache_count): New. (do_set_tcache_unsorted_limit): New. * manual/probes.texi: Document new probes. * malloc/arena.c: Add new tcache tunables. * elf/dl-tunables.list: Likewise. * manual/tunables.texi: Document them. * NEWS: Mention the per-thread cache.
2017-07-06 17:37:30 +00:00
@deftp Tunable glibc.malloc.tcache_max
The maximum size of a request (in bytes) which may be met via the
per-thread cache. The default (and maximum) value is 1032 bytes on
64-bit systems and 516 bytes on 32-bit systems.
@end deftp
@deftp Tunable glibc.malloc.tcache_count
The maximum number of chunks of each size to cache. The default is 7.
Small tcache improvements Change the tcache->counts[] entries to uint16_t - this removes the limit set by char and allows a larger tcache. Remove a few redundant asserts. bench-malloc-thread with 4 threads is ~15% faster on Cortex-A72. Reviewed-by: DJ Delorie <dj@redhat.com> * malloc/malloc.c (MAX_TCACHE_COUNT): Increase to UINT16_MAX. (tcache_put): Remove redundant assert. (tcache_get): Remove redundant asserts. (__libc_malloc): Check tcache count is not zero. * manual/tunables.texi (glibc.malloc.tcache_count): Update maximum.
2019-05-17 17:16:20 +00:00
The upper limit is 65535. If set to zero, the per-thread cache is effectively
Fix tcache count maximum (BZ #24531) The tcache counts[] array is a char, which has a very small range and thus may overflow. When setting tcache_count tunable, there is no overflow check. However the tunable must not be larger than the maximum value of the tcache counts[] array, otherwise it can overflow when filling the tcache. [BZ #24531] * malloc/malloc.c (MAX_TCACHE_COUNT): New define. (do_set_tcache_count): Only update if count is small enough. * manual/tunables.texi (glibc.malloc.tcache_count): Document max value.
2019-05-10 15:38:21 +00:00
disabled.
Add per-thread cache to malloc * config.make.in: Enable experimental malloc option. * configure.ac: Likewise. * configure: Regenerate. * manual/install.texi: Document it. * INSTALL: Regenerate. * malloc/Makefile: Likewise. * malloc/malloc.c: Add per-thread cache (tcache). (tcache_put): New. (tcache_get): New. (tcache_thread_freeres): New. (tcache_init): New. (__libc_malloc): Use cached chunks if available. (__libc_free): Initialize tcache if needed. (__libc_realloc): Likewise. (__libc_calloc): Likewise. (_int_malloc): Prefill tcache when appropriate. (_int_free): Likewise. (do_set_tcache_max): New. (do_set_tcache_count): New. (do_set_tcache_unsorted_limit): New. * manual/probes.texi: Document new probes. * malloc/arena.c: Add new tcache tunables. * elf/dl-tunables.list: Likewise. * manual/tunables.texi: Document them. * NEWS: Mention the per-thread cache.
2017-07-06 17:37:30 +00:00
The approximate maximum overhead of the per-thread cache is thus equal
to the number of bins times the chunk count in each bin times the size
of each chunk. With defaults, the approximate maximum overhead of the
per-thread cache is approximately 236 KB on 64-bit systems and 118 KB
on 32-bit systems.
@end deftp
@deftp Tunable glibc.malloc.tcache_unsorted_limit
When the user requests memory and the request cannot be met via the
per-thread cache, the arenas are used to meet the request. At this
time, additional chunks will be moved from existing arena lists to
pre-fill the corresponding cache. While copies from the fastbins,
smallbins, and regular bins are bounded and predictable due to the bin
sizes, copies from the unsorted bin are not bounded, and incur
additional time penalties as they need to be sorted as they're
scanned. To make scanning the unsorted list more predictable and
bounded, the user may set this tunable to limit the number of chunks
that are scanned from the unsorted list while searching for chunks to
pre-fill the per-thread cache with. The default, or when set to zero,
is no limit.
* manual/tunables.texi: Add missing @end deftp.
2017-07-06 23:54:13 +00:00
@end deftp
Add per-thread cache to malloc * config.make.in: Enable experimental malloc option. * configure.ac: Likewise. * configure: Regenerate. * manual/install.texi: Document it. * INSTALL: Regenerate. * malloc/Makefile: Likewise. * malloc/malloc.c: Add per-thread cache (tcache). (tcache_put): New. (tcache_get): New. (tcache_thread_freeres): New. (tcache_init): New. (__libc_malloc): Use cached chunks if available. (__libc_free): Initialize tcache if needed. (__libc_realloc): Likewise. (__libc_calloc): Likewise. (_int_malloc): Prefill tcache when appropriate. (_int_free): Likewise. (do_set_tcache_max): New. (do_set_tcache_count): New. (do_set_tcache_unsorted_limit): New. * manual/probes.texi: Document new probes. * malloc/arena.c: Add new tcache tunables. * elf/dl-tunables.list: Likewise. * manual/tunables.texi: Document them. * NEWS: Mention the per-thread cache.
2017-07-06 17:37:30 +00:00
Add glibc.malloc.mxfast tunable * elf/dl-tunables.list: Add glibc.malloc.mxfast. * manual/tunables.texi: Document it. * malloc/malloc.c (do_set_mxfast): New. (__libc_mallopt): Call it. * malloc/arena.c: Add mxfast tunable. * malloc/tst-mxfast.c: New. * malloc/Makefile: Add it. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2019-08-08 23:09:43 +00:00
@deftp Tunable glibc.malloc.mxfast
Improve documentation for malloc etc. (BZ#27719) Cover key corner cases (e.g., whether errno is set) that are well settled in glibc, fix some examples to avoid integer overflow, and update some other dated examples (code needed for K&R C, e.g.). * manual/charset.texi (Non-reentrant String Conversion): * manual/filesys.texi (Symbolic Links): * manual/memory.texi (Allocating Cleared Space): * manual/socket.texi (Host Names): * manual/string.texi (Concatenating Strings): * manual/users.texi (Setting Groups): Use reallocarray instead of realloc, to avoid integer overflow issues. * manual/filesys.texi (Scanning Directory Content): * manual/memory.texi (The GNU Allocator, Hooks for Malloc): * manual/tunables.texi: Use code font for 'malloc' instead of roman font. (Symbolic Links): Don't assume readlink return value fits in 'int'. * manual/memory.texi (Memory Allocation and C, Basic Allocation) (Malloc Examples, Alloca Example): * manual/stdio.texi (Formatted Output Functions): * manual/string.texi (Concatenating Strings, Collation Functions): Omit pointer casts that are needed only in ancient K&R C. * manual/memory.texi (Basic Allocation): Say that malloc sets errno on failure. Say "convert" rather than "cast", since casts are no longer needed. * manual/memory.texi (Basic Allocation): * manual/string.texi (Concatenating Strings): In examples, use C99 declarations after statements for brevity. * manual/memory.texi (Malloc Examples): Add portability notes for malloc (0), errno setting, and PTRDIFF_MAX. (Changing Block Size): Say that realloc (p, 0) acts like (p ? (free (p), NULL) : malloc (0)). Add xreallocarray example, since other examples can use it. Add portability notes for realloc (0, 0), realloc (p, 0), PTRDIFF_MAX, and improve notes for reallocating to the same size. (Allocating Cleared Space): Reword now-confusing discussion about replacement, and xref "Replacing malloc". * manual/stdio.texi (Formatted Output Functions): Don't assume message size fits in 'int'. * manual/string.texi (Concatenating Strings): Fix undefined behavior involving arithmetic on a freed pointer.
2021-04-12 02:06:00 +00:00
One of the optimizations @code{malloc} uses is to maintain a series of ``fast
Add glibc.malloc.mxfast tunable * elf/dl-tunables.list: Add glibc.malloc.mxfast. * manual/tunables.texi: Document it. * malloc/malloc.c (do_set_mxfast): New. (__libc_mallopt): Call it. * malloc/arena.c: Add mxfast tunable. * malloc/tst-mxfast.c: New. * malloc/Makefile: Add it. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2019-08-08 23:09:43 +00:00
bins'' that hold chunks up to a specific size. The default and
maximum size which may be held this way is 80 bytes on 32-bit systems
or 160 bytes on 64-bit systems. Applications which value size over
speed may choose to reduce the size of requests which are serviced
from fast bins with this tunable. Note that the value specified
Improve documentation for malloc etc. (BZ#27719) Cover key corner cases (e.g., whether errno is set) that are well settled in glibc, fix some examples to avoid integer overflow, and update some other dated examples (code needed for K&R C, e.g.). * manual/charset.texi (Non-reentrant String Conversion): * manual/filesys.texi (Symbolic Links): * manual/memory.texi (Allocating Cleared Space): * manual/socket.texi (Host Names): * manual/string.texi (Concatenating Strings): * manual/users.texi (Setting Groups): Use reallocarray instead of realloc, to avoid integer overflow issues. * manual/filesys.texi (Scanning Directory Content): * manual/memory.texi (The GNU Allocator, Hooks for Malloc): * manual/tunables.texi: Use code font for 'malloc' instead of roman font. (Symbolic Links): Don't assume readlink return value fits in 'int'. * manual/memory.texi (Memory Allocation and C, Basic Allocation) (Malloc Examples, Alloca Example): * manual/stdio.texi (Formatted Output Functions): * manual/string.texi (Concatenating Strings, Collation Functions): Omit pointer casts that are needed only in ancient K&R C. * manual/memory.texi (Basic Allocation): Say that malloc sets errno on failure. Say "convert" rather than "cast", since casts are no longer needed. * manual/memory.texi (Basic Allocation): * manual/string.texi (Concatenating Strings): In examples, use C99 declarations after statements for brevity. * manual/memory.texi (Malloc Examples): Add portability notes for malloc (0), errno setting, and PTRDIFF_MAX. (Changing Block Size): Say that realloc (p, 0) acts like (p ? (free (p), NULL) : malloc (0)). Add xreallocarray example, since other examples can use it. Add portability notes for realloc (0, 0), realloc (p, 0), PTRDIFF_MAX, and improve notes for reallocating to the same size. (Allocating Cleared Space): Reword now-confusing discussion about replacement, and xref "Replacing malloc". * manual/stdio.texi (Formatted Output Functions): Don't assume message size fits in 'int'. * manual/string.texi (Concatenating Strings): Fix undefined behavior involving arithmetic on a freed pointer.
2021-04-12 02:06:00 +00:00
includes @code{malloc}'s internal overhead, which is normally the size of one
Add glibc.malloc.mxfast tunable * elf/dl-tunables.list: Add glibc.malloc.mxfast. * manual/tunables.texi: Document it. * malloc/malloc.c (do_set_mxfast): New. (__libc_mallopt): Call it. * malloc/arena.c: Add mxfast tunable. * malloc/tst-mxfast.c: New. * malloc/Makefile: Add it. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2019-08-08 23:09:43 +00:00
pointer, so add 4 on 32-bit systems or 8 on 64-bit systems to the size
passed to @code{malloc} for the largest bin size to enable.
@end deftp
malloc: Add madvise support for Transparent Huge Pages Linux Transparent Huge Pages (THP) current supports three different states: 'never', 'madvise', and 'always'. The 'never' is self-explanatory and 'always' will enable THP for all anonymous pages. However, 'madvise' is still the default for some system and for such case THP will be only used if the memory range is explicity advertise by the program through a madvise(MADV_HUGEPAGE) call. To enable it a new tunable is provided, 'glibc.malloc.hugetlb', where setting to a value diffent than 0 enables the madvise call. This patch issues the madvise(MADV_HUGEPAGE) call after a successful mmap() call at sysmalloc() with sizes larger than the default huge page size. The madvise() call is disable is system does not support THP or if it has the mode set to "never" and on Linux only support one page size for THP, even if the architecture supports multiple sizes. To test is a new rule is added tests-malloc-hugetlb1, which run the addes tests with the required GLIBC_TUNABLE setting. Checked on x86_64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
2021-08-13 11:36:29 +00:00
@deftp Tunable glibc.malloc.hugetlb
This tunable controls the usage of Huge Pages on @code{malloc} calls. The
default value is @code{0}, which disables any additional support on
@code{malloc}.
Setting its value to @code{1} enables the use of @code{madvise} with
@code{MADV_HUGEPAGE} after memory allocation with @code{mmap}. It is enabled
only if the system supports Transparent Huge Page (currently only on Linux).
malloc: Add Huge Page support for mmap With the morecore hook removed, there is not easy way to provide huge pages support on with glibc allocator without resorting to transparent huge pages. And some users and programs do prefer to use the huge pages directly instead of THP for multiple reasons: no splitting, re-merging by the VM, no TLB shootdowns for running processes, fast allocation from the reserve pool, no competition with the rest of the processes unlike THP, no swapping all, etc. This patch extends the 'glibc.malloc.hugetlb' tunable: the value '2' means to use huge pages directly with the system default size, while a positive value means and specific page size that is matched against the supported ones by the system. Currently only memory allocated on sysmalloc() is handled, the arenas still uses the default system page size. To test is a new rule is added tests-malloc-hugetlb2, which run the addes tests with the required GLIBC_TUNABLE setting. On systems without a reserved huge pages pool, is just stress the mmap(MAP_HUGETLB) allocation failure. To improve test coverage it is required to create a pool with some allocated pages. Checked on x86_64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
2021-08-16 18:08:27 +00:00
Setting its value to @code{2} enables the use of Huge Page directly with
@code{mmap} with the use of @code{MAP_HUGETLB} flag. The huge page size
to use will be the default one provided by the system. A value larger than
@code{2} specifies huge page size, which will be matched against the system
supported ones. If provided value is invalid, @code{MAP_HUGETLB} will not
be used.
malloc: Add madvise support for Transparent Huge Pages Linux Transparent Huge Pages (THP) current supports three different states: 'never', 'madvise', and 'always'. The 'never' is self-explanatory and 'always' will enable THP for all anonymous pages. However, 'madvise' is still the default for some system and for such case THP will be only used if the memory range is explicity advertise by the program through a madvise(MADV_HUGEPAGE) call. To enable it a new tunable is provided, 'glibc.malloc.hugetlb', where setting to a value diffent than 0 enables the madvise call. This patch issues the madvise(MADV_HUGEPAGE) call after a successful mmap() call at sysmalloc() with sizes larger than the default huge page size. The madvise() call is disable is system does not support THP or if it has the mode set to "never" and on Linux only support one page size for THP, even if the architecture supports multiple sizes. To test is a new rule is added tests-malloc-hugetlb1, which run the addes tests with the required GLIBC_TUNABLE setting. Checked on x86_64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
2021-08-13 11:36:29 +00:00
@end deftp
rtld: Add rtld.nns tunable for the number of supported namespaces TLS_STATIC_SURPLUS is 1664 bytes currently which is not enough to support DL_NNS (== 16) number of dynamic link namespaces, if we assume 192 bytes of TLS are reserved for libc use and 144 bytes are reserved for other system libraries that use IE TLS. A new tunable is introduced to control the number of supported namespaces and to adjust the surplus static TLS size as follows: surplus_tls = 192 * (rtld.nns-1) + 144 * rtld.nns + 512 The default is rtld.nns == 4 and then the surplus TLS size is the same as before, so the behaviour is unchanged by default. If an application creates more namespaces than the rtld.nns setting allows, then it is not guaranteed to work, but the limit is not checked. So existing usage will continue to work, but in the future if an application creates more than 4 dynamic link namespaces then the tunable will need to be set. In this patch DL_NNS is a fixed value and provides a maximum to the rtld.nns setting. Static linking used fixed 2048 bytes surplus TLS, this is changed so the same contract is used as for dynamic linking. With static linking DL_NNS == 1 so rtld.nns tunable is forced to 1, so by default the surplus TLS is reduced to 144 + 512 = 656 bytes. This change is not expected to cause problems. Tested on aarch64-linux-gnu and x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-06-09 08:57:28 +00:00
@node Dynamic Linking Tunables
@section Dynamic Linking Tunables
@cindex dynamic linking tunables
@cindex rtld tunables
@deftp {Tunable namespace} glibc.rtld
Dynamic linker behavior can be modified by setting the
following tunables in the @code{rtld} namespace:
@end deftp
@deftp Tunable glibc.rtld.nns
Sets the number of supported dynamic link namespaces (see @code{dlmopen}).
Currently this limit can be set between 1 and 16 inclusive, the default is 4.
Each link namespace consumes some memory in all thread, and thus raising the
limit will increase the amount of memory each thread uses. Raising the limit
rtld: Account static TLS surplus for audit modules The new static TLS surplus size computation is surplus_tls = 192 * (nns-1) + 144 * nns + 512 where nns is controlled via the rtld.nns tunable. This commit accounts audit modules too so nns = rtld.nns + audit modules. rtld.nns should only include the namespaces required by the application, namespaces for audit modules are accounted on top of that so audit modules don't use up the static TLS that is reserved for the application. This allows loading many audit modules without tuning rtld.nns or using up static TLS, and it fixes FAIL: elf/tst-auditmany Note that DL_NNS is currently a hard upper limit for nns, and if rtld.nns + audit modules go over the limit that's a fatal error. By default rtld.nns is 4 which allows 12 audit modules. Counting the audit modules is based on existing audit string parsing code, we cannot use GLRO(dl_naudit) before the modules are actually loaded.
2020-07-07 09:49:11 +00:00
is useful when your application uses more than 4 dynamic link namespaces as
created by @code{dlmopen} with an lmid argument of @code{LM_ID_NEWLM}.
Dynamic linker audit modules are loaded in their own dynamic link namespaces,
but they are not accounted for in @code{glibc.rtld.nns}. They implicitly
increase the per-thread memory usage as necessary, so this tunable does
not need to be changed to allow many audit modules e.g. via @env{LD_AUDIT}.
rtld: Add rtld.nns tunable for the number of supported namespaces TLS_STATIC_SURPLUS is 1664 bytes currently which is not enough to support DL_NNS (== 16) number of dynamic link namespaces, if we assume 192 bytes of TLS are reserved for libc use and 144 bytes are reserved for other system libraries that use IE TLS. A new tunable is introduced to control the number of supported namespaces and to adjust the surplus static TLS size as follows: surplus_tls = 192 * (rtld.nns-1) + 144 * rtld.nns + 512 The default is rtld.nns == 4 and then the surplus TLS size is the same as before, so the behaviour is unchanged by default. If an application creates more namespaces than the rtld.nns setting allows, then it is not guaranteed to work, but the limit is not checked. So existing usage will continue to work, but in the future if an application creates more than 4 dynamic link namespaces then the tunable will need to be set. In this patch DL_NNS is a fixed value and provides a maximum to the rtld.nns setting. Static linking used fixed 2048 bytes surplus TLS, this is changed so the same contract is used as for dynamic linking. With static linking DL_NNS == 1 so rtld.nns tunable is forced to 1, so by default the surplus TLS is reduced to 144 + 512 = 656 bytes. This change is not expected to cause problems. Tested on aarch64-linux-gnu and x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-06-09 08:57:28 +00:00
@end deftp
rtld: Avoid using up static TLS surplus for optimizations [BZ #25051] On some targets static TLS surplus area can be used opportunistically for dynamically loaded modules such that the TLS access then becomes faster (TLSDESC and powerpc TLS optimization). However we don't want all surplus TLS to be used for this optimization because dynamically loaded modules with initial-exec model TLS can only use surplus TLS. The new contract for surplus static TLS use is: - libc.so can have up to 192 bytes of IE TLS, - other system libraries together can have up to 144 bytes of IE TLS. - Some "optional" static TLS is available for opportunistic use. The optional TLS is now tunable: rtld.optional_static_tls, so users can directly affect the allocated static TLS size. (Note that module unloading with dlclose does not reclaim static TLS. After the optional TLS runs out, TLS access is no longer optimized to use static TLS.) The default setting of rtld.optional_static_tls is 512 so the surplus TLS is 3*192 + 4*144 + 512 = 1664 by default, the same as before. Fixes BZ #25051. Tested on aarch64-linux-gnu and x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-06-10 12:40:40 +00:00
@deftp Tunable glibc.rtld.optional_static_tls
Sets the amount of surplus static TLS in bytes to allocate at program
startup. Every thread created allocates this amount of specified surplus
static TLS. This is a minimum value and additional space may be allocated
for internal purposes including alignment. Optional static TLS is used for
optimizing dynamic TLS access for platforms that support such optimizations
e.g. TLS descriptors or optimized TLS access for POWER (@code{DT_PPC64_OPT}
and @code{DT_PPC_OPT}). In order to make the best use of such optimizations
the value should be as many bytes as would be required to hold all TLS
variables in all dynamic loaded shared libraries. The value cannot be known
by the dynamic loader because it doesn't know the expected set of shared
libraries which will be loaded. The existing static TLS space cannot be
changed once allocated at process startup. The default allocation of
optional static TLS is 512 bytes and is allocated in every thread.
@end deftp
elf: Fix slow DSO sorting behavior in dynamic loader (BZ #17645) This second patch contains the actual implementation of a new sorting algorithm for shared objects in the dynamic loader, which solves the slow behavior that the current "old" algorithm falls into when the DSO set contains circular dependencies. The new algorithm implemented here is simply depth-first search (DFS) to obtain the Reverse-Post Order (RPO) sequence, a topological sort. A new l_visited:1 bitfield is added to struct link_map to more elegantly facilitate such a search. The DFS algorithm is applied to the input maps[nmap-1] backwards towards maps[0]. This has the effect of a more "shallow" recursion depth in general since the input is in BFS. Also, when combined with the natural order of processing l_initfini[] at each node, this creates a resulting output sorting closer to the intuitive "left-to-right" order in most cases. Another notable implementation adjustment related to this _dl_sort_maps change is the removing of two char arrays 'used' and 'done' in _dl_close_worker to represent two per-map attributes. This has been changed to simply use two new bit-fields l_map_used:1, l_map_done:1 added to struct link_map. This also allows discarding the clunky 'used' array sorting that _dl_sort_maps had to sometimes do along the way. Tunable support for switching between different sorting algorithms at runtime is also added. A new tunable 'glibc.rtld.dynamic_sort' with current valid values 1 (old algorithm) and 2 (new DFS algorithm) has been added. At time of commit of this patch, the default setting is 1 (old algorithm). Signed-off-by: Chung-Lin Tang <cltang@codesourcery.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2021-10-21 13:41:22 +00:00
@deftp Tunable glibc.rtld.dynamic_sort
Sets the algorithm to use for DSO sorting, valid values are @samp{1} and
@samp{2}. For value of @samp{1}, an older O(n^3) algorithm is used, which is
long time tested, but may have performance issues when dependencies between
shared objects contain cycles due to circular dependencies. When set to the
value of @samp{2}, a different algorithm is used, which implements a
topological sort through depth-first search, and does not exhibit the
performance issues of @samp{1}.
elf: Use new dependency sorting algorithm by default The default has to change eventually, and there are no known failures that require a delay. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2021-12-14 11:37:44 +00:00
The default value of this tunable is @samp{2}.
elf: Fix slow DSO sorting behavior in dynamic loader (BZ #17645) This second patch contains the actual implementation of a new sorting algorithm for shared objects in the dynamic loader, which solves the slow behavior that the current "old" algorithm falls into when the DSO set contains circular dependencies. The new algorithm implemented here is simply depth-first search (DFS) to obtain the Reverse-Post Order (RPO) sequence, a topological sort. A new l_visited:1 bitfield is added to struct link_map to more elegantly facilitate such a search. The DFS algorithm is applied to the input maps[nmap-1] backwards towards maps[0]. This has the effect of a more "shallow" recursion depth in general since the input is in BFS. Also, when combined with the natural order of processing l_initfini[] at each node, this creates a resulting output sorting closer to the intuitive "left-to-right" order in most cases. Another notable implementation adjustment related to this _dl_sort_maps change is the removing of two char arrays 'used' and 'done' in _dl_close_worker to represent two per-map attributes. This has been changed to simply use two new bit-fields l_map_used:1, l_map_done:1 added to struct link_map. This also allows discarding the clunky 'used' array sorting that _dl_sort_maps had to sometimes do along the way. Tunable support for switching between different sorting algorithms at runtime is also added. A new tunable 'glibc.rtld.dynamic_sort' with current valid values 1 (old algorithm) and 2 (new DFS algorithm) has been added. At time of commit of this patch, the default setting is 1 (old algorithm). Signed-off-by: Chung-Lin Tang <cltang@codesourcery.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2021-10-21 13:41:22 +00:00
@end deftp
rtld: Avoid using up static TLS surplus for optimizations [BZ #25051] On some targets static TLS surplus area can be used opportunistically for dynamically loaded modules such that the TLS access then becomes faster (TLSDESC and powerpc TLS optimization). However we don't want all surplus TLS to be used for this optimization because dynamically loaded modules with initial-exec model TLS can only use surplus TLS. The new contract for surplus static TLS use is: - libc.so can have up to 192 bytes of IE TLS, - other system libraries together can have up to 144 bytes of IE TLS. - Some "optional" static TLS is available for opportunistic use. The optional TLS is now tunable: rtld.optional_static_tls, so users can directly affect the allocated static TLS size. (Note that module unloading with dlclose does not reclaim static TLS. After the optional TLS runs out, TLS access is no longer optimized to use static TLS.) The default setting of rtld.optional_static_tls is 512 so the surplus TLS is 3*192 + 4*144 + 512 = 1664 by default, the same as before. Fixes BZ #25051. Tested on aarch64-linux-gnu and x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-06-10 12:40:40 +00:00
Add elision tunables This patch adds several new tunables to control the behavior of elision on supported platforms[1]. Since elision now depends on tunables, we should always *compile* with elision enabled, and leave the code disabled, but available for runtime selection. This gives us *much* better compile-time testing of the existing code to avoid bit-rot[2]. Tested on ppc, ppc64, ppc64le, s390x and x86_64. [1] This part of the patch was initially proposed by Paul Murphy but was "staled" because the framework have changed since the patch was originally proposed: https://patchwork.sourceware.org/patch/10342/ [2] This part of the patch was inititally proposed as a RFC by Carlos O'Donnell. Make sense to me integrate this on the patch: https://sourceware.org/ml/libc-alpha/2017-05/msg00335.html * elf/dl-tunables.list: Add elision parameters. * manual/tunables.texi: Add entries about elision tunable. * sysdeps/unix/sysv/linux/powerpc/elision-conf.c: Add callback functions to dynamically enable/disable elision. Add multiple callbacks functions to set elision parameters. Deleted __libc_enable_secure check. * sysdeps/unix/sysv/linux/s390/elision-conf.c: Likewise. * sysdeps/unix/sysv/linux/x86/elision-conf.c: Likewise. * configure: Regenerated. * configure.ac: Option enable_lock_elision was deleted. * config.h.in: ENABLE_LOCK_ELISION flag was deleted. * config.make.in: Remove references to enable_lock_elision. * manual/install.texi: Elision configure option was removed. * INSTALL: Regenerated to remove enable_lock_elision. * nptl/Makefile: Disable elision so it can verify error case for destroying a mutex. * sysdeps/powerpc/nptl/elide.h: Cleanup ENABLE_LOCK_ELISION check. Deleted macros for the case when ENABLE_LOCK_ELISION was not defined. * sysdeps/s390/configure: Regenerated. * sysdeps/s390/configure.ac: Remove references to enable_lock_elision.. * nptl/tst-mutex8.c: Deleted all #ifndef ENABLE_LOCK_ELISION from the test. * sysdeps/powerpc/powerpc32/sysdep.h: Deleted all ENABLE_LOCK_ELISION checks. * sysdeps/powerpc/powerpc64/sysdep.h: Likewise. * sysdeps/powerpc/sysdep.h: Likewise. * sysdeps/s390/nptl/bits/pthreadtypes-arch.h: Likewise. * sysdeps/unix/sysv/linux/powerpc/force-elision.h: Likewise. * sysdeps/unix/sysv/linux/s390/elision-conf.h: Likewise. * sysdeps/unix/sysv/linux/s390/force-elision.h: Likewise. * sysdeps/unix/sysv/linux/s390/lowlevellock.h: Likewise. * sysdeps/unix/sysv/linux/s390/Makefile: Remove references to enable-lock-elision. Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
2017-12-05 16:24:14 +00:00
@node Elision Tunables
@section Elision Tunables
@cindex elision tunables
@cindex tunables, elision
@deftp {Tunable namespace} glibc.elision
Contended locks are usually slow and can lead to performance and scalability
issues in multithread code. Lock elision will use memory transactions to under
certain conditions, to elide locks and improve performance.
Elision behavior can be modified by setting the following tunables in
the @code{elision} namespace:
@end deftp
@deftp Tunable glibc.elision.enable
The @code{glibc.elision.enable} tunable enables lock elision if the feature is
supported by the hardware. If elision is not supported by the hardware this
tunable has no effect.
Elision tunables are supported for 64-bit Intel, IBM POWER, and z System
architectures.
@end deftp
@deftp Tunable glibc.elision.skip_lock_busy
The @code{glibc.elision.skip_lock_busy} tunable sets how many times to use a
non-transactional lock after a transactional failure has occurred because the
lock is already acquired. Expressed in number of lock acquisition attempts.
The default value of this tunable is @samp{3}.
@end deftp
@deftp Tunable glibc.elision.skip_lock_internal_abort
The @code{glibc.elision.skip_lock_internal_abort} tunable sets how many times
the thread should avoid using elision if a transaction aborted for any reason
other than a different thread's memory accesses. Expressed in number of lock
acquisition attempts.
The default value of this tunable is @samp{3}.
@end deftp
@deftp Tunable glibc.elision.skip_lock_after_retries
The @code{glibc.elision.skip_lock_after_retries} tunable sets how many times
to try to elide a lock with transactions, that only failed due to a different
thread's memory accesses, before falling back to regular lock.
Expressed in number of lock elision attempts.
This tunable is supported only on IBM POWER, and z System architectures.
The default value of this tunable is @samp{3}.
@end deftp
@deftp Tunable glibc.elision.tries
The @code{glibc.elision.tries} sets how many times to retry elision if there is
chance for the transaction to finish execution e.g., it wasn't
aborted due to the lock being already acquired. If elision is not supported
by the hardware this tunable is set to @samp{0} to avoid retries.
The default value of this tunable is @samp{3}.
@end deftp
@deftp Tunable glibc.elision.skip_trylock_internal_abort
The @code{glibc.elision.skip_trylock_internal_abort} tunable sets how many
times the thread should avoid trying the lock if a transaction aborted due to
reasons other than a different thread's memory accesses. Expressed in number
of try lock attempts.
The default value of this tunable is @samp{3}.
@end deftp
Mutex: Add pthread mutex tunables This patch does not have any functionality change, we only provide a spin count tunes for pthread adaptive spin mutex. The tunable glibc.pthread.mutex_spin_count tunes can be used by system administrator to squeeze system performance according to different hardware capabilities and workload characteristics. The maximum value of spin count is limited to 32767 to avoid the overflow of mutex->__data.__spins variable with the possible type of short in pthread_mutex_lock (). The default value of spin count is set to 100 with the reference to the previous number of times of spinning via trylock. This value would be architecture-specific and can be tuned with kinds of benchmarks to fit most cases in future. I would extend my appreciation sincerely to H.J.Lu for his help to refine this patch series. * manual/tunables.texi (POSIX Thread Tunables): New node. * nptl/Makefile (libpthread-routines): Add pthread_mutex_conf. * nptl/nptl-init.c: Include pthread_mutex_conf.h (__pthread_initialize_minimal_internal) [HAVE_TUNABLES]: Call __pthread_tunables_init. * nptl/pthreadP.h (MAX_ADAPTIVE_COUNT): Remove. (max_adaptive_count): Define. * nptl/pthread_mutex_conf.c: New file. * nptl/pthread_mutex_conf.h: New file. * sysdeps/generic/adaptive_spin_count.h: New file. * sysdeps/nptl/dl-tunables.list: New file. * nptl/pthread_mutex_lock.c (__pthread_mutex_lock): Use max_adaptive_count () not MAX_ADAPTIVE_COUNT. * nptl/pthread_mutex_timedlock.c (__pthrad_mutex_timedlock): Likewise. Suggested-by: Andi Kleen <andi.kleen@intel.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com> Signed-off-by: Kemi.wang <kemi.wang@intel.com>
2018-12-01 15:03:33 +00:00
@node POSIX Thread Tunables
@section POSIX Thread Tunables
@cindex pthread mutex tunables
@cindex thread mutex tunables
@cindex mutex tunables
@cindex tunables thread mutex
@deftp {Tunable namespace} glibc.pthread
The behavior of POSIX threads can be tuned to gain performance improvements
according to specific hardware capabilities and workload characteristics by
setting the following tunables in the @code{pthread} namespace:
@end deftp
@deftp Tunable glibc.pthread.mutex_spin_count
The @code{glibc.pthread.mutex_spin_count} tunable sets the maximum number of times
a thread should spin on the lock before calling into the kernel to block.
Adaptive spin is used for mutexes initialized with the
@code{PTHREAD_MUTEX_ADAPTIVE_NP} GNU extension. It affects both
@code{pthread_mutex_lock} and @code{pthread_mutex_timedlock}.
The thread spins until either the maximum spin count is reached or the lock
is acquired.
The default value of this tunable is @samp{100}.
@end deftp
nptl: Add glibc.pthread.stack_cache_size tunable The valgrind/helgrind test suite needs a way to make stack dealloction more prompt, and this feature seems to be generally useful. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-06-28 13:48:58 +00:00
@deftp Tunable glibc.pthread.stack_cache_size
This tunable configures the maximum size of the stack cache. Once the
stack cache exceeds this size, unused thread stacks are returned to
the kernel, to bring the cache size below this limit.
The value is measured in bytes. The default is @samp{41943040}
Fix misspellings in manual/ -- BZ 25337
2023-05-27 16:41:44 +00:00
(forty mibibytes).
nptl: Add glibc.pthread.stack_cache_size tunable The valgrind/helgrind test suite needs a way to make stack dealloction more prompt, and this feature seems to be generally useful. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-06-28 13:48:58 +00:00
@end deftp
nptl: Add glibc.pthread.rseq tunable to control rseq registration This tunable allows applications to register the rseq area instead of glibc. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com> Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-12-09 08:49:32 +00:00
@deftp Tunable glibc.pthread.rseq
The @code{glibc.pthread.rseq} tunable can be set to @samp{0}, to disable
restartable sequences support in @theglibc{}. This enables applications
to perform direct restartable sequence registration with the kernel.
The default is @samp{1}, which means that @theglibc{} performs
registration on behalf of the application.
Restartable sequences are a Linux-specific extension.
@end deftp
Created tunable to force small pages on stack allocation. Created tunable glibc.pthread.stack_hugetlb to control when hugepages can be used for stack allocation. In case THP are enabled and glibc.pthread.stack_hugetlb is set to 0, glibc will madvise the kernel not to use allow hugepages for stack allocations. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2023-04-14 15:12:20 +00:00
@deftp Tunable glibc.pthread.stack_hugetlb
This tunable controls whether to use Huge Pages in the stacks created by
@code{pthread_create}. This tunable only affects the stacks created by
@theglibc{}, it has no effect on stack assigned with
@code{pthread_attr_setstack}.
The default is @samp{1} where the system default value is used. Setting
its value to @code{0} enables the use of @code{madvise} with
@code{MADV_NOHUGEPAGE} after stack creation with @code{mmap}.
This is a memory utilization optimization, since internal glibc setup of either
the thread descriptor and the guard page might force the kernel to move the
thread stack originally backup by Huge Pages to default pages.
@end deftp
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
@node Hardware Capability Tunables
@section Hardware Capability Tunables
@cindex hardware capability tunables
@cindex hwcap tunables
@cindex tunables, hwcap
x86: Rename glibc.tune.ifunc to glibc.tune.hwcaps Rename glibc.tune.ifunc to glibc.tune.hwcaps and move it to sysdeps/x86/dl-tunables.list since it is x86 specicifc. Also change type of data_cache_size, data_cache_size and non_temporal_threshold to unsigned long int to match size_t. Remove usage DEFAULT_STRLEN from cpu-tunables.c. * elf/dl-tunables.list (glibc.tune.ifunc): Removed. * sysdeps/x86/dl-tunables.list (glibc.tune.hwcaps): New. Remove security_level on all fields. * manual/tunables.texi: Replace ifunc with hwcaps. * sysdeps/x86/cpu-features.c (TUNABLE_CALLBACK (set_ifunc)): Renamed to .. (TUNABLE_CALLBACK (set_hwcaps)): This. (init_cpu_features): Updated. * sysdeps/x86/cpu-features.h (cpu_features): Change type of data_cache_size, data_cache_size and non_temporal_threshold to unsigned long int. * sysdeps/x86/cpu-tunables.c (DEFAULT_STRLEN): Removed. (TUNABLE_CALLBACK (set_ifunc)): Renamed to ... (TUNABLE_CALLBACK (set_hwcaps)): This. Update comments. Don't use DEFAULT_STRLEN.
2017-06-21 17:20:24 +00:00
@cindex hwcaps tunables
@cindex tunables, hwcaps
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
@cindex data_cache_size tunables
@cindex tunables, data_cache_size
@cindex shared_cache_size tunables
@cindex tunables, shared_cache_size
@cindex non_temporal_threshold tunables
@cindex tunables, non_temporal_threshold
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp {Tunable namespace} glibc.cpu
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
Behavior of @theglibc{} can be tuned to assume specific hardware capabilities
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
by setting the following tunables in the @code{cpu} namespace:
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.hwcap_mask
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
This tunable supersedes the @env{LD_HWCAP_MASK} environment variable and is
identical in features.
manual: Fix spelling of "Auxiliary."
2018-01-23 19:40:44 +00:00
The @code{AT_HWCAP} key in the Auxiliary Vector specifies instruction set
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
extensions available in the processor at runtime for some architectures. The
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@code{glibc.cpu.hwcap_mask} tunable allows the user to mask out those
tunables: Add LD_HWCAP_MASK to tunables Add LD_HWCAP_MASK to tunables in preparation of it being removed from rtld.c. This allows us to read LD_HWCAP_MASK much earlier so that it can influence IFUNC resolution in aarch64. This patch does not actually do anything other than read the LD_HWCAP_MASK variable and add the tunables way to set the LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable. In a follow-up patch, the _dl_hwcap_mask will be replaced with glibc.tune.hwcap_mask to complete the transition. * elf/dl-tunables.list: Add glibc.tune.hwcap_mask. * scripts/gen-tunables.awk: Include dl-procinfo.h. * manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-04-17 04:30:35 +00:00
capabilities at runtime, thus disabling use of those extensions.
@end deftp
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.hwcaps
The @code{glibc.cpu.hwcaps=-xxx,yyy,-zzz...} tunable allows the user to
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
enable CPU/ARCH feature @code{yyy}, disable CPU/ARCH feature @code{xxx}
and @code{zzz} where the feature name is case-sensitive and has to match
manual: fix reference to source file
2022-05-31 14:21:32 +00:00
the ones in @code{sysdeps/x86/include/cpu-features.h}.
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
S390: Influence hwcaps/stfle via GLIBC_TUNABLES. This patch enables the option to influence hwcaps and stfle bits used by the s390 specific ifunc-resolvers. The currently x86-specific tunable glibc.cpu.hwcaps is also used on s390x to achieve the task. In addition the user can also set a CPU arch-level like z13 instead of single HWCAP and STFLE features. Note that the tunable only handles the features which are really used in the IFUNC-resolvers. All others are ignored as the values are only used inside glibc. Thus we can influence: - HWCAP_S390_VXRS (z13) - HWCAP_S390_VXRS_EXT (z14) - HWCAP_S390_VXRS_EXT2 (z15) - STFLE_MIE3 (z15) The influenced hwcap/stfle-bits are stored in the s390-specific cpu_features struct which also contains reserved fields for future usage. The ifunc-resolvers and users of stfle bits are adjusted to use the information from cpu_features struct. On 31bit, the ELF_MACHINE_IRELATIVE macro is now also defined. Otherwise the new ifunc-resolvers segfaults as they depend on the not yet processed_rtld_global_ro@GLIBC_PRIVATE relocation.
2023-02-02 13:57:50 +00:00
On s390x, the supported HWCAP and STFLE features can be found in
@code{sysdeps/s390/cpu-features.c}. In addition the user can also set
a CPU arch-level like @code{z13} instead of single HWCAP and STFLE features.
PowerPC: Influence cpu/arch hwcap features via GLIBC_TUNABLES This patch enables the option to influence hwcaps used by PowerPC. The environment variable, GLIBC_TUNABLES=glibc.cpu.hwcaps=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature xxx and zzz, where the feature name is case-sensitive and has to match the ones mentioned in the file{sysdeps/powerpc/dl-procinfo.c}. Note that the hwcap tunables only used in the IFUNC selection. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2023-08-01 12:41:17 +00:00
On powerpc, the supported HWCAP and HWCAP2 features can be found in
@code{sysdeps/powerpc/dl-procinfo.c}.
This tunable is specific to i386, x86-64, s390x and powerpc.
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.cached_memopt
The @code{glibc.cpu.cached_memopt=[0|1]} tunable allows the user to
powerpc: POWER8 memcpy optimization for cached memory On POWER8, unaligned memory accesses to cached memory has little impact on performance as opposed to its ancestors. It is disabled by default and will only be available when the tunable glibc.tune.cached_memopt is set to 1. __memcpy_power8_cached __memcpy_power7 ============================================================ max-size=4096: 33325.70 ( 12.65%) 38153.00 max-size=8192: 32878.20 ( 11.17%) 37012.30 max-size=16384: 33782.20 ( 11.61%) 38219.20 max-size=32768: 33296.20 ( 11.30%) 37538.30 max-size=65536: 33765.60 ( 10.53%) 37738.40 * manual/tunables.texi (Hardware Capability Tunables): Document glibc.tune.cached_memopt. * sysdeps/powerpc/cpu-features.c: New file. * sysdeps/powerpc/cpu-features.h: New file. * sysdeps/powerpc/dl-procinfo.c [!IS_IN(ldconfig)]: Add _dl_powerpc_cpu_features. * sysdeps/powerpc/dl-tunables.list: New file. * sysdeps/powerpc/ldsodefs.h: Include cpu-features.h. * sysdeps/powerpc/powerpc32/power4/multiarch/init-arch.h (INIT_ARCH): Initialize use_aligned_memopt. * sysdeps/powerpc/powerpc64/dl-machine.h [defined(SHARED && IS_IN(rtld))]: Restrict dl_platform_init availability and initialize CPU features used by tunables. * sysdeps/powerpc/powerpc64/multiarch/Makefile (sysdep_routines): Add memcpy-power8-cached. * sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c: Add __memcpy_power8_cached. * sysdeps/powerpc/powerpc64/multiarch/memcpy.c: Likewise. * sysdeps/powerpc/powerpc64/multiarch/memcpy-power8-cached.S: New file. Reviewed-by: Rajalakshmi Srinivasaraghavan <raji@linux.vnet.ibm.com>
2017-12-11 19:39:42 +00:00
enable optimizations recommended for cacheable memory. If set to
@code{1}, @theglibc{} assumes that the process memory image consists
of cacheable (non-device) memory only. The default, @code{0},
indicates that the process may use device memory.
This tunable is specific to powerpc, powerpc64 and powerpc64le.
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.name
The @code{glibc.cpu.name=xxx} tunable allows the user to tell @theglibc{} to
tunables, aarch64: New tunable to override cpu Add a new tunable (glibc.tune.cpu) to override CPU identification on aarch64. This is useful in two cases: one where it is desirable to pretend to be another CPU for purposes of testing or because routines written for that CPU are beneficial for specific workloads and second where the underlying kernel does not support emulation of MRS to get the MIDR of the CPU. * elf/dl-tunables.h (tunable_is_name): Move from... * elf/dl-tunables.c (is_name): ... here. (parse_tunables, __tunables_init): Adjust. * manual/tunables.texi: Document glibc.tune.cpu. * sysdeps/aarch64/dl-tunables.list: New file. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (struct cpu_list): New type. (cpu_list): New list of CPU names and their MIDR. (get_midr_from_mcpu): New function. (init_cpu_features): Override MIDR if necessary.
2017-06-30 17:28:39 +00:00
assume that the CPU is @code{xxx} where xxx may have one of these values:
AArch64: Remove Falkor memcpy The latest implementations of memcpy are actually faster than the Falkor implementations [1], so remove the falkor/phecda ifuncs for memcpy and the now unused IS_FALKOR/IS_PHECDA defines. [1] https://sourceware.org/pipermail/libc-alpha/2022-December/144227.html Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
2023-10-26 16:30:36 +00:00
@code{generic}, @code{thunderxt88}, @code{thunderx2t99},
aarch64: Added optimized memcpy and memmove for A64FX This patch optimizes the performance of memcpy/memmove for A64FX [1] which implements ARMv8-A SVE and has L1 64KB cache per core and L2 8MB cache per NUMA node. The performance optimization makes use of Scalable Vector Register with several techniques such as loop unrolling, memory access alignment, cache zero fill, and software pipelining. SVE assembler code for memcpy/memmove is implemented as Vector Length Agnostic code so theoretically it can be run on any SOC which supports ARMv8-A SVE standard. We confirmed that all testcases have been passed by running 'make check' and 'make xcheck' not only on A64FX but also on ThunderX2. And also we confirmed that the SVE 512 bit vector register performance is roughly 4 times better than Advanced SIMD 128 bit register and 8 times better than scalar 64 bit register by running 'make bench'. [1] https://github.com/fujitsu/A64FX Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com> Reviewed-by: Szabolcs Nagy <Szabolcs.Nagy@arm.com>
2021-05-27 07:42:35 +00:00
@code{thunderx2t99p1}, @code{ares}, @code{emag}, @code{kunpeng},
@code{a64fx}.
tunables, aarch64: New tunable to override cpu Add a new tunable (glibc.tune.cpu) to override CPU identification on aarch64. This is useful in two cases: one where it is desirable to pretend to be another CPU for purposes of testing or because routines written for that CPU are beneficial for specific workloads and second where the underlying kernel does not support emulation of MRS to get the MIDR of the CPU. * elf/dl-tunables.h (tunable_is_name): Move from... * elf/dl-tunables.c (is_name): ... here. (parse_tunables, __tunables_init): Adjust. * manual/tunables.texi: Document glibc.tune.cpu. * sysdeps/aarch64/dl-tunables.list: New file. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (struct cpu_list): New type. (cpu_list): New list of CPU names and their MIDR. (get_midr_from_mcpu): New function. (init_cpu_features): Override MIDR if necessary.
2017-06-30 17:28:39 +00:00
This tunable is specific to aarch64.
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.x86_data_cache_size
The @code{glibc.cpu.x86_data_cache_size} tunable allows the user to set
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
data cache size in bytes for use in memory and string routines.
This tunable is specific to i386 and x86-64.
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.x86_shared_cache_size
The @code{glibc.cpu.x86_shared_cache_size} tunable allows the user to
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
set shared cache size in bytes for use in memory and string routines.
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.x86_non_temporal_threshold
The @code{glibc.cpu.x86_non_temporal_threshold} tunable allows the user
Reversing calculation of __x86_shared_non_temporal_threshold The __x86_shared_non_temporal_threshold determines when memcpy on x86 uses non_temporal stores to avoid pushing other data out of the last level cache. This patch proposes to revert the calculation change made by H.J. Lu's patch of June 2, 2017. H.J. Lu's patch selected a threshold suitable for a single thread getting maximum performance. It was tuned using the single threaded large memcpy micro benchmark on an 8 core processor. The last change changes the threshold from using 3/4 of one thread's share of the cache to using 3/4 of the entire cache of a multi-threaded system before switching to non-temporal stores. Multi-threaded systems with more than a few threads are server-class and typically have many active threads. If one thread consumes 3/4 of the available cache for all threads, it will cause other active threads to have data removed from the cache. Two examples show the range of the effect. John McCalpin's widely parallel Stream benchmark, which runs in parallel and fetches data sequentially, saw a 20% slowdown with this patch on an internal system test of 128 threads. This regression was discovered when comparing OL8 performance to OL7. An example that compares normal stores to non-temporal stores may be found at https://vgatherps.github.io/2018-09-02-nontemporal/. A simple test shows performance loss of 400 to 500% due to a failure to use nontemporal stores. These performance losses are most likely to occur when the system load is heaviest and good performance is critical. The tunable x86_non_temporal_threshold can be used to override the default for the knowledgable user who really wants maximum cache allocation to a single thread in a multi-threaded system. The manual entry for the tunable has been expanded to provide more information about its purpose. modified: sysdeps/x86/cacheinfo.c modified: manual/tunables.texi
2020-09-28 20:11:28 +00:00
to set threshold in bytes for non temporal store. Non temporal stores
give a hint to the hardware to move data directly to memory without
displacing other data from the cache. This tunable is used by some
platforms to determine when to use non temporal stores in operations
like memmove and memcpy.
tunables: Add IFUNC selection and cache sizes The current IFUNC selection is based on microbenchmarks in glibc. It should give the best performance for most workloads. But other choices may have better performance for a particular workload or on the hardware which wasn't available at the selection was made. The environment variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz, where the feature name is case-sensitive and has to match the ones in cpu-features.h. It can be used by glibc developers to override the IFUNC selection to tune for a new processor or improve performance for a particular workload. It isn't intended for normal end users. NOTE: the IFUNC selection may change over time. Please check all multiarch implementations when experimenting. Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is provided to set threshold to use non temporal store to NUMBER, GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set shared cache size. * elf/dl-tunables.list (tune): Add ifunc, x86_non_temporal_threshold, x86_data_cache_size and x86_shared_cache_size. * manual/tunables.texi: Document glibc.tune.ifunc, glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size and glibc.tune.x86_non_temporal_threshold. * sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86/cacheinfo.c (init_cacheinfo): Check and get data cache size, shared cache size and non temporal threshold from cpu_features. * sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE): New. [HAVE_TUNABLES] Include <unistd.h>. [HAVE_TUNABLES] Include <elf/dl-tunables.h>. [HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise. [HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set IFUNC selection, data cache size, shared cache size and non temporal threshold. * sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size, shared_cache_size and non_temporal_threshold.
2017-06-20 15:33:29 +00:00
This tunable is specific to i386 and x86-64.
@end deftp
x86/CET: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk * manual/tunables.texi: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk.
2018-07-18 18:34:35 +00:00
x86: Add thresholds for "rep movsb/stosb" to tunables Add x86_rep_movsb_threshold and x86_rep_stosb_threshold to tunables to update thresholds for "rep movsb" and "rep stosb" at run-time. Note that the user specified threshold for "rep movsb" smaller than the minimum threshold will be ignored. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-07-06 18:48:09 +00:00
@deftp Tunable glibc.cpu.x86_rep_movsb_threshold
The @code{glibc.cpu.x86_rep_movsb_threshold} tunable allows the user to
set threshold in bytes to start using "rep movsb". The value must be
greater than zero, and currently defaults to 2048 bytes.
This tunable is specific to i386 and x86-64.
@end deftp
@deftp Tunable glibc.cpu.x86_rep_stosb_threshold
The @code{glibc.cpu.x86_rep_stosb_threshold} tunable allows the user to
set threshold in bytes to start using "rep stosb". The value must be
greater than zero, and currently defaults to 2048 bytes.
This tunable is specific to i386 and x86-64.
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.x86_ibt
The @code{glibc.cpu.x86_ibt} tunable allows the user to control how
x86/CET: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk * manual/tunables.texi: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk.
2018-07-18 18:34:35 +00:00
indirect branch tracking (IBT) should be enabled. Accepted values are
@code{on}, @code{off}, and @code{permissive}. @code{on} always turns
on IBT regardless of whether IBT is enabled in the executable and its
dependent shared libraries. @code{off} always turns off IBT regardless
of whether IBT is enabled in the executable and its dependent shared
libraries. @code{permissive} is the same as the default which disables
IBT on non-CET executables and shared libraries.
This tunable is specific to i386 and x86-64.
@end deftp
Rename the glibc.tune namespace to glibc.cpu The glibc.tune namespace is vaguely named since it is a 'tunable', so give it a more specific name that describes what it refers to. Rename the tunable namespace to 'cpu' to more accurately reflect what it encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since glibc.cpu.cpu is weird. * NEWS: Mention the change. * elf/dl-tunables.list: Rename tune namespace to cpu. * sysdeps/powerpc/dl-tunables.list: Likewise. * sysdeps/x86/dl-tunables.list: Likewise. * sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to cpu.name. * elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust. * elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise. * manual/README.tunables: Likewise. * manual/tunables.texi: Likewise. * sysdeps/powerpc/cpu-features.c: Likewise. * sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features): Likewise. * sysdeps/x86/cpu-features.c: Likewise. * sysdeps/x86/cpu-features.h: Likewise. * sysdeps/x86/cpu-tunables.c: Likewise. * sysdeps/x86_64/Makefile: Likewise. * sysdeps/x86/dl-cet.c: Likewise. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-08-02 18:19:19 +00:00
@deftp Tunable glibc.cpu.x86_shstk
The @code{glibc.cpu.x86_shstk} tunable allows the user to control how
x86/CET: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk * manual/tunables.texi: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk.
2018-07-18 18:34:35 +00:00
the shadow stack (SHSTK) should be enabled. Accepted values are
@code{on}, @code{off}, and @code{permissive}. @code{on} always turns on
SHSTK regardless of whether SHSTK is enabled in the executable and its
dependent shared libraries. @code{off} always turns off SHSTK regardless
of whether SHSTK is enabled in the executable and its dependent shared
libraries. @code{permissive} changes how dlopen works on non-CET shared
libraries. By default, when SHSTK is enabled, dlopening a non-CET shared
library returns an error. With @code{permissive}, it turns off SHSTK
instead.
This tunable is specific to i386 and x86-64.
@end deftp
elf: Add a tunable to control use of tagged memory Add a new glibc tunable: mem.tagging. This is a decimal constant in the range 0-255 but used as a bit-field. Bit 0 enables use of tagged memory in the malloc family of functions. Bit 1 enables precise faulting of tag failure on platforms where this can be controlled. Other bits are currently unused, but if set will cause memory tag checking for the current process to be enabled in the kernel.
2020-12-21 15:03:03 +00:00
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
@deftp Tunable glibc.cpu.prefer_map_32bit_exec
tunables.texi: Change \code{1} to @code{1} Update 317f1c0a8a x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656]
2023-02-23 04:04:26 +00:00
When this tunable is set to @code{1}, shared libraries of non-setuid
x86-64: Add glibc.cpu.prefer_map_32bit_exec [BZ #28656] Crossing 2GB boundaries with indirect calls and jumps can use more branch prediction resources on Intel Golden Cove CPU (see the "Misprediction for Branches >2GB" section in Intel 64 and IA-32 Architectures Optimization Reference Manual.) There is visible performance improvement on workloads with many PLT calls when executable and shared libraries are mmapped below 2GB. Add the Prefer_MAP_32BIT_EXEC bit so that mmap will try to map executable or denywrite pages in shared libraries with MAP_32BIT first. NB: Prefer_MAP_32BIT_EXEC reduces bits available for address space layout randomization (ASLR), which is always disabled for SUID programs and can only be enabled by the tunable, glibc.cpu.prefer_map_32bit_exec, or the environment variable, LD_PREFER_MAP_32BIT_EXEC. This works only between shared libraries or between shared libraries and executables with addresses below 2GB. PIEs are usually loaded at a random address above 4GB by the kernel.
2023-01-26 16:26:18 +00:00
programs will be loaded below 2GB with MAP_32BIT.
Note that the @env{LD_PREFER_MAP_32BIT_EXEC} environment is an alias of
this tunable.
This tunable is specific to 64-bit x86-64.
@end deftp
elf: Add ELF_DYNAMIC_AFTER_RELOC to rewrite PLT Add ELF_DYNAMIC_AFTER_RELOC to allow target specific processing after relocation. For x86-64, add #define DT_X86_64_PLT (DT_LOPROC + 0) #define DT_X86_64_PLTSZ (DT_LOPROC + 1) #define DT_X86_64_PLTENT (DT_LOPROC + 3) 1. DT_X86_64_PLT: The address of the procedure linkage table. 2. DT_X86_64_PLTSZ: The total size, in bytes, of the procedure linkage table. 3. DT_X86_64_PLTENT: The size, in bytes, of a procedure linkage table entry. With the r_addend field of the R_X86_64_JUMP_SLOT relocation set to the memory offset of the indirect branch instruction. Define ELF_DYNAMIC_AFTER_RELOC for x86-64 to rewrite the PLT section with direct branch after relocation when the lazy binding is disabled. PLT rewrite is disabled by default since SELinux may disallow modifying code pages and ld.so can't detect it in all cases. Use $ export GLIBC_TUNABLES=glibc.cpu.plt_rewrite=1 to enable PLT rewrite with 32-bit direct jump at run-time or $ export GLIBC_TUNABLES=glibc.cpu.plt_rewrite=2 to enable PLT rewrite with 32-bit direct jump and on APX processors with 64-bit absolute jump at run-time. Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
2024-01-05 04:19:39 +00:00
@deftp Tunable glibc.cpu.plt_rewrite
When this tunable is set to @code{1}, the dynamic linker will rewrite
the PLT section with 32-bit direct jump. When it is set to @code{2},
the dynamic linker will rewrite the PLT section with 32-bit direct
jump and on APX processors with 64-bit absolute jump.
This tunable is specific to x86-64 and effective only when the lazy
binding is disabled.
@end deftp
elf: Add a tunable to control use of tagged memory Add a new glibc tunable: mem.tagging. This is a decimal constant in the range 0-255 but used as a bit-field. Bit 0 enables use of tagged memory in the malloc family of functions. Bit 1 enables precise faulting of tag failure on platforms where this can be controlled. Other bits are currently unused, but if set will cause memory tag checking for the current process to be enabled in the kernel.
2020-12-21 15:03:03 +00:00
@node Memory Related Tunables
@section Memory Related Tunables
@cindex memory related tunables
@deftp {Tunable namespace} glibc.mem
This tunable namespace supports operations that affect the way @theglibc{}
and the process manage memory.
@end deftp
@deftp Tunable glibc.mem.tagging
If the hardware supports memory tagging, this tunable can be used to
control the way @theglibc{} uses this feature. At present this is only
Fix misspellings in manual/ -- BZ 25337
2023-05-27 16:41:44 +00:00
supported on AArch64 systems with the MTE extension; it is ignored for
elf: Add a tunable to control use of tagged memory Add a new glibc tunable: mem.tagging. This is a decimal constant in the range 0-255 but used as a bit-field. Bit 0 enables use of tagged memory in the malloc family of functions. Bit 1 enables precise faulting of tag failure on platforms where this can be controlled. Other bits are currently unused, but if set will cause memory tag checking for the current process to be enabled in the kernel.
2020-12-21 15:03:03 +00:00
all other systems.
This tunable takes a value between 0 and 255 and acts as a bitmask
that enables various capabilities.
Improve documentation for malloc etc. (BZ#27719) Cover key corner cases (e.g., whether errno is set) that are well settled in glibc, fix some examples to avoid integer overflow, and update some other dated examples (code needed for K&R C, e.g.). * manual/charset.texi (Non-reentrant String Conversion): * manual/filesys.texi (Symbolic Links): * manual/memory.texi (Allocating Cleared Space): * manual/socket.texi (Host Names): * manual/string.texi (Concatenating Strings): * manual/users.texi (Setting Groups): Use reallocarray instead of realloc, to avoid integer overflow issues. * manual/filesys.texi (Scanning Directory Content): * manual/memory.texi (The GNU Allocator, Hooks for Malloc): * manual/tunables.texi: Use code font for 'malloc' instead of roman font. (Symbolic Links): Don't assume readlink return value fits in 'int'. * manual/memory.texi (Memory Allocation and C, Basic Allocation) (Malloc Examples, Alloca Example): * manual/stdio.texi (Formatted Output Functions): * manual/string.texi (Concatenating Strings, Collation Functions): Omit pointer casts that are needed only in ancient K&R C. * manual/memory.texi (Basic Allocation): Say that malloc sets errno on failure. Say "convert" rather than "cast", since casts are no longer needed. * manual/memory.texi (Basic Allocation): * manual/string.texi (Concatenating Strings): In examples, use C99 declarations after statements for brevity. * manual/memory.texi (Malloc Examples): Add portability notes for malloc (0), errno setting, and PTRDIFF_MAX. (Changing Block Size): Say that realloc (p, 0) acts like (p ? (free (p), NULL) : malloc (0)). Add xreallocarray example, since other examples can use it. Add portability notes for realloc (0, 0), realloc (p, 0), PTRDIFF_MAX, and improve notes for reallocating to the same size. (Allocating Cleared Space): Reword now-confusing discussion about replacement, and xref "Replacing malloc". * manual/stdio.texi (Formatted Output Functions): Don't assume message size fits in 'int'. * manual/string.texi (Concatenating Strings): Fix undefined behavior involving arithmetic on a freed pointer.
2021-04-12 02:06:00 +00:00
Bit 0 (the least significant bit) causes the @code{malloc}
subsystem to allocate
elf: Add a tunable to control use of tagged memory Add a new glibc tunable: mem.tagging. This is a decimal constant in the range 0-255 but used as a bit-field. Bit 0 enables use of tagged memory in the malloc family of functions. Bit 1 enables precise faulting of tag failure on platforms where this can be controlled. Other bits are currently unused, but if set will cause memory tag checking for the current process to be enabled in the kernel.
2020-12-21 15:03:03 +00:00
tagged memory, with each allocation being assigned a random tag.
Bit 1 enables precise faulting mode for tag violations on systems that
support deferred tag violation reporting. This may cause programs
to run more slowly.
AArch64: Add asymmetric faulting mode for tag violations in mem.tagging tunable The new asymmetric mode is available when HWCAP2_MTE3 is set (support is available), bit2 is set in the tunable (user request per application), and the system is configured such that the asymmetric mode is preferred over sync or async (per-cpu system-wide setting). Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
2022-06-27 18:00:50 +00:00
Bit 2 enables either precise or deferred faulting mode for tag violations
whichever is preferred by the system.
elf: Add a tunable to control use of tagged memory Add a new glibc tunable: mem.tagging. This is a decimal constant in the range 0-255 but used as a bit-field. Bit 0 enables use of tagged memory in the malloc family of functions. Bit 1 enables precise faulting of tag failure on platforms where this can be controlled. Other bits are currently unused, but if set will cause memory tag checking for the current process to be enabled in the kernel.
2020-12-21 15:03:03 +00:00
Other bits are currently reserved.
@Theglibc{} startup code will automatically enable memory tagging
support in the kernel if this tunable has any non-zero value.
The default value is @samp{0}, which disables all memory tagging.
@end deftp
gmon: improve mcount overflow handling [BZ# 27576] When mcount overflows, no gmon.out file is generated, but no message is printed to the user, leaving the user with no idea why, and thinking maybe there is some bug - which is how BZ 27576 ended up being logged. Print a message to stderr in this case so the user knows what is going on. As a comment in sys/gmon.h acknowledges, the hardcoded MAXARCS value is too small for some large applications, including the test case in that BZ. Rather than increase it, add tunables to enable MINARCS and MAXARCS to be overridden at runtime (glibc.gmon.minarcs and glibc.gmon.maxarcs). So if a user gets the mcount overflow error, they can try increasing maxarcs (they might need to increase minarcs too if the heuristic is wrong in their case.) Note setting minarcs/maxarcs too large can cause monstartup to fail with an out of memory error. If you set them large enough, it can cause an integer overflow in calculating the buffer size. I haven't done anything to defend against that - it would not generally be a security vulnerability, since these tunables will be ignored in suid/sgid programs (due to the SXID_ERASE default), and if you can set GLIBC_TUNABLES in the environment of a process, you can take it over anyway (LD_PRELOAD, LD_LIBRARY_PATH, etc). I thought about modifying the code of monstartup to defend against integer overflows, but doing so is complicated, and I realise the existing code is susceptible to them even prior to this change (e.g. try passing a pathologically large highpc argument to monstartup), so I decided just to leave that possibility in-place. Add a test case which demonstrates mcount overflow and the tunables. Document the new tunables in the manual. Signed-off-by: Simon Kissane <skissane@gmail.com> Reviewed-by: DJ Delorie <dj@redhat.com>
2023-02-11 09:12:13 +00:00
elf: Add glibc.mem.decorate_maps tunable The PR_SET_VMA_ANON_NAME support is only enabled through a configurable kernel switch, mainly because assigning a name to a anonymous virtual memory area might prevent that area from being merged with adjacent virtual memory areas. For instance, with the following code: void *p1 = mmap (NULL, 1024 * 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); void *p2 = mmap (p1 + (1024 * 4096), 1024 * 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); The kernel will potentially merge both mappings resulting in only one segment of size 0x800000. If the segment is names with PR_SET_VMA_ANON_NAME with different names, it results in two mappings. Although this will unlikely be an issue for pthread stacks and malloc arenas (since for pthread stacks the guard page will result in a PROT_NONE segment, similar to the alignment requirement for the arena block), it still might prevent the mmap memory allocated for detail malloc. There is also another potential scalability issue, where the prctl requires to take the mmap global lock which is still not fully fixed in Linux [1] (for pthread stacks and arenas, it is mitigated by the stack cached and the arena reuse). So this patch disables anonymous mapping annotations as default and add a new tunable, glibc.mem.decorate_maps, can be used to enable it. [1] https://lwn.net/Articles/906852/ Checked on x86_64-linux-gnu and aarch64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-01 12:56:11 +00:00
@deftp Tunable glibc.mem.decorate_maps
If the kernel supports naming anonymous virtual memory areas (since
Linux version 5.17, although not always enabled by some kernel
configurations), this tunable can be used to control whether
@theglibc{} decorates the underlying memory obtained from operating
system with a string describing its usage (for instance, on the thread
stack created by @code{ptthread_create} or memory allocated by
@code{malloc}).
The process mappings can be obtained by reading the @code{/proc/<pid>maps}
(with @code{pid} being either the @dfn{process ID} or @code{self} for the
process own mapping).
This tunable takes a value of 0 and 1, where 1 enables the feature.
The default value is @samp{0}, which disables the decoration.
@end deftp
gmon: improve mcount overflow handling [BZ# 27576] When mcount overflows, no gmon.out file is generated, but no message is printed to the user, leaving the user with no idea why, and thinking maybe there is some bug - which is how BZ 27576 ended up being logged. Print a message to stderr in this case so the user knows what is going on. As a comment in sys/gmon.h acknowledges, the hardcoded MAXARCS value is too small for some large applications, including the test case in that BZ. Rather than increase it, add tunables to enable MINARCS and MAXARCS to be overridden at runtime (glibc.gmon.minarcs and glibc.gmon.maxarcs). So if a user gets the mcount overflow error, they can try increasing maxarcs (they might need to increase minarcs too if the heuristic is wrong in their case.) Note setting minarcs/maxarcs too large can cause monstartup to fail with an out of memory error. If you set them large enough, it can cause an integer overflow in calculating the buffer size. I haven't done anything to defend against that - it would not generally be a security vulnerability, since these tunables will be ignored in suid/sgid programs (due to the SXID_ERASE default), and if you can set GLIBC_TUNABLES in the environment of a process, you can take it over anyway (LD_PRELOAD, LD_LIBRARY_PATH, etc). I thought about modifying the code of monstartup to defend against integer overflows, but doing so is complicated, and I realise the existing code is susceptible to them even prior to this change (e.g. try passing a pathologically large highpc argument to monstartup), so I decided just to leave that possibility in-place. Add a test case which demonstrates mcount overflow and the tunables. Document the new tunables in the manual. Signed-off-by: Simon Kissane <skissane@gmail.com> Reviewed-by: DJ Delorie <dj@redhat.com>
2023-02-11 09:12:13 +00:00
@node gmon Tunables
@section gmon Tunables
@cindex gmon tunables
@deftp {Tunable namespace} glibc.gmon
This tunable namespace affects the behaviour of the gmon profiler.
gmon is a component of @theglibc{} which is normally used in
conjunction with gprof.
When GCC compiles a program with the @code{-pg} option, it instruments
the program with calls to the @code{mcount} function, to record the
program's call graph. At program startup, a memory buffer is allocated
to store this call graph; the size of the buffer is calculated using a
heuristic based on code size. If during execution, the buffer is found
to be too small, profiling will be aborted and no @file{gmon.out} file
will be produced. In that case, you will see the following message
printed to standard error:
@example
mcount: call graph buffer size limit exceeded, gmon.out will not be generated
@end example
Most of the symbols discussed in this section are defined in the header
@code{sys/gmon.h}. However, some symbols (for example @code{mcount})
are not defined in any header file, since they are only intended to be
called from code generated by the compiler.
@end deftp
@deftp Tunable glibc.mem.minarcs
The heuristic for sizing the call graph buffer is known to be
insufficient for small programs; hence, the calculated value is clamped
to be at least a minimum size. The default minimum (in units of
call graph entries, @code{struct tostruct}), is given by the macro
@code{MINARCS}. If you have some program with an unusually complex
call graph, for which the heuristic fails to allocate enough space,
you can use this tunable to increase the minimum to a larger value.
@end deftp
@deftp Tunable glibc.mem.maxarcs
To prevent excessive memory consumption when profiling very large
programs, the call graph buffer is allowed to have a maximum of
@code{MAXARCS} entries. For some very large programs, the default
value of @code{MAXARCS} defined in @file{sys/gmon.h} is too small; in
that case, you can use this tunable to increase it.
Note the value of the @code{maxarcs} tunable must be greater or equal
to that of the @code{minarcs} tunable; if this constraint is violated,
a warning will printed to standard error at program startup, and
the @code{minarcs} value will be used as the maximum as well.
Setting either tunable too high may result in a call graph buffer
whose size exceeds the available memory; in that case, an out of memory
error will be printed at program startup, the profiler will be
disabled, and no @file{gmon.out} file will be generated.
@end deftp
Reference in New Issue Copy Permalink