Commit Graph

10 Commits

Author SHA1 Message Date
DJ Delorie
d5c3fafc43 Add per-thread cache to malloc
* config.make.in: Enable experimental malloc option.
* configure.ac: Likewise.
* configure: Regenerate.
* manual/install.texi: Document it.
* INSTALL: Regenerate.
* malloc/Makefile: Likewise.
* malloc/malloc.c: Add per-thread cache (tcache).
(tcache_put): New.
(tcache_get): New.
(tcache_thread_freeres): New.
(tcache_init): New.
(__libc_malloc): Use cached chunks if available.
(__libc_free): Initialize tcache if needed.
(__libc_realloc): Likewise.
(__libc_calloc): Likewise.
(_int_malloc): Prefill tcache when appropriate.
(_int_free): Likewise.
(do_set_tcache_max): New.
(do_set_tcache_count): New.
(do_set_tcache_unsorted_limit): New.
* manual/probes.texi: Document new probes.
* malloc/arena.c: Add new tcache tunables.
* elf/dl-tunables.list: Likewise.
* manual/tunables.texi: Document them.
* NEWS: Mention the per-thread cache.
2017-07-06 13:37:30 -04:00
H.J. Lu
03feacb562 x86: Rename glibc.tune.ifunc to glibc.tune.hwcaps
Rename glibc.tune.ifunc to glibc.tune.hwcaps and move it to
sysdeps/x86/dl-tunables.list since it is x86 specicifc.  Also
change type of data_cache_size, data_cache_size and
non_temporal_threshold to unsigned long int to match size_t.
Remove usage DEFAULT_STRLEN from cpu-tunables.c.

	* elf/dl-tunables.list (glibc.tune.ifunc): Removed.
	* sysdeps/x86/dl-tunables.list (glibc.tune.hwcaps): New.
	Remove security_level on all fields.
	* manual/tunables.texi: Replace ifunc with hwcaps.
	* sysdeps/x86/cpu-features.c (TUNABLE_CALLBACK (set_ifunc)):
	Renamed to ..
	(TUNABLE_CALLBACK (set_hwcaps)): This.
	(init_cpu_features): Updated.
	* sysdeps/x86/cpu-features.h (cpu_features): Change type of
	data_cache_size, data_cache_size and non_temporal_threshold to
	unsigned long int.
	* sysdeps/x86/cpu-tunables.c (DEFAULT_STRLEN): Removed.
	(TUNABLE_CALLBACK (set_ifunc)): Renamed to ...
	(TUNABLE_CALLBACK (set_hwcaps)): This.  Update comments.  Don't
	use DEFAULT_STRLEN.
2017-06-21 10:21:37 -07:00
H.J. Lu
da69a35566 Move x86 specific tunables to x86/dl-tunables.list
* elf/dl-tunables.list: Move x86 specific tunables to ...
	* sysdeps/x86/dl-tunables.list: Here.  New file.
2017-06-20 14:03:31 -07:00
H.J. Lu
905947c304 tunables: Add IFUNC selection and cache sizes
The current IFUNC selection is based on microbenchmarks in glibc.  It
should give the best performance for most workloads.  But other choices
may have better performance for a particular workload or on the hardware
which wasn't available at the selection was made.  The environment
variable, GLIBC_TUNABLES=glibc.tune.ifunc=-xxx,yyy,-zzz...., can be used
to enable CPU/ARCH feature yyy, disable CPU/ARCH feature yyy and zzz,
where the feature name is case-sensitive and has to match the ones in
cpu-features.h.  It can be used by glibc developers to override the
IFUNC selection to tune for a new processor or improve performance for
a particular workload.  It isn't intended for normal end users.

NOTE: the IFUNC selection may change over time.  Please check all
multiarch implementations when experimenting.

Also, GLIBC_TUNABLES=glibc.tune.x86_non_temporal_threshold=NUMBER is
provided to set threshold to use non temporal store to NUMBER,
GLIBC_TUNABLES=glibc.tune.x86_data_cache_size=NUMBER to set data cache
size, GLIBC_TUNABLES=glibc.tune.x86_shared_cache_size=NUMBER to set
shared cache size.

	* elf/dl-tunables.list (tune): Add ifunc,
	x86_non_temporal_threshold,
	x86_data_cache_size and x86_shared_cache_size.
	* manual/tunables.texi: Document glibc.tune.ifunc,
	glibc.tune.x86_data_cache_size, glibc.tune.x86_shared_cache_size
	and glibc.tune.x86_non_temporal_threshold.
	* sysdeps/unix/sysv/linux/x86/dl-sysdep.c: New file.
	* sysdeps/x86/cpu-tunables.c: Likewise.
	* sysdeps/x86/cacheinfo.c
	(init_cacheinfo): Check and get data cache size, shared cache
	size and non temporal threshold from cpu_features.
	* sysdeps/x86/cpu-features.c [HAVE_TUNABLES] (TUNABLE_NAMESPACE):
	New.
	[HAVE_TUNABLES] Include <unistd.h>.
	[HAVE_TUNABLES] Include <elf/dl-tunables.h>.
	[HAVE_TUNABLES] (TUNABLE_CALLBACK (set_ifunc)): Likewise.
	[HAVE_TUNABLES] (init_cpu_features): Use TUNABLE_GET to set
	IFUNC selection, data cache size, shared cache size and non
	temporal threshold.
	* sysdeps/x86/cpu-features.h (cpu_features): Add data_cache_size,
	shared_cache_size and non_temporal_threshold.
2017-06-20 08:37:28 -07:00
Siddhesh Poyarekar
ea9b0ecbf0 tunables: Add LD_HWCAP_MASK to tunables
Add LD_HWCAP_MASK to tunables in preparation of it being removed from
rtld.c.  This allows us to read LD_HWCAP_MASK much earlier so that it
can influence IFUNC resolution in aarch64.

This patch does not actually do anything other than read the
LD_HWCAP_MASK variable and add the tunables way to set the
LD_HWCAP_MASK, i.e. via the glibc.tune.hwcap_mask tunable.  In a
follow-up patch, the _dl_hwcap_mask will be replaced with
glibc.tune.hwcap_mask to complete the transition.

	* elf/dl-tunables.list: Add glibc.tune.hwcap_mask.
	* scripts/gen-tunables.awk: Include dl-procinfo.h.
	* manual/tunables.texi: Document glibc.tune.hwcap_mask.
2017-06-07 11:11:37 +05:30
Sunyeop Lee
65eff7fbdb Update old tunables framework document/script.
Since commit 8b9e9c3c0b, security_level replaces
is_secure. There were some old files need to be updated.

2017-03-23  Sunyeop Lee  <sunyeop97@gmail.com>

	* README.tunables: Updated descriptions.
	* elf/dl-tunables.list: Fixed typo: SXID_NONE -> NONE.
	* scripts/gen-tunables.awk: Updated the code related to the commit.
2017-03-24 11:08:28 +05:30
Siddhesh Poyarekar
8b9e9c3c0b tunables: Fix environment variable processing for setuid binaries (bz #21073)
Florian Weimer pointed out that we have three different kinds of
environment variables (and hence tunables):

1. Variables that are removed for setxid processes
2. Variables that are ignored in setxid processes but is passed on to
   child processes
3. Variables that are passed on to child processes all the time

Tunables currently only does (2) and (3) when it should be doing (1)
for MALLOC_CHECK_.  This patch enhances the is_secure flag in tunables
to an enum value that can specify which of the above three categories
the tunable (and its envvar alias) belongs to.

The default is for tunables to be in (1).  Hence, all of the malloc
tunables barring MALLOC_CHECK_ are explicitly specified to belong to
category (2).  There were discussions around abolishing category (2)
completely but we can do that as a separate exercise in 2.26.

Tested on x86_64 to verify that there are no regressions.

	[BZ #21073]
	* elf/dl-tunable-types.h (tunable_seclevel_t): New enum.
	* elf/dl-tunables.c (tunables_strdup): Remove.
	(get_next_env): Also return the previous envp.
	(parse_tunables): Erase tunables of category
	TUNABLES_SECLEVEL_SXID_ERASE.
	(maybe_enable_malloc_check): Make MALLOC_CHECK_
	TUNABLE_SECLEVEL_NONE if /etc/setuid-debug is accessible.
	(__tunables_init)[TUNABLES_FRONTEND ==
	TUNABLES_FRONTEND_valstring]: Update GLIBC_TUNABLES envvar
	after parsing.
	[TUNABLES_FRONTEND != TUNABLES_FRONTEND_valstring]: Erase
	tunable envvars of category TUNABLES_SECLEVEL_SXID_ERASE.
	* elf/dl-tunables.h (struct _tunable): Change member is_secure
	to security_level.
	* elf/dl-tunables.list: Add security_level annotations for all
	tunables.
	* scripts/gen-tunables.awk: Recognize and generate enum values
	for security_level.
	* elf/tst-env-setuid.c: New test case.
	* elf/tst-env-setuid-tunables: new test case.
	* elf/Makefile (tests-static): Add them.
2017-02-02 15:50:16 +05:30
Siddhesh Poyarekar
d054a81ab3 tunables: Avoid getenv calls and disable glibc.malloc.check by default
Builds with --enable-tunables failed on i686 because a call to getenv
got snuck into tunables, which pulled in strncmp.  This patch fixes
this build failure by making the glibc.malloc.check check even
simpler.  The previous approach was convoluted where the tunable was
disabled using an unsetenv and overwriting the tunable value with
colons.  The easier way is to simply mark the tunable as insecure by
default (i.e. won't be read for AT_SECURE programs) and then enabled
only when the /etc/suid-debug file is found.

This also ends up removing a bunch of functions that were specially
reimplemented (strlen, unsetenv) to avoid calling into string
routines.

Tested on x86_64 and i686.

	* elf/dl-tunables.c (tunables_unsetenv): Remove function.
	(min_strlen): Likewise.
	(disable_tunable): Likewise.
	(maybe_disable_malloc_check): Rename to
	maybe_enable_malloc_check.
	(maybe_enable_malloc_check): Enable glibc.malloc.check tunable
	if /etc/suid-debug file exists.
	(__tunables_init): Update caller.
	* elf/dl-tunables.list (glibc.malloc.check): Don't mark as
	secure.
2017-01-10 16:35:58 +05:30
Joseph Myers
bfff8b1bec Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
Siddhesh Poyarekar
67e58f3941 Add framework for tunables
The tunables framework allows us to uniformly manage and expose global
variables inside glibc as switches to users.  tunables/README has
instructions for glibc developers to add new tunables.

Tunables support can be enabled by passing the --enable-tunables
configure flag to the configure script.  This patch only adds a
framework and does not pose any limitations on how tunable values are
read from the user.  It also adds environment variables used in malloc
behaviour tweaking to the tunables framework as a PoC of the
compatibility interface.

	* manual/install.texi: Add --enable-tunables option.
	* INSTALL: Regenerate.
	* README.tunables: New file.
	* Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
	(before-compile): Generate dl-tunable-list.h early.
	* config.h.in: Add HAVE_TUNABLES.
	* config.make.in: Add have-tunables.
	* configure.ac: Add --enable-tunables option.
	* configure: Regenerate.
	* csu/init-first.c (__libc_init_first): Move
	__libc_init_secure earlier...
	* csu/init-first.c (LIBC_START_MAIN):... to here.
	Include dl-tunables.h, libc-internal.h.
	(LIBC_START_MAIN) [!SHARED]: Initialize tunables for static
	binaries.
	* elf/Makefile (dl-routines): Add dl-tunables.
	* elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE
	namespace.
	* elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_
	only when !HAVE_TUNABLES.
	* elf/rtld.c (process_envvars): Likewise.
	* elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h
	(_dl_sysdep_start): Call __tunables_init.
	* elf/dl-tunable-types.h: New file.
	* elf/dl-tunables.c: New file.
	* elf/dl-tunables.h: New file.
	* elf/dl-tunables.list: New file.
	* malloc/tst-malloc-usable-static.c: New test case.
	* malloc/Makefile (tests-static): Add it.
	* malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h.
	Define TUNABLE_NAMESPACE.
	(DL_TUNABLE_CALLBACK (set_mallopt_check)): New function.
	(DL_TUNABLE_CALLBACK_FNDECL): New macro.  Use it to define
	callback functions.
	(ptmalloc_init): Set tunable values.
	* scripts/gen-tunables.awk: New file.
	* sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h.
	(_dl_sysdep_start): Call __tunables_init.
2016-12-31 23:49:24 +05:30