mirror of
https://sourceware.org/git/glibc.git
synced 2024-12-26 04:31:03 +00:00
9bcd12d223
The GNU implementation of wcrtomb assumes that there are at least MB_CUR_MAX bytes available in the destination buffer passed to wcrtomb as the first argument. This is not compatible with the POSIX definition, which only requires enough space for the input wide character. This does not break much in practice because when users supply buffers smaller than MB_CUR_MAX (e.g. in ncurses), they compute and dynamically allocate the buffer, which results in enough spare space (thanks to usable_size in malloc and padding in alloca) that no actual buffer overflow occurs. However when the code is built with _FORTIFY_SOURCE, it runs into the hard check against MB_CUR_MAX in __wcrtomb_chk and hence fails. It wasn't evident until now since dynamic allocations would result in wcrtomb not being fortified but since _FORTIFY_SOURCE=3, that limitation is gone, resulting in such code failing. To fix this problem, introduce an internal buffer that is MB_LEN_MAX long and use that to perform the conversion and then copy the resultant bytes into the destination buffer. Also move the fortification check into the main implementation, which checks the result after conversion and aborts if the resultant byte count is greater than the destination buffer size. One complication is that applications that assume the MB_CUR_MAX limitation to be gone may not be able to run safely on older glibcs if they use static destination buffers smaller than MB_CUR_MAX; dynamic allocations will always have enough spare space that no actual overruns will occur. One alternative to fixing this is to bump symbol version to prevent them from running on older glibcs but that seems too strict a constraint. Instead, since these users will only have made this decision on reading the manual, I have put a note in the manual warning them about the pitfalls of having static buffers smaller than MB_CUR_MAX and running them on older glibc. Benchmarking: The wcrtomb microbenchmark shows significant increases in maximum execution time for all locales, ranging from 10x for ar_SA.UTF-8 to 1.5x-2x for nearly everything else. The mean execution time however saw practically no impact, with some results even being quicker, indicating that cache locality has a much bigger role in the overhead. Given that the additional copy uses a temporary buffer inside wcrtomb, it's likely that a hot path will end up putting that buffer (which is responsible for the additional overhead) in a similar place on stack, giving the necessary cache locality to negate the overhead. However in situations where wcrtomb ends up getting called at wildly different spots on the call stack (or is on different call stacks, e.g. with threads or different execution contexts) and is still a hotspot, the performance lag will be visible. Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org> |
||
---|---|---|
.. | ||
examples | ||
argp.texi | ||
arith.texi | ||
charset.texi | ||
check-safety.sh | ||
conf.texi | ||
contrib.texi | ||
creature.texi | ||
crypt.texi | ||
ctype.texi | ||
debug.texi | ||
dir | ||
dynlink.texi | ||
errno.texi | ||
fdl-1.3.texi | ||
filesys.texi | ||
freemanuals.texi | ||
getopt.texi | ||
header.texi | ||
install-plain.texi | ||
install.texi | ||
intro.texi | ||
io.texi | ||
ipc.texi | ||
job.texi | ||
lang.texi | ||
lgpl-2.1.texi | ||
libc-texinfo.sh | ||
libc.texinfo | ||
libcbook.texi | ||
llio.texi | ||
locale.texi | ||
macros.texi | ||
maint.texi | ||
Makefile | ||
math.texi | ||
memory.texi | ||
message.texi | ||
nss.texi | ||
nsswitch.texi | ||
pattern.texi | ||
pipe.texi | ||
platform.texi | ||
probes.texi | ||
process.texi | ||
README.pretty-printers | ||
README.tunables | ||
resource.texi | ||
search.texi | ||
setjmp.texi | ||
signal.texi | ||
socket.texi | ||
startup.texi | ||
stdio-fp.c | ||
stdio.texi | ||
string.texi | ||
summary.pl | ||
sysinfo.texi | ||
syslog.texi | ||
terminal.texi | ||
texinfo.tex | ||
texis.awk | ||
threads.texi | ||
time.texi | ||
tsort.awk | ||
tunables.texi | ||
users.texi | ||
xtract-typefun.awk |
TUNABLE FRAMEWORK ================= Tunables is a feature in the GNU C Library that allows application authors and distribution maintainers to alter the runtime library behaviour to match their workload. The tunable framework allows modules within glibc to register variables that may be tweaked through an environment variable. It aims to enforce a strict namespace rule to bring consistency to naming of these tunable environment variables across the project. This document is a guide for glibc developers to add tunables to the framework. ADDING A NEW TUNABLE -------------------- The TOP_NAMESPACE macro is defined by default as 'glibc'. If distributions intend to add their own tunables, they should do so in a different top namespace by overriding the TOP_NAMESPACE macro for that tunable. Downstream implementations are discouraged from using the 'glibc' top namespace for tunables they don't already have consensus to push upstream. There are three steps to adding a tunable: 1. Add a tunable to the list and fully specify its properties: For each tunable you want to add, make an entry in elf/dl-tunables.list. The format of the file is as follows: TOP_NAMESPACE { NAMESPACE1 { TUNABLE1 { # tunable attributes, one per line } # A tunable with default attributes, i.e. string variable. TUNABLE2 TUNABLE3 { # its attributes } } NAMESPACE2 { ... } } The list of allowed attributes are: - type: Data type. Defaults to STRING. Allowed types are: INT_32, UINT_64, SIZE_T and STRING. Numeric types may be in octal or hexadecimal format too. - minval: Optional minimum acceptable value. For a string type this is the minimum length of the value. - maxval: Optional maximum acceptable value. For a string type this is the maximum length of the value. - default: Specify an optional default value for the tunable. - env_alias: An alias environment variable - security_level: Specify security level of the tunable for AT_SECURE binaries. Valid values are: SXID_ERASE: (default) Do not read and do not pass on to child processes. SXID_IGNORE: Do not read, but retain for non-AT_SECURE child processes. NONE: Read all the time. 2. Use TUNABLE_GET/TUNABLE_SET/TUNABLE_SET_WITH_BOUNDS to get and set tunables. 3. OPTIONAL: If tunables in a namespace are being used multiple times within a specific module, set the TUNABLE_NAMESPACE macro to reduce the amount of typing. GETTING AND SETTING TUNABLES ---------------------------- When the TUNABLE_NAMESPACE macro is defined, one may get tunables in that module using the TUNABLE_GET macro as follows: val = TUNABLE_GET (check, int32_t, TUNABLE_CALLBACK (check_callback)) where 'check' is the tunable name, 'int32_t' is the C type of the tunable and 'check_callback' is the function to call if the tunable got initialized to a non-default value. The macro returns the value as type 'int32_t'. The callback function should be defined as follows: void TUNABLE_CALLBACK (check_callback) (int32_t *valp) { ... } where it can expect the tunable value to be passed in VALP. Tunables in the module can be updated using: TUNABLE_SET (check, val) where 'check' is the tunable name and 'val' is a value of same type. To get and set tunables in a different namespace from that module, use the full form of the macros as follows: val = TUNABLE_GET_FULL (glibc, cpu, hwcap_mask, uint64_t, NULL) TUNABLE_SET_FULL (glibc, cpu, hwcap_mask, val) where 'glibc' is the top namespace, 'cpu' is the tunable namespace and the remaining arguments are the same as the short form macros. The minimum and maximum values can updated together with the tunable value using: TUNABLE_SET_WITH_BOUNDS (check, val, min, max) where 'check' is the tunable name, 'val' is a value of same type, 'min' and 'max' are the minimum and maximum values of the tunable. To set the minimum and maximum values of tunables in a different namespace from that module, use the full form of the macros as follows: val = TUNABLE_GET_FULL (glibc, cpu, hwcap_mask, uint64_t, NULL) TUNABLE_SET_WITH_BOUNDS_FULL (glibc, cpu, hwcap_mask, val, min, max) where 'glibc' is the top namespace, 'cpu' is the tunable namespace and the remaining arguments are the same as the short form macros. When TUNABLE_NAMESPACE is not defined in a module, TUNABLE_GET is equivalent to TUNABLE_GET_FULL, so you will need to provide full namespace information for both macros. Likewise for TUNABLE_SET, TUNABLE_SET_FULL, TUNABLE_SET_WITH_BOUNDS and TUNABLE_SET_WITH_BOUNDS_FULL. ** IMPORTANT NOTE ** The tunable list is set as read-only after the dynamic linker relocates itself, so setting tunable values must be limited only to tunables within the dynamic linker, that too before relocation. FUTURE WORK ----------- The framework currently only allows a one-time initialization of variables through environment variables and in some cases, modification of variables via an API call. A future goals for this project include: - Setting system-wide and user-wide defaults for tunables through some mechanism like a configuration file. - Allow tweaking of some tunables at runtime