glibc/manual
Joseph Myers cb7be1590e Use gen-as-const.py to process .pysym files.
This patch eliminates the gen-py-const.awk variant of gen-as-const,
switching to use of gnu-as-const.py (with a new --python option) to
process .pysym files (i.e., to generate nptl_lock_constants.py), as
the syntax of those files is identical to that of .sym files.

Note that the generated nptl_lock_constants.py is *not* identical to
the version generated by the awk script.  Apart from the trivial
changes (comment referencing the new script, and output being sorted),
the constant FUTEX_WAITERS, PTHREAD_MUTEXATTR_FLAG_BITS,
PTHREAD_MUTEXATTR_FLAG_PSHARED and PTHREAD_MUTEX_PRIO_CEILING_MASK are
now output as positive rather than negative constants (on x86_64
anyway; maybe not necessarily on 32-bit systems):

< FUTEX_WAITERS = -2147483648
---
> FUTEX_WAITERS = 2147483648

< PTHREAD_MUTEXATTR_FLAG_BITS = -251662336
< PTHREAD_MUTEXATTR_FLAG_PSHARED = -2147483648
---
> PTHREAD_MUTEXATTR_FLAG_BITS = 4043304960
> PTHREAD_MUTEXATTR_FLAG_PSHARED = 2147483648

< PTHREAD_MUTEX_PRIO_CEILING_MASK = -524288
---
> PTHREAD_MUTEX_PRIO_CEILING_MASK = 4294443008

This is because gen-as-const has a cast of the constant value to long
int, which gen-py-const lacks.

I think the positive values are more logically correct, since the
constants in question are in fact unsigned in C.  But to reliably
produce gen-as-const.py output for constants that always (in C and
Python) reflects the signedness of values with the high bit of "long
int" set would mean more complicated logic needs to be used in
computing values.

The more correct positive values by themselves produce a failure of
nptl/test-mutexattr-printers, because masking with
~PTHREAD_MUTEXATTR_FLAG_BITS & ~PTHREAD_MUTEX_NO_ELISION_NP now leaves
a bit -1 << 32 in the Python value, resulting in a KeyError exception.
To avoid that, places masking with ~ of one of the constants in
question are changed to mask with 0xffffffff as well (this reflects
how ~ in Python applies to an infinite-precision integer whereas ~ in
C does not do any promotions beyond the width of int).

Tested for x86_64.

	* scripts/gen-as-const.py (main): Handle --python option.
	* scripts/gen-py-const.awk: Remove.
	* Makerules (py-const-script): Use gen-as-const.py.
	($(py-const)): Likewise.
	* nptl/nptl-printers.py (MutexPrinter.read_status_no_robust): Mask
	with 0xffffffff together with ~(PTHREAD_MUTEX_PRIO_CEILING_MASK).
	(MutexAttributesPrinter.read_values): Mask with 0xffffffff
	together with ~PTHREAD_MUTEXATTR_FLAG_BITS and
	~PTHREAD_MUTEX_NO_ELISION_NP.
	* manual/README.pretty-printers: Update reference to
	gen-py-const.awk.
2018-12-10 22:56:59 +00:00
..
examples manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
argp.texi manual: Complete @standards in argp.texi. 2017-06-16 01:19:30 -07:00
arith.texi Fix typo in the documentation of gcvt 2018-11-01 15:52:35 -03:00
charset.texi Fix missing @ before texinfo command 2018-04-06 08:56:24 +02:00
check-safety.sh Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
conf.texi [manual] Job control is no longer optional. 2018-10-17 14:10:51 -04:00
contrib.texi Update contrib.texi contributions. 2018-07-31 15:34:30 -04:00
creature.texi manual: Update _DEFAULT_SOURCE. [BZ #22862] 2018-02-21 02:38:42 -08:00
crypt.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
ctype.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
debug.texi Add manual documentation for threads.h 2018-07-24 14:07:31 -03:00
dir
errno.texi hurd: Fix errno* generation 2018-10-31 10:32:39 +01:00
fdl-1.3.texi
filesys.texi Add the statx function 2018-07-10 16:14:45 +02:00
freemanuals.texi
getopt.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
header.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
install-plain.texi
install.texi Patch to require Python 3.4 or later to build glibc. 2018-10-29 15:28:05 +00:00
intro.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
io.texi
ipc.texi manual/ipc.texi: Fix AC-safety notes. 2014-04-08 17:12:15 -04:00
job.texi [manual] Job control is no longer optional. 2018-10-17 14:10:51 -04:00
lang.texi manual: Rewrite the section on widths of integer types. 2017-08-10 20:28:28 -07:00
lgpl-2.1.texi
libc-texinfo.sh Remove add-ons mechanism. 2017-10-05 15:58:13 +00:00
libc.texinfo Update copyright dates not handled by scripts/update-copyrights. 2018-01-01 00:41:16 +00:00
libcbook.texi
libdl.texi * manual/libdl.texi: New. 2014-01-31 23:23:59 -02:00
llio.texi Fix documentation build with old makeinfo 2018-06-28 13:36:06 +01:00
locale.texi Use STRFMON_LDBL_IS_DBL instead of __ldbl_is_dbl. 2018-11-16 09:21:14 -02:00
macros.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
maint.texi Remove add-ons mechanism. 2017-10-05 15:58:13 +00:00
Makefile Patch to require Python 3.4 or later to build glibc. 2018-10-29 15:28:05 +00:00
math.texi Skeleton documentation for the RISC-V port 2018-01-29 10:22:26 -08:00
memory.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
message.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
nss.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
nsswitch.texi
pattern.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
pipe.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
platform.texi manual: Fix a syntax error. 2018-02-16 08:21:47 -08:00
probes.texi malloc: tcache double free check 2018-11-20 13:24:09 -05:00
process.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
README.pretty-printers Use gen-as-const.py to process .pysym files. 2018-12-10 22:56:59 +00:00
README.tunables Rename the glibc.tune namespace to glibc.cpu 2018-08-02 23:49:19 +05:30
resource.texi Add getcpu 2018-12-07 09:17:50 -08:00
search.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
setjmp.texi manual: Document getcontext uc_stack value on Linux [BZ #759] 2017-08-08 16:16:43 -03:00
signal.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
socket.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
startup.texi Revert: 2018-05-07 01:40:30 -03:00
stdio-fp.c
stdio.texi Document interaction with GCC built-ins in the Customizing Printf 2018-06-29 10:45:19 -06:00
string.texi Disallow use of DES encryption functions in new programs. 2018-06-29 16:53:18 +02:00
summary.pl Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
sysinfo.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
syslog.texi manual: Replace summary.awk with summary.pl. 2017-06-15 21:26:20 -07:00
terminal.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
texinfo.tex Update miscellaneous files from upstream sources. 2018-12-06 17:21:47 +00:00
texis.awk
threads.texi Add manual documentation for threads.h 2018-07-24 14:07:31 -03:00
time.texi manual: Touch up documentation for strftime and ALTMON_* constants. 2018-01-22 19:36:51 -08:00
tsort.awk Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
tunables.texi Mutex: Add pthread mutex tunables 2018-12-01 08:19:20 -08:00
users.texi manual: Revise crypt.texi. 2018-06-29 16:53:37 +02:00
xtract-typefun.awk Make shebang interpreter directives consistent 2016-01-07 04:03:21 -05:00

			TUNABLE FRAMEWORK
			=================

Tunables is a feature in the GNU C Library that allows application authors and
distribution maintainers to alter the runtime library behaviour to match their
workload.

The tunable framework allows modules within glibc to register variables that
may be tweaked through an environment variable.  It aims to enforce a strict
namespace rule to bring consistency to naming of these tunable environment
variables across the project.  This document is a guide for glibc developers to
add tunables to the framework.

ADDING A NEW TUNABLE
--------------------

The TOP_NAMESPACE macro is defined by default as 'glibc'.  If distributions
intend to add their own tunables, they should do so in a different top
namespace by overriding the TOP_NAMESPACE macro for that tunable.  Downstream
implementations are discouraged from using the 'glibc' top namespace for
tunables they don't already have consensus to push upstream.

There are three steps to adding a tunable:

1. Add a tunable to the list and fully specify its properties:

For each tunable you want to add, make an entry in elf/dl-tunables.list.  The
format of the file is as follows:

TOP_NAMESPACE {
  NAMESPACE1 {
    TUNABLE1 {
      # tunable attributes, one per line
    }
    # A tunable with default attributes, i.e. string variable.
    TUNABLE2
    TUNABLE3 {
      # its attributes
    }
  }
  NAMESPACE2 {
    ...
  }
}

The list of allowed attributes are:

- type:			Data type.  Defaults to STRING.  Allowed types are:
			INT_32, UINT_64, SIZE_T and STRING.  Numeric types may
			be in octal or hexadecimal format too.

- minval:		Optional minimum acceptable value.  For a string type
			this is the minimum length of the value.

- maxval:		Optional maximum acceptable value.  For a string type
			this is the maximum length of the value.

- default:		Specify an optional default value for the tunable.

- env_alias:		An alias environment variable

- security_level:	Specify security level of the tunable.  Valid values:

			SXID_ERASE: (default) Don't read for AT_SECURE binaries and
				    removed so that child processes can't read it.
			SXID_IGNORE: Don't read for AT_SECURE binaries, but retained for
				     non-AT_SECURE subprocesses.
			NONE: Read all the time.

2. Use TUNABLE_GET/TUNABLE_SET to get and set tunables.

3. OPTIONAL: If tunables in a namespace are being used multiple times within a
   specific module, set the TUNABLE_NAMESPACE macro to reduce the amount of
   typing.

GETTING AND SETTING TUNABLES
----------------------------

When the TUNABLE_NAMESPACE macro is defined, one may get tunables in that
module using the TUNABLE_GET macro as follows:

  val = TUNABLE_GET (check, int32_t, TUNABLE_CALLBACK (check_callback))

where 'check' is the tunable name, 'int32_t' is the C type of the tunable and
'check_callback' is the function to call if the tunable got initialized to a
non-default value.  The macro returns the value as type 'int32_t'.

The callback function should be defined as follows:

  void
  TUNABLE_CALLBACK (check_callback) (int32_t *valp)
  {
  ...
  }

where it can expect the tunable value to be passed in VALP.

Tunables in the module can be updated using:

  TUNABLE_SET (check, int32_t, val)

where 'check' is the tunable name, 'int32_t' is the C type of the tunable and
'val' is a value of same type.

To get and set tunables in a different namespace from that module, use the full
form of the macros as follows:

  val = TUNABLE_GET_FULL (glibc, cpu, hwcap_mask, uint64_t, NULL)

  TUNABLE_SET_FULL (glibc, cpu, hwcap_mask, uint64_t, val)

where 'glibc' is the top namespace, 'cpu' is the tunable namespace and the
remaining arguments are the same as the short form macros.

When TUNABLE_NAMESPACE is not defined in a module, TUNABLE_GET is equivalent to
TUNABLE_GET_FULL, so you will need to provide full namespace information for
both macros.  Likewise for TUNABLE_SET and TUNABLE_SET_FULL.

** IMPORTANT NOTE **

The tunable list is set as read-only after the dynamic linker relocates itself,
so setting tunable values must be limited only to tunables within the dynamic
linker, that too before relocation.

FUTURE WORK
-----------

The framework currently only allows a one-time initialization of variables
through environment variables and in some cases, modification of variables via
an API call.  A future goals for this project include:

- Setting system-wide and user-wide defaults for tunables through some
  mechanism like a configuration file.

- Allow tweaking of some tunables at runtime