Commit Graph

746 Commits

Author SHA1 Message Date
H.J. Lu
0aac205a81 x86: Update _dl_tlsdesc_dynamic to preserve caller-saved registers
Compiler generates the following instruction sequence for GNU2 dynamic
TLS access:

	leaq	tls_var@TLSDESC(%rip), %rax
	call	*tls_var@TLSCALL(%rax)

or

	leal	tls_var@TLSDESC(%ebx), %eax
	call	*tls_var@TLSCALL(%eax)

CALL instruction is transparent to compiler which assumes all registers,
except for EFLAGS and RAX/EAX, are unchanged after CALL.  When
_dl_tlsdesc_dynamic is called, it calls __tls_get_addr on the slow
path.  __tls_get_addr is a normal function which doesn't preserve any
caller-saved registers.  _dl_tlsdesc_dynamic saved and restored integer
caller-saved registers, but didn't preserve any other caller-saved
registers.  Add _dl_tlsdesc_dynamic IFUNC functions for FNSAVE, FXSAVE,
XSAVE and XSAVEC to save and restore all caller-saved registers.  This
fixes BZ #31372.

Add GLRO(dl_x86_64_runtime_resolve) with GLRO(dl_x86_tlsdesc_dynamic)
to optimize elf_machine_runtime_setup.
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
2024-02-28 09:02:56 -08:00
Adhemerval Zanella
f4c142bb9f arm: Use _dl_find_object on __gnu_Unwind_Find_exidx (BZ 31405)
Instead of __dl_iterate_phdr. On ARM dlfo_eh_frame/dlfo_eh_count
maps to PT_ARM_EXIDX vaddr start / length.

On a Neoverse N1 machine with 160 cores, the following program:

  $ cat test.c
  #include <stdlib.h>
  #include <pthread.h>
  #include <assert.h>

  enum {
    niter = 1024,
    ntimes = 128,
  };

  static void *
  tf (void *arg)
  {
    int a = (int) arg;

    for (int i = 0; i < niter; i++)
      {
        void *p[ntimes];
        for (int j = 0; j < ntimes; j++)
  	p[j] = malloc (a * 128);
        for (int j = 0; j < ntimes; j++)
  	free (p[j]);
      }

    return NULL;
  }

  int main (int argc, char *argv[])
  {
    enum { nthreads = 16 };
    pthread_t t[nthreads];

    for (int i = 0; i < nthreads; i ++)
      assert (pthread_create (&t[i], NULL, tf, (void *) i) == 0);

    for (int i = 0; i < nthreads; i++)
      {
        void *r;
        assert (pthread_join (t[i], &r) == 0);
        assert (r == NULL);
      }

    return 0;
  }
  $ arm-linux-gnueabihf-gcc -fsanitize=address test.c -o test

Improves from ~15s to 0.5s.

Checked on arm-linux-gnueabihf.
2024-02-23 08:50:00 -03:00
Carlos O'Donell
94aa256a47 elf: Fix tst-nodeps2 test failure.
After 78ca44da01
("elf: Relocate libc.so early during startup and dlmopen (bug 31083)")
we start seeing tst-nodeps2 failures when building the testsuite with
--enable-hard-coded-path-in-tests.

When building the testsuite with --enable-hard-coded-path-in-tests
the tst-nodeps2-mod.so is not built with the required DT_RUNPATH
values and the test escapes the test framework and loads the system
libraries and aborts. The fix is to use the existing
$(link-test-modules-rpath-link) variable to set DT_RUNPATH correctly.

No regressions on x86_64.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2024-01-10 14:08:26 -05:00
Paul Eggert
dff8da6b3e Update copyright dates with scripts/update-copyrights 2024-01-01 10:53:40 -08:00
Szabolcs Nagy
980450f126 elf: Add TLS modid reuse test for bug 29039
This is a minimal regression test for bug 29039 which only affects
targets with TLSDESC and a reproducer requires that

1) Have modid gaps (closed modules) with old generation.
2) Update a DTV to a newer generation (needs a newer dlopen).
3) But do not update the closed gap entry in that DTV.
4) Reuse the modid gap for a new module (another dlopen).
5) Use dynamic TLSDESC in that new module with old generation (bug).
6) Access TLS via this TLSDESC and the now outdated DTV.

However step (3) in practice rarely happens: during DTV update the
entries for closed modids are initialized to "unallocated" and then
dynamic TLSDESC calls __tls_get_addr independently of its generation.
The only exception to this is DTV setup at thread creation (gaps are
initialized to NULL instead of unallocated) or DTV resize where the
gap entries are outside the previous DTV array (again NULL instead
of unallocated, and this requires loading > DTV_SURPLUS modules).

So the bug can only cause NULL (+ offset) dereference, not use after
free. And the easiest way to get (3) is via thread creation.

Note that step (5) requires that the newly loaded module has larger
TLS than the remaining optional static TLS. And for (6) there cannot
be other TLS access or dlopen in the thread that updates the DTV.

Tested on aarch64-linux-gnu.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2023-12-20 08:45:48 +00:00
Stefan Liebler
3150cc0c90 Fix elf/tst-env-setuid[-static] if test needs to be rerun.
If /tmp is mounted nosuid and make xcheck is run,
then tst-env-setuid fails UNSUPPORTED with "SGID failed: GID and EGID match"
and /var/tmp/tst-sonamemove-runmod1.so.profile is created.

If you then try to rerun the test with a suid mounted test-dir
(the SGID binary is created in test-dir which defaults to /tmp)
with something like that:
make tst-env-setuid-ENV="TMPDIR=..." t=elf/tst-env-setuid test
the test fails as the LD_PROFILE output file is still available
from the previous run.

Thus this patch removes the LD_PROFILE output file in parent
before spawning the SGID binary.

Even if LD_PROFILE is not supported anymore in static binaries,
use a different library and thus output file for tst-env-setuid
and tst-env-setuid-static in order to not interfere if both
tests are run in parallel.

Furthermore the checks in test_child are now more verbose.
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2023-12-20 08:41:06 +01:00
Florian Weimer
78ca44da01 elf: Relocate libc.so early during startup and dlmopen (bug 31083)
This makes it more likely that objects without dependencies can
use IFUNC resolvers in libc.so.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-11-27 11:28:13 +01:00
Adhemerval Zanella
55f41ef8de elf: Remove LD_PROFILE for static binaries
The _dl_non_dynamic_init does not parse LD_PROFILE, which does not
enable profile for dlopen objects.  Since dlopen is deprecated for
static objects, it is better to remove the support.

It also allows to trim down libc.a of profile support.

Checked on x86_64-linux-gnu.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-11-21 16:15:42 -03:00
Adhemerval Zanella
4a133885a7 elf: Ignore LD_PROFILE for setuid binaries
Loader does not ignore LD_PROFILE in secure-execution mode (different
than man-page states [1]), rather it uses a different path
(/var/profile) and ignore LD_PROFILE_OUTPUT.

Allowing secure-execution profiling is already a non good security
boundary, since it enables different code paths and extra OS access by
the process.  But by ignoring LD_PROFILE_OUTPUT, the resulting profile
file might also be acceded in a racy manner since the file name does not
use any process-specific information (such as pid, timing, etc.).

Another side-effect is it forces lazy binding even on libraries that
might be with DF_BIND_NOW.

[1] https://man7.org/linux/man-pages/man8/ld.so.8.html
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-11-21 16:15:42 -03:00
Adhemerval Zanella
11f7e3dd8f elf: Add all malloc tunable to unsecvars
Some environment variables allow alteration of allocator behavior
across setuid boundaries, where a setuid program may ignore the
tunable, but its non-setuid child can read it and adjust the memory
allocator behavior accordingly.

Most library behavior tunings is limited to the current process and does
not bleed in scope; so it is unclear how pratical this misfeature is.
If behavior change across privilege boundaries is desirable, it would be
better done with a wrapper program around the non-setuid child that sets
these envvars, instead of using the setuid process as the messenger.

The patch as fixes tst-env-setuid, where it fail if any unsecvars is
set.  It also adds a dynamic test, although it requires
--enable-hardcoded-path-in-tests so kernel correctly sets the setuid
bit (using the loader command directly would require to set the
setuid bit on the loader itself, which is not a usual deployment).

Co-authored-by: Siddhesh Poyarekar <siddhesh@sourceware.org>

Checked on x86_64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-21 16:15:42 -03:00
Adhemerval Zanella
9c96c87d60 elf: Ignore GLIBC_TUNABLES for setuid/setgid binaries
The tunable privilege levels were a retrofit to try and keep the malloc
tunable environment variables' behavior unchanged across security
boundaries.  However, CVE-2023-4911 shows how tricky can be
tunable parsing in a security-sensitive environment.

Not only parsing, but the malloc tunable essentially changes some
semantics on setuid/setgid processes.  Although it is not a direct
security issue, allowing users to change setuid/setgid semantics is not
a good security practice, and requires extra code and analysis to check
if each tunable is safe to use on all security boundaries.

It also means that security opt-in features, like aarch64 MTE, would
need to be explicit enabled by an administrator with a wrapper script
or with a possible future system-wide tunable setting.

Co-authored-by: Siddhesh Poyarekar  <siddhesh@sourceware.org>
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-21 16:15:42 -03:00
Adhemerval Zanella
a8dcffb306 elf: Handle non-directory name in search path (BZ 31035)
The open_path stops if a relative path in search path contains a
component that is a non directory (for instance, if the component
is an existing file).

For instance:

  $ cat > lib.c <<EOF
  > void foo (void) {}
  > EOF
  $ gcc -shared -fPIC -o lib.so lib.c
  $ cat > main.c <<EOF
  extern void foo ();
  int main () { foo (); return 0; }
  EOF
  $ gcc -o main main.c lib.so
  $ LD_LIBRARY_PATH=. ./main
  $ LD_LIBRARY_PATH=non-existing/path:. ./main
  $ LD_LIBRARY_PATH=$(pwd)/main:. ./main
  $ LD_LIBRARY_PATH=./main:. ./main
  ./main: error while loading shared libraries: lib.so: cannot open shared object file: No such file or directory

The invalid './main' should be ignored as a non-existent one,
instead as a valid but non accessible file.

Absolute paths do not trigger this issue because their status are
initialized as 'unknown' and open_path check if this is a directory.

Checked on x86_64-linux-gnu.

Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
2023-11-16 11:01:51 -03:00
Yanzhang Wang
2d27dc1c75 test: Run the tst-tls-allocation-failure-static-patched with test-wrapper.
If we use cross test with ssh, this test needs to be ran on the remote.
2023-11-13 15:55:33 +01:00
Adhemerval Zanella
bf033c0072 elf: Add glibc.mem.decorate_maps tunable
The PR_SET_VMA_ANON_NAME support is only enabled through a configurable
kernel switch, mainly because assigning a name to a
anonymous virtual memory area might prevent that area from being
merged with adjacent virtual memory areas.

For instance, with the following code:

   void *p1 = mmap (NULL,
                    1024 * 4096,
                    PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS,
                    -1,
                    0);

   void *p2 = mmap (p1 + (1024 * 4096),
                    1024 * 4096,
                    PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS,
                    -1,
                    0);

The kernel will potentially merge both mappings resulting in only one
segment of size 0x800000.  If the segment is names with
PR_SET_VMA_ANON_NAME with different names, it results in two mappings.

Although this will unlikely be an issue for pthread stacks and malloc
arenas (since for pthread stacks the guard page will result in
a PROT_NONE segment, similar to the alignment requirement for the arena
block), it still might prevent the mmap memory allocated for detail
malloc.

There is also another potential scalability issue, where the prctl
requires
to take the mmap global lock which is still not fully fixed in Linux
[1] (for pthread stacks and arenas, it is mitigated by the stack
cached and the arena reuse).

So this patch disables anonymous mapping annotations as default and
add a new tunable, glibc.mem.decorate_maps, can be used to enable
it.

[1] https://lwn.net/Articles/906852/

Checked on x86_64-linux-gnu and aarch64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-07 10:27:57 -03:00
Adhemerval Zanella
fee9e40a8d malloc: Decorate malloc maps
Add anonymous mmap annotations on loader malloc, malloc when it
allocates memory with mmap, and on malloc arena.  The /proc/self/maps
will now print:

   [anon: glibc: malloc arena]
   [anon: glibc: malloc]
   [anon: glibc: loader malloc]

On arena allocation, glibc annotates only the read/write mapping.

Checked on x86_64-linux-gnu and aarch64-linux-gnu.
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-07 10:27:20 -03:00
Adhemerval Zanella
6afce56c19 nptl: Decorate thread stack on pthread_create
Linux 4.5 removed thread stack annotations due to the complexity of
computing them [1], and Linux added PR_SET_VMA_ANON_NAME on 5.17
as a way to name anonymous virtual memory areas.

This patch adds decoration on the stack created and used by
pthread_create, for glibc crated thread stack the /proc/self/maps will
now show:

  [anon: glibc: pthread stack: <tid>]

And for user-provided stacks:

  [anon: glibc: pthread user stack: <tid>]

The guard page is not decorated, and the mapping name is cleared when
the thread finishes its execution (so the cached stack does not have any
name associated).

Checked on x86_64-linux-gnu aarch64 aarch64-linux-gnu.

[1] 65376df582

Co-authored-by: Ian Rogers <irogers@google.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
2023-11-07 10:27:20 -03:00
Adhemerval Zanella
e6e3c66688 crypt: Remove libcrypt support
All the crypt related functions, cryptographic algorithms, and
make requirements are removed,  with only the exception of md5
implementation which is moved to locale folder since it is
required by localedef for integrity protection (libc's
locale-reading code does not check these, but localedef does
generate them).

Besides thec code itself, both internal documentation and the
manual is also adjusted.  This allows to remove both --enable-crypt
and --enable-nss-crypt configure options.

Checked with a build for all affected ABIs.

Co-authored-by: Zack Weinberg <zack@owlfolio.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-10-30 13:03:59 -03:00
Florian Weimer
3d9265467e elf: Check that --list-diagnostics output has the expected syntax
Parts of elf/tst-rtld-list-diagnostics.py have been copied from
scripts/tst-ld-trace.py.

The abnf module is entirely optional and used to verify the
ABNF grammar as included in the manual.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2023-08-25 14:19:16 +02:00
Carlos O'Donell
ac14b7ee06 elf: Sort Makefile variables.
Sort Makefile variables using scrips/sort-makefile-lines.py.

No code generation changes observed in non-test binary artifacts.
No regressions on x86_64 and i686.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2023-06-02 21:43:05 -04:00
Florian Weimer
d0f07f7df8 elf: Make more functions available for binding during dlclose (bug 30425)
Previously, after destructors for a DSO have been invoked, ld.so refused
to bind against that DSO in all cases.  Relax this restriction somewhat
if the referencing object is itself a DSO that is being unloaded.  This
assumes that the symbol reference is not going to be stored anywhere.

The situation in the test case can arise fairly easily with C++ and
objects that are built with different optimization levels and therefore
define different functions with vague linkage.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-05-30 13:25:50 +02:00
Paul Pluzhnikov
630da022cb Fix misspellings in elf/ -- BZ 25337
Applying this commit results in bit-identical libc.so.6.
The elf/ld-linux-x86-64.so.2 does change, but only in .note.gnu.build-id

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-05-29 23:00:12 +00:00
Roy Eldar
d2b1af4305 elf: add test for dl-printf
This patch checks _dl_debug_vdprintf, by passing various inputs to
_dl_dprintf and comparing the output with invocations of snprintf.

Signed-off-by: Roy Eldar <royeldar0@gmail.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
2023-05-25 18:51:23 +02:00
Qixing ksyx Xue
dbfc83bdca elf: Add test for locating libraries in root dir (bug 30435)
When dlopen is being called, efforts have been made to improve
future lookup performance. This includes marking a search path
as non-existent using `stat`. However, if the root directory
is given as a search path, there exists a bug which erroneously
marks it as non-existing.

The bug is reproduced under the following sequence:

  1. dlopen is called to open a shared library, with at least:
     1) a dependency 'A.so' not directly under the '/' directory
        (e.g. /lib/A.so), and
     2) another dependency 'B.so' resides in '/'.
  2. for this bug to reproduce, 'A.so' should be searched *before* 'B.so'.
  3. it first tries to find 'A.so' in /, (e.g. /A.so):
     - this will (obviously) fail,
     - since it's the first time we have seen the '/' directory,
       its 'status' is 'unknown'.
  4. `buf[buflen - namelen - 1] = '\0'` is executed:
     - it intends to remove the leaf and its final slash,
     - because of the speciality of '/', its buflen == namelen + 1,
     - it erroneously clears the entire buffer.
  6. it then calls 'stat' with the empty buffer:
     - which will result in an error.
  7. so it marks '/' as 'nonexisting', future lookups will not consider
     this path.
  8. while /B.so *does* exist, failure to look it up in the '/'
     directory leads to a 'cannot open shared object file' error.

This patch fixes the bug by preventing 'buflen', an index to put '\0',
from being set to 0, so that the root '/' is always kept.
Relative search paths are always considered as 'existing' so this
wont be affected.

Writeup by Moody Liu <mooodyhunter@outlook.com>

Suggested-by: Carlos O'Donell <carlos@redhat.com>
Signed-off-by: Qixing ksyx Xue <qixingxue@outlook.com>
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-05-25 11:10:54 -04:00
Carlos O'Donell
a08e854d00 elf: Adjust tests in Makefile.
Sort tests against updated scripts/sort-makefile-lines.py.

No changes in generated code.
No regressions on x86_64 and i686.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-05-18 12:33:44 -04:00
Carlos O'Donell
aa492359f1 elf: Reformat Makefile.
Fix list terminator whitspace.
Sort using scripts/sort-makefile-lines.py.

No code generation changes observed in binary artifacts.
No regressions on x86_64 and i686.
2023-05-16 07:17:19 -04:00
Andreas Schwab
ab5aa2ee3d dlopen: skip debugger notification for DSO loaded from sprof (bug 30258)
Avoid inconsistent state in the debugger interface.
2023-05-09 14:36:34 +02:00
Adhemerval Zanella Netto
33237fe83d Remove --enable-tunables configure option
And make always supported.  The configure option was added on glibc 2.25
and some features require it (such as hwcap mask, huge pages support, and
lock elisition tuning).  It also simplifies the build permutations.

Changes from v1:
 * Remove glibc.rtld.dynamic_sort changes, it is orthogonal and needs
   more discussion.
 * Cleanup more code.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2023-03-29 14:33:06 -03:00
Romain Geissler
bb557fff2e elf: Take into account ${sysconfdir} in elf/tst-ldconfig-p.sh
Take into account ${sysconfdir} in elf/tst-ldconfig-p.sh.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2023-03-27 16:58:08 -03:00
Adhemerval Zanella Netto
91fc5b9990 Remove --with-default-link configure option
Now that there is no need to use a special linker script to hardening
internal data structures, remove the --with-default-link configure
option and associated definitions.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-03-27 13:57:55 -03:00
Adhemerval Zanella Netto
3020f72618 libio: Remove the usage of __libc_IO_vtables
Instead of using a special ELF section along with a linker script
directive to put the IO vtables within the RELRO section, the libio
vtables are all moved to an array marked as data.relro (so linker
will place in the RELRO segment without the need of extra directives).

To avoid static linking namespace issues and including all vtable
referenced objects, all required function pointers are set to weak alias.

Checked on x86_64-linux-gnu, i686-linux-gnu, and aarch64-linux-gnu.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-03-27 13:57:55 -03:00
Arsen Arsenović
f7e751affb elf: Add missing dependency between resolvfail and testobj1.so
It was possible to run this test individually and have it fail because
it can't find testobj1.so.  This patch adds that dependency, to prevent
such issues.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2023-03-10 17:53:18 +01:00
Arsen Arsenović
6c7388d0b9 elf: Add -z lazy to some more tests
Some toolchains, such as that used on Gentoo Hardened, set -z now out of
the box.  This trips up a couple of tests.

Reviewed-by: Florian Weimer <fweimer@redhat.com>
2023-03-10 17:53:15 +01:00
Joan Bruguera
1b0ea8c5d8 elf: Restore ldconfig libc6 implicit soname logic [BZ #30125]
While cleaning up old libc version support, the deprecated libc4 code was
accidentally kept in `implicit_soname`, instead of the libc6 code.

This causes additional symlinks to be created by `ldconfig` for libraries
without a soname, e.g. a library `libsomething.123.456.789` without a soname
will create a `libsomething.123` -> `libsomething.123.456.789` symlink.

As the libc6 version of the `implicit_soname` code is a trivial `xstrdup`,
just inline it and remove `implicit_soname` altogether.

Some further simplification looks possible (e.g. the call to `create_links`
looks like a no-op if `soname == NULL`, other than the verbose printfs), but
logic is kept as-is for now.

Fixes: BZ #30125
Fixes: 8ee878592c ("Assume only FLAG_ELF_LIBC6 suport")
Signed-off-by: Joan Bruguera <joanbrugueram@gmail.com>

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2023-02-20 09:32:46 -03:00
Florian Weimer
9fd63e3537 elf: Smoke-test ldconfig -p against system /etc/ld.so.cache
The test is sufficient to detect the ldconfig bug fixed in
commit 9fe6f63638 ("elf: Fix 64 time_t
support for installed statically binaries").

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-02-08 18:11:04 +01:00
Joseph Myers
6d7e8eda9b Update copyright dates with scripts/update-copyrights 2023-01-06 21:14:39 +00:00
Florian Weimer
118816de33 libio: Convert __vswprintf_internal to buffers (bug 27857)
Always null-terminate the buffer and set E2BIG if the buffer is too
small.  This fixes bug 27857.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-12-19 18:56:55 +01:00
Florian Weimer
5365acc567 libio: Convert __obstack_vprintf_internal to buffers (bug 27124)
This fixes bug 27124 because the problematic built-in vtable is gone.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-12-19 18:56:55 +01:00
Florian Weimer
fb9bd841b8 libio: Convert __vsprintf_internal to buffers
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-12-19 18:56:55 +01:00
Florian Weimer
e88b9f0e5c stdio-common: Convert vfprintf and related functions to buffers
vfprintf is entangled with vfwprintf (of course), __printf_fp,
__printf_fphex, __vstrfmon_l_internal, and the strfrom family of
functions.  The latter use the internal snprintf functionality,
so vsnprintf is converted as well.

The simples conversion is __printf_fphex, followed by
__vstrfmon_l_internal and __printf_fp, and finally
__vfprintf_internal and __vfwprintf_internal.  __vsnprintf_internal
and strfrom* are mostly consuming the new interfaces, so they
are comparatively simple.

__printf_fp is a public symbol, so the FILE *-based interface
had to preserved.

The __printf_fp rewrite does not change the actual binary-to-decimal
conversion algorithm, and digits are still not emitted directly to
the target buffer.  However, the staging buffer now uses bytes
instead of wide characters, and one buffer copy is eliminated.

The changes are at least performance-neutral in my testing.
Floating point printing and snprintf improved measurably, so that
this Lua script

  for i=1,5000000 do
      print(i, i * math.pi)
  end

runs about 5% faster for me.  To preserve fprintf performance for
a simple "%d" format, this commit has some logic changes under
LABEL (unsigned_number) to avoid additional function calls.  There
are certainly some very easy performance improvements here: binary,
octal and hexadecimal formatting can easily avoid the temporary work
buffer (the number of digits can be computed ahead-of-time using one
of the __builtin_clz* built-ins). Decimal formatting can use a
specialized version of _itoa_word for base 10.

The existing (inconsistent) width handling between strfmon and printf
is preserved here.  __print_fp_buffer_1 would have to use
__translated_number_width to achieve ISO conformance for printf.

Test expectations in libio/tst-vtables-common.c are adjusted because
the internal staging buffer merges all virtual function calls into
one.

In general, stack buffer usage is greatly reduced, particularly for
unbuffered input streams.  __printf_fp can still use a large buffer
in binary128 mode for %g, though.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-12-19 18:56:54 +01:00
Florian Weimer
659fe9fdd1 stdio-common: Introduce buffers for implementing printf
These buffers will eventually be used instead of FILE * objects
to implement printf functions.  The multibyte buffer is struct
__printf_buffer, the wide buffer is struct __wprintf_buffer.

To enable writing type-generic code, the header files
printf_buffer-char.h and printf_buffer-wchar_t.h define the
Xprintf macro differently, enabling Xprintf (buffer) to stand
for __printf_buffer and __wprintf_buffer as appropriate.  For
common cases, macros like Xprintf_buffer are provided as a more
syntactically convenient shortcut.

Buffer-specific flush callbacks are implemented with a switch
statement instead of a function pointer, to avoid hardening issues
similar to those of libio vtables.  struct __printf_buffer_as_file
is needed to support custom printf specifiers because the public
interface for that requires passing a FILE *, which is why there
is a trapdoor back from these buffers to FILE * streams.

Since the immediate user of these interfaces knows when processing
has finished, there is no flush callback for the end of processing,
only a flush callback for the intermediate buffer flush.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-12-19 18:56:54 +01:00
Florian Weimer
ee1ada1bdb elf: Rework exception handling in the dynamic loader [BZ #25486]
The old exception handling implementation used function interposition
to replace the dynamic loader implementation (no TLS support) with the
libc implementation (TLS support).  This results in problems if the
link order between the dynamic loader and libc is reversed (bug 25486).

The new implementation moves the entire implementation of the
exception handling functions back into the dynamic loader, using
THREAD_GETMEM and THREAD_SETMEM for thread-local data support.
These depends on Hurd support for these macros, added in commit
b65a82e4e7 ("hurd: Add THREAD_GET/SETMEM/_NC").

One small obstacle is that the exception handling facilities are used
before the TCB has been set up, so a check is needed if the TCB is
available.  If not, a regular global variable is used to store the
exception handling information.

Also rename dl-error.c to dl-catch.c, to avoid confusion with the
dlerror function.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2022-11-03 09:39:31 +01:00
H.J. Lu
bbff275fb1 elf: Build tst-relr-mod[34]a.so with $(LDFLAGS-rpath-ORIGIN)
When --enable-hardcoded-path-in-tests is used only with DT_RUNPATH,
elf/tst-relr3 and elf/tst-relr4 failed to run.  Their dependency
libraries, tst-relr-mod3a.so and tst-relr-mod4a.so, are failed to
load since DT_RUNPATH on executable doesn't apply to them.  Build
tst-relr-mod3a.so and tst-relr-mod4a.so with $(LDFLAGS-rpath-ORIGIN)
to add DT_RUNPATH for their dependency libraries.
Reviewed-by: Fangrui Song <maskray@google.com>
2022-10-31 09:00:26 -07:00
Florian Weimer
6f360366f7 elf: Introduce to _dl_call_fini
This consolidates the destructor invocations from _dl_fini and
dlclose.  Remove the micro-optimization that avoids
calling _dl_call_fini if they are no destructors (as dlclose is quite
expensive anyway).  The debug log message is now printed
unconditionally.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-10-27 11:36:44 +02:00
Adhemerval Zanella
5355f9ca7b elf: Remove -fno-tree-loop-distribute-patterns usage on dl-support
Besides the option being gcc specific, this approach is still fragile
and not future proof since we do not know if this will be the only
optimization option gcc will add that transforms loops to memset
(or any libcall).

This patch adds a new header, dl-symbol-redir-ifunc.h, that can b
used to redirect the compiler generated libcalls to port the generic
memset implementation if required.

Checked on x86_64-linux-gnu and aarch64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2022-10-10 10:32:28 -03:00
Javier Pello
6099908fb8 elf: Remove legacy hwcaps support from the dynamic loader
Remove support for the legacy hwcaps subdirectories from the dynamic
loader.

Signed-off-by: Javier Pello <devel@otheo.eu>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
2022-10-06 07:59:48 -03:00
Florian Weimer
d0e357ff45 elf: Call __libc_early_init for reused namespaces (bug 29528)
libc_map is never reset to NULL, neither during dlclose nor on a
dlopen call which reuses the namespace structure.  As a result, if a
namespace is reused, its libc is not initialized properly.  The most
visible result is a crash in the <ctype.h> functions.

To prevent similar bugs on namespace reuse from surfacing,
unconditionally initialize the chosen namespace to zero using memset.
2022-08-26 21:15:43 +02:00
Florian Weimer
89baed0b93 Revert "Detect ld.so and libc.so version inconsistency during startup"
This reverts commit 6f85dbf102.

Once this change hits the release branches, it will require relinking
of all statically linked applications before static dlopen works
again, for the majority of updates on release branches: The NEWS file
is regularly updated with bug references, so the __libc_early_init
suffix changes, and static dlopen cannot find the function anymore.

While this ABI check is still technically correct (we do require
rebuilding & relinking after glibc updates to keep static dlopen
working), it is too drastic for stable release branches.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-08-25 18:46:43 +02:00
Florian Weimer
6f85dbf102 Detect ld.so and libc.so version inconsistency during startup
The files NEWS, include/link.h, and sysdeps/generic/ldsodefs.h
contribute to the version fingerprint used for detection.  The
fingerprint can be further refined using the --with-extra-version-id
configure argument.

_dl_call_libc_early_init is replaced with _dl_lookup_libc_early_init.
The new function is used store a pointer to libc.so's
__libc_early_init function in the libc_map_early_init member of the
ld.so namespace structure.  This function pointer can then be called
directly, so the separate invocation function is no longer needed.

The versioned symbol lookup needs the symbol versioning data
structures, so the initialization of libc_map and libc_map_early_init
is now done from _dl_check_map_versions, after this information
becomes available.  (_dl_map_object_from_fd does not set this up
in time, so the initialization code had to be moved from there.)
This means that the separate initialization code can be removed from
dl_main because _dl_check_map_versions covers all maps, including
the initial executable loaded by the kernel.  The lookup still happens
before relocation and the invocation of IFUNC resolvers, so IFUNC
resolvers are protected from ABI mismatch.

The __libc_early_init function pointer is not protected because
so little code runs between the pointer write and the invocation
(only dynamic linker code and IFUNC resolvers).

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2022-08-24 17:35:57 +02:00
Florian Weimer
dd2315a866 elf: Run tst-audit-tlsdesc, tst-audit-tlsdesc-dlopen everywhere
The test is valid for all TLS models, but we want to make a reasonable
effort to test the GNU2 model specifically.  For example, aarch64
defaults to GNU2, but does not have -mtls-dialect=gnu2, and the test
was not run there.

Suggested-by: Martin Coufal <mcoufal@redhat.com>
2022-08-16 12:00:03 +02:00
Florian Weimer
9001cb1102 assert: Do not use stderr in libc-internal assert
Redirect internal assertion failures to __libc_assert_fail, based on
based on __libc_message, which writes directly to STDERR_FILENO
and calls abort.  Also disable message translation and reword the
error message slightly (adjusting stdlib/tst-bz20544 accordingly).

As a result of these changes, malloc no longer needs its own
redefinition of __assert_fail.

__libc_assert_fail needs to be stubbed out during rtld dependency
analysis because the rtld rebuilds turn __libc_assert_fail into
__assert_fail, which is unconditionally provided by elf/dl-minimal.c.

This change is not possible for the public assert macro and its
__assert_fail function because POSIX requires that the diagnostic
is written to stderr.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-08-03 11:43:04 +02:00