Commit Graph

37275 Commits

Author SHA1 Message Date
Florian Weimer
dde76856ba nptl: Re-sort Versions file
Due to an incorrect conflict resolution, libc/GLIBC_2.2 section was no
longer sorted lexicographically.
2021-05-03 14:12:55 +02:00
H.J. Lu
cf2c57526b x86: Set rep_movsb_threshold to 2112 on processors with FSRM
The glibc memcpy benchmark on Intel Core i7-1065G7 (Ice Lake) showed
that REP MOVSB became faster after 2112 bytes:

                                      Vector Move       REP MOVSB
length=2112, align1=0, align2=0:        24.20             24.40
length=2112, align1=1, align2=0:        26.07             23.13
length=2112, align1=0, align2=1:        27.18             28.13
length=2112, align1=1, align2=1:        26.23             25.16
length=2176, align1=0, align2=0:        23.18             22.52
length=2176, align1=2, align2=0:        25.45             22.52
length=2176, align1=0, align2=2:        27.14             27.82
length=2176, align1=2, align2=2:        22.73             25.56
length=2240, align1=0, align2=0:        24.62             24.25
length=2240, align1=3, align2=0:        29.77             27.15
length=2240, align1=0, align2=3:        35.55             29.93
length=2240, align1=3, align2=3:        34.49             25.15
length=2304, align1=0, align2=0:        34.75             26.64
length=2304, align1=4, align2=0:        32.09             22.63
length=2304, align1=0, align2=4:        28.43             31.24

Use REP MOVSB for data size > 2112 bytes in memcpy on processors with
fast short REP MOVSB (FSRM).

	* sysdeps/x86/dl-cacheinfo.h (dl_init_cacheinfo): Set
	rep_movsb_threshold to 2112 on processors with fast short REP
	MOVSB (FSRM).
2021-05-03 05:08:22 -07:00
H.J. Lu
98544f5bcf bench-memcpy: Collect data from 2KB to 4KB
Collect data on memcpy from 2KB to 4KB with the 64-byte increment value.
2021-05-03 05:08:22 -07:00
Alyssa Ross
b03e4d7bd2 stdio: fix vfscanf with matches longer than INT_MAX (bug 27650)
Patterns like %*[ can safely be used to match a great many characters,
and it's quite realisitic to use them for more than INT_MAX characters
from an IO stream.

With the previous approach, after INT_MAX characters (v)fscanf would
return successfully, indicating an end to the match, even though there
wasn't one.
2021-05-03 10:34:11 +02:00
Florian Weimer
c2fd60a586 nptl: Move pthread_yield into libc, as a compatibility symbol
And deprecate it in <pthread.h>, redirecting it to sched_yield
for the time being.

The symbol was moved using scripts/move-symbol-to-libc.py.

No GLIBC_2.34 symbol version is added because of the compatibility
symbol status.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:23:44 +02:00
Florian Weimer
0505ae4e3b nptl: Move pthread_rwlockattr_setpshared into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:45 +02:00
Florian Weimer
b444409479 nptl: Move pthread_rwlockattr_setkind_np into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:42 +02:00
Florian Weimer
dbfa12e201 nptl: Move pthread_rwlockattr_init into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:38 +02:00
Florian Weimer
0af0ccc2d5 nptl: Move pthread_rwlockattr_getpshared into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:35 +02:00
Florian Weimer
9f13a95b6f nptl: Move pthread_rwlockattr_getkind_np into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:31 +02:00
Florian Weimer
a8841e00d6 nptl: Move pthread_rwlockattr_destroy into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:25 +02:00
Florian Weimer
e8a9597159 nptl: Move pthread_rwlock_trywrlock into libc
And __pthread_rwlock_trywrlock as a compatibility symbol.

Remove the unused __libc_rwlock_trywrlock macro.

The symbols were moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:20 +02:00
Florian Weimer
c96dddd731 nptl: Move pthread_rwlock_tryrdlock into libc
And __pthread_rwlock_tryrdlock as a compatibility symbol.

Remove the unused __libc_rwlock_tryrdlock macro.

The symbols were moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:16 +02:00
Florian Weimer
e7d2d48e56 nptl: Move pthread_rwlock_timedwrlock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:12 +02:00
Florian Weimer
2392feb171 nptl: Move pthread_rwlock_timedrdlock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:18:04 +02:00
Florian Weimer
63627246f3 nptl: Move pthread_rwlock_init into libc
And __pthread_rwlock_init as a compatibility symbol.

__libc_rwlock_init is changed to call __pthread_rwlock_init directly.

The symbols were moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:17:59 +02:00
Florian Weimer
504ac633b0 nptl: Move pthread_rwlock_destroy into libc
And __pthread_rwlock_destroy as a compatibility symbol.

rwlocks do not need finalization, so change __libc_rwlock_fini to do
nothing.

The symbols were moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:17:36 +02:00
Florian Weimer
2cf5b43bcb nptl: Move pthread_rwlock_clockwrlock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:17:27 +02:00
Florian Weimer
0ace9b197c nptl: Move pthread_rwlock_clockrdlock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 09:16:49 +02:00
Florian Weimer
2c5c5c87c9 nptl: Move tss_set into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

__pthread_setspecific@@GLIBC_2.34 is no longer needed after the move,
so it is removed with this commit, too.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:18:08 +02:00
Florian Weimer
a062ba3836 nptl: Move tss_get into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

__pthread_getspecific@@GLIBC_2.34 is no longer needed after the move,
so it is removed with this commit, too.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:18:01 +02:00
Florian Weimer
deb317d510 nptl: Move tss_delete into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

__pthread_key_delete@@GLIBC_PRIVATE is no longer needed after that,
so it is removed as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:17:38 +02:00
Florian Weimer
e4036b1ce9 nptl: Move tss_create into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

__pthread_key_create@@GLIBC_2.34 is no longer needed by glibc
itself with this change, but __pthread_key_create is used by
libstdc++, so it still has to be exported as a public symbol.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:17:29 +02:00
Florian Weimer
96e61709b4 nptl: Move thrd_exit into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_exit@@GLIBC_PRIVATE symbol is no longer needed
after this change, so remove it.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:17:15 +02:00
Florian Weimer
ae4a5ca074 nptl: Move mtx_unlock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

__pthread_mutex_unlock@GLIBC_2.34 is not removed in this commit
because it is still used from nptl/nptl-init.c.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:16:48 +02:00
Florian Weimer
08970190fc nptl: Move mtx_trylock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_mutex_trylock@@GLIBC_2.34 symbol version is no longer
needed because the call is now internal to libc, so remove it with
this commit.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:58 +02:00
Florian Weimer
032a9e17a7 nptl: Move mtx_timedlock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_mutex_timedlock@@GLIBC_PRIVATE export is no longer
needed, so it is removed with this commit.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:53 +02:00
Florian Weimer
b7863c732e nptl: Move mtx_lock into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

__pthread_mutex_lock@GLIBC_2.34 is not removed in this commit
because it is still used from nptl/nptl-init.c.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:41 +02:00
Florian Weimer
f44c9a24ed nptl: Move mtx_init into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The calls to __pthread_mutex_init, __pthread_mutexattr_init,
__pthread_mutexattr_settype are now private and no longer need
to be exported.  This allows the removal of the newly added
GLIBC_2.34 symbol versions for those functions.

Also clean up some weak declarations in <libc-lockP.h> for
these functions.  They are not needed and potentially incorrect
for static linking of mtx_init.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:22 +02:00
Florian Weimer
17b67416c5 nptl: Move mtx_destroy into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_mutex_destroy@@GLIBC_2.34 symbol is no longer
neded because this commit makes __pthread_mutex_destroy@GLIBC_2.0
a compatibility symbol, so remove the new symbol version.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:15 +02:00
Florian Weimer
cf3fff1c19 nptl: Move cnd_wait into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_cond_wait@@GLIBC_PRIVATE symbol is no longer
neded, so remove that as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:10 +02:00
Florian Weimer
08a31ef923 nptl: Move cnd_timedwait into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_cond_timedwait@@GLIBC_PRIVATE symbol is no longer
neded, so remove that as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:15:04 +02:00
Florian Weimer
eef936eb45 nptl: Move cnd_signal into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_cond_signal@@GLIBC_PRIVATE symbol is no longer
neded, so remove that as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:14:49 +02:00
Florian Weimer
3eab466e56 nptl: Move cnd_init into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_cond_init@@GLIBC_PRIVATE symbol is no longer
neded, so remove that as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:14:38 +02:00
Florian Weimer
f11c293a4c nptl: Move cnd_destroy into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_cond_destroy@@GLIBC_PRIVATE symbol is no longer
neded, so remove that as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:14:28 +02:00
Florian Weimer
0431f171da nptl: Move cnd_broadcast into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

The __pthread_cond_broadcast@@GLIBC_PRIVATE symbol is no longer
neded, so remove that as well.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:14:17 +02:00
Florian Weimer
575295fc83 nptl: Move call_once into libc
The symbol was moved using scripts/move-symbol-to-libc.py.

This change also turns __pthread_once into a compatibility symbol
because after the call_once move, an internal call to __pthread_once
can be used.  This an adjustment to __libc_once: Outside libc (e.g.,
in nscd), it has to call pthread_once.  With __pthread_once as a
compatibility symbol, it is no longer to add a new GLIBC_2.34
version after the move from libpthread, and this commit removes
the new __pthread_once@@GLIBC_2.34 version.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:14:01 +02:00
Florian Weimer
ad201afe5c Add pthread-in-libc, libpthread-routines-var, librt-routines-var
These make variables can be used to add routines to different
libraries for the Hurd and Linux builds.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-05-03 08:13:32 +02:00
Raoni Fassina Firmino
23fdf8178c powerpc64le: Optimize memset for POWER10
This implementation is based on __memset_power8 and integrates a lot
of suggestions from Anton Blanchard.

The biggest difference is that it makes extensive use of stxvl to
alignment and tail code to avoid branches and small stores.  It has
three main execution paths:

a) "Short lengths" for lengths up to 64 bytes, avoiding as many
   branches as possible.

b) "General case" for larger lengths, it has an alignment section
   using stxvl to avoid branches, a 128 bytes loop and then a tail
   code, again using stxvl with few branches.

c) "Zeroing cache blocks" for lengths from 256 bytes upwards and set
   value being zero.  It is mostly the __memset_power8 code but the
   alignment phase was simplified because, at this point, address is
   already 16-bytes aligned and also changed to use vector stores.
   The tail code was also simplified to reuse the general case tail.

All unaligned stores use stxvl instructions that do not generate
alignment interrupts on POWER10, making it safe to use on
caching-inhibited memory.

On average, this implementation provides something around 30%
improvement when compared to __memset_power8.

Reviewed-by: Matheus Castanho <msc@linux.ibm.com>
Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
2021-04-30 18:12:08 -03:00
Tulio Magno Quites Machado Filho
e941e0ae80 powerpc64le: Optimize memcpy for POWER10
This implementation is based on __memcpy_power8_cached and integrates
suggestions from Anton Blanchard.
It benefits from loads and stores with length for short lengths and for
tail code, simplifying the code.

All unaligned memory accesses use instructions that do not generate
alignment interrupts on POWER10, making it safe to use on
caching-inhibited memory.

The main loop has also been modified in order to increase instruction
throughput by reducing the dependency on updates from previous iterations.

On average, this implementation provides around 30% improvement when
compared to __memcpy_power7 and 10% improvement in comparison to
__memcpy_power8_cached.
2021-04-30 18:12:08 -03:00
Lucas A. M. Magalhaes
dd59655e93 powerpc64le: Optimized memmove for POWER10
This patch was initially based on the __memmove_power7 with some ideas
from strncpy implementation for Power 9.

Improvements from __memmove_power7:

1. Use lxvl/stxvl for alignment code.

   The code for Power 7 uses branches when the input is not naturally
   aligned to the width of a vector. The new implementation uses
   lxvl/stxvl instead which reduces pressure on GPRs. It also allows
   the removal of branch instructions, implicitly removing branch stalls
   and mispredictions.

2. Use of lxv/stxv and lxvl/stxvl pair is safe to use on Cache Inhibited
   memory.

   On Power 10 vector load and stores are safe to use on CI memory for
   addresses unaligned to 16B. This code takes advantage of this to
   do unaligned loads.

   The unaligned loads don't have a significant performance impact by
   themselves. However doing so decreases register pressure on GPRs
   and interdependence stalls on load/store pairs. This also improved
   readability as there are now less code paths for different alignments.
   Finally this reduces the overall code size.

3. Improved performance.

   This version runs on average about 30% better than memmove_power7
   for lengths  larger than 8KB. For input lengths shorter than 8KB
   the improvement is smaller, it has on average about 17% better
   performance.

   This version has a degradation of about 50% for input lengths
   in the 0 to 31 bytes range when dest is unaligned.

Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
2021-04-30 18:12:08 -03:00
Joseph Myers
e046d73e5f Update kernel version to 5.12 in tst-mman-consts.py.
This patch updates the kernel version in the test tst-mman-consts.py
to 5.12.  (There are no new MAP_* constants covered by this test in
5.12 that need any other header changes.)

Tested with build-many-glibcs.py.
2021-04-29 10:50:11 +00:00
Joseph Myers
4a7c342605 Update syscall lists for Linux 5.12.
Linux 5.12 has one new syscall, mount_setattr.  Update
syscall-names.list and regenerate the arch-syscall.h headers with
build-many-glibcs.py update-syscalls.

Tested with build-many-glibcs.py.
2021-04-28 17:19:24 +00:00
Paul Zimmermann
e6eef0adc5 regenerate ulps on x86_64 with -march=native
On x86_64, when configuring glibc with CFLAGS="-O2 -g -march=native",
some tests fail. After this patch, "make check" succeeds.

Tested on Intel Core i5-4590 with gcc 10.2.1.
2021-04-28 12:46:00 +02:00
Martin Sebor
b25b067491 Pass a valid pointer to pthread_setspecific to avoid GCC 11 warning. 2021-04-27 19:07:49 -06:00
Martin Sebor
a1561c3bbe Add __attribute_access_none to disable GCC warnings [BZ #27714]
GCC 11 warns when a pointer to an uninitialized object is passed
to a function that takes a const-qualified argument.  This is done
on the assumption that most such functions read from the object.
For the rare case of a function that doesn't, GCC 11 extends
attribute access to add a new mode called none.

POSIX pthread_setspecific() is one such rare function that takes
a const void* argument but that doesn't read from the object it
points to.  To suppress the -Wmaybe-uninitialized issued by GCC
11 when the address of an uninitialized object is passed to it
(e.g., the result of malloc()), this change #defines
__attr_access_none in cdefs.h and uses the macro on the function
in sysdeps/htl/pthread.h and sysdeps/nptl/pthread.h.
2021-04-27 13:01:55 -06:00
Joseph Myers
876cdfd154 Use Linux 5.12 and GCC 11 branch in build-many-glibcs.py.
This patch makes build-many-glibcs.py use Linux 5.12 and GCC 11
branch.

Tested with build-many-glibcs.py (host-libraries, compilers and glibcs
builds).
2021-04-27 15:19:08 +00:00
Raphael Moreira Zinsly
25cb72820a powerpc: Add log IFUNC multiarch support for POWER10
Checked on ppc64le built without --with-cpu, with --with-cpu=power9
and with --disable-multi-arch.

Reviewed-by: Matheus Castanho <msc@linux.ibm.com>
2021-04-26 10:10:29 -03:00
Hanataka Shinya
82292c99b2 LC_COLLATE: Fix last character ellipsis handling (Bug 22668)
During ellipsis processing the collation cursor was not correctly
moved to the end of the ellipsis after processing.

The code inserted the new entry after the cursor, but before the
real end of the ellipsis:
                                [cursor]
... element_t <-> element_t <-> element_t <-> element_t
                  "<U0000>"     "<U0001>"     "<U007F>"
                  startp                      endp

At the end of the function we have:

                  [cursor]
... element_t <-> element_t <-> element_t
                  "<U007E>"     "<U007F>"
                                endp

The cursor should be pointing at endp, the last element in the
doubly-linked list, otherwise when execution returns to the
caller we will start inserting the next line after <U007E>.

Subsequent operations end up unlinking the ellipsis end entry or
just leaving it in the list dangling from the end.  This kind of
dangling is immediately visible in C.UTF-8 with the following
sorting from strcoll:
<U0010FFFF>
<U0000FFFF>
<U000007FF>
<U0000007F>

With the cursor correctly adjusted the end entry is correctly given
the right location and thus the right weight.

Retested and no regressions on x86_64 and i686.

Co-authored-by: Carlos O'Donell <carlos@redhat.com>
2021-04-26 08:03:32 -04:00
Noah Goldstein
7f3e7c262c x86: Optimize strchr-evex.S
No bug. This commit optimizes strchr-evex.S. The optimizations are
mostly small things such as save an ALU in the alignment process,
saving a few instructions in the loop return. The one significant
change is saving 2 instructions in the 4x loop. test-strchr,
test-strchrnul, test-wcschr, and test-wcschrnul are all passing.

Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
2021-04-25 10:04:39 -07:00