Current 'non_temporal_threshold' set to 'non_temporal_threshold_lowbound'
on Zhaoxin processors without ERMS. The default
'non_temporal_threshold_lowbound' is too small for the KH-40000 and KX-7000
Zhaoxin processors, this patch updates the value to
'shared / cachesize_non_temporal_divisor'.
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
This patch optimizes large size copy using normal store when src > dst
and overlap. Make it the same as the logic in memmove-vec-unaligned-erms.S.
Current memmove-ssse3 use '__x86_shared_cache_size_half' as the non-
temporal threshold, this patch updates that value to
'__x86_shared_non_temporal_threshold'. Currently, the
__x86_shared_non_temporal_threshold is cpu-specific, and different CPUs
will have different values based on the related nt-benchmark results.
However, in memmove-ssse3, the nontemporal threshold uses
'__x86_shared_cache_size_half', which sounds unreasonable.
The performance is not changed drastically although shows overall
improvements without any major regressions or gains.
Results on Zhaoxin KX-7000:
bench-memcpy geometric_mean(N=20) New / Original: 0.999
bench-memcpy-random geometric_mean(N=20) New / Original: 0.999
bench-memcpy-large geometric_mean(N=20) New / Original: 0.978
bench-memmove geometric_mean(N=20) New / Original: 1.000
bench-memmmove-large geometric_mean(N=20) New / Original: 0.962
Results on Intel Core i5-6600K:
bench-memcpy geometric_mean(N=20) New / Original: 1.001
bench-memcpy-random geometric_mean(N=20) New / Original: 0.999
bench-memcpy-large geometric_mean(N=20) New / Original: 1.001
bench-memmove geometric_mean(N=20) New / Original: 0.995
bench-memmmove-large geometric_mean(N=20) New / Original: 0.936
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Fix code formatting under the Zhaoxin branch and add comments for
different Zhaoxin models.
Unaligned AVX load are slower on KH-40000 and KX-7000, so disable
the AVX_Fast_Unaligned_Load.
Enable Prefer_No_VZEROUPPER and Fast_Unaligned_Load features to
use sse2_unaligned version of memset,strcpy and strcat.
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Qualcom's new core, oryon-1, has a different characteristics for
memset than the current versions of memset. For non-zero, larger
sizes, using GPRs rather than the SIMD stores is ~30% faster.
For even larger sizes, using the nontemporal stores is needed
not to polute the L1/L2 caches.
For zero values, using `dc zva` should be used. Since we
know the size will always be 64 bytes, we don't need to figure
out the size there.
I started with the emag memset and added back the `dc zva` code.
Changes since v1:
* v3: Fix comment formating
Signed-off-by: Andrew Pinski <quic_apinski@quicinc.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Qualcomm's new core (oryon-1) has a different performance characteristic
than other cores. For memcpy, it is faster to use the GPRs to
do the copy for large sizes (2x faster). For even larger sizes,
it is better to use the nontemporal load/store instructions so
we don't pollute the L1/L2 caches.
For smaller sizes, the characteristic are very similar to
other cores.
I used the thunderx memcpy as a starting point and expanded from there.
Changes since v1:
* v2: Fix ordering in Makefile.
* v3: Fix comment grammar about the ldnp/stnp instructions.
Signed-off-by: Andrew Pinski <quic_apinski@quicinc.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The fcntl.h fortify wrapper for clang added by 86889e22db
missed the __fortify_clang_overload_arg and and also added the
mode argument for the __fortify_function_error_function function,
which leads clang to be able to correct resolve which overloaded
function it should emit.
Checked on x86_64-linux-gnu.
Reported-by: Khem Raj <raj.khem@gmail.com>
Tested-by: Khem Raj <raj.khem@gmail.com>
The mqueue.h fortify wrapper for clang added by c23107effb
is not fully correct, where correct 4 argument usage are not
being correctly handled. For instance, while building socat 1.8
with a yocto clang based system shows:
./socat-1.8.0.0/xio-posixmq.c:119:8: error: 'mq_open' is unavailable: mq_open can be called either with 2 or 4 arguments
119 | mqd = mq_open(name, oflag, opt_mode, NULL);
| ^
[...] /usr/include/bits/mqueue2.h:66:8: note: 'mq_open' has been explicitly marked unavailable here
66 | __NTH (mq_open (const char *__name, int __oflag, mode_t mode,
| ^
1 error generated.
The correct way to define the wrapper is to set invalid usage
with __fortify_clang_unavailable (for the case with 5 or more
arguments), followed by the expected ones. This fix make mq_open
similar to current open wrappers.
[1] http://www.dest-unreach.org/socat/
Reported-by: Khem Raj <raj.khem@gmail.com>
Acked-by: Khem Raj <raj.khem@gmail.com>
With gcc 14, I get this warning/werror when building the localedata tests:
tests-mbwc/tsp_common.c: In function ‘result.constprop.isra’:
tests-mbwc/tsp_common.c:55:43: error: ‘%s’ directive writing up to 92 bytes into a region of size between 0 and 114 [-Werror=format-overflow=]
55 | sprintf (result_rec, "%s:%s:%d:%d:%d:%c:%s\n", func, loc, rec_no, seq_no,
| ^~
In file included from ../include/bits/stdio2.h:1,
from ../libio/stdio.h:980,
from ../include/stdio.h:14,
from tests-mbwc/tsp_common.c:10:
In function ‘sprintf’,
inlined from ‘result.constprop.isra’ at tests-mbwc/tsp_common.c:55:3:
../libio/bits/stdio2.h:30:10: note: ‘__builtin___sprintf_chk’ output between 20 and 234 bytes into a destination of size 132
30 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
31 | __glibc_objsize (__s), __fmt,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
32 | __va_arg_pack ());
| ~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
This patch now gets rid of using sprintf and the result_rec buffer and just
prints to fp directly.
The test does not necessarily trigger the crash, depending on memcmp
behavior. A crash was observed in __memcmp_ia32 on i686 builds.
Reviewed-by: Paul Eggert <eggert@cs.ucla.edu>
* manual/string.texi: For strnlen (s, maxlen), do not say that s must
be of size maxlen, as it can be smaller if it is null-terminated.
This should help avoid confusion such as seen in
<https://lists.gnu.org/r/bug-gnulib/2024-06/msg00280.html>.
Mention that strnlen and wcsnlen have been in POSIX since
POSIX.1-2008.
This recently came up during a cleanup to remove misaligned accesses
from the RISC-V port.
Link: https://sourceware.org/pipermail/libc-alpha/2022-June/139961.html
Suggested-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Reviewed-by: Fangrui Song <maskray@google.com>
asm volatile ("movfcsr2gr $t0, $fcsr0" ::: "$t0");
asm volatile ("st.d $t0, %0" :"=m"(restore_fcsr));
generate to the following instructions with -Og flag:
movfcsr2gr $t0, $zero
addi.d $t0, $sp, 2047(0x7ff)
addi.d $t0, $t0, 77(0x4d)
st.w $t0, $t0, 0
fcsr0 register and restore_fcsr variable are both stored in t0 register.
Change to:
asm volatile ("movfcsr2gr %0, $fcsr0" :"=r"(restore_fcsr));
to avoid restore_fcsr address in t0.
Comparing float value using memcmp because float value cannot be
directly compared for equality.
Put LOAD_REGISTER_FCSR and SAVE_REGISTER_FCC after LOAD_REGISTER_FLOAT.
Some float instructions may change fcsr register.
If the pidfd_spawn/pidfd_spawnp helper process succeeds, but evecve
fails for some reason (either with an invalid/non-existent, memory
allocation, etc.) the resulting pidfd is never closed, nor returned
to caller (so it can call close).
Since the process creation failed, it should be up to posix_spawn to
also, close the file descriptor in this case (similar to what it
does to reap the process).
This patch also changes the waitpid with waitid (P_PIDFD) for pidfd
case, to avoid a possible pid re-use.
Checked on x86_64-linux-gnu.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The atomic_spin_nop() macro can be used to run arch-specific
code in the body of a spin loop to potentially improve efficiency.
RISC-V's Zihintpause extension includes a PAUSE instruction for
this use-case, which is encoded as a HINT, which means that it
behaves like a NOP on systems that don't implement Zihintpause.
Binutils supports Zihintpause since 2.36, so this patch uses
the ".insn" directive to keep the code compatible with older
toolchains.
Signed-off-by: Christoph Müllner <christoph.muellner@vrull.eu>
Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
MIPSr6 has MADDF.s/MADDF.d instructions, which are fused.
In MIPS ISA, double support can be subsetted. Only FMAF is enabled
for this case.
* sysdeps/mips/fpu/math-use-builtins-fma.h
Signed-off-by: YunQiang Su <syq@gcc.gnu.org>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
The upcoming parisc (hppa) v6.11 Linux kernel will include vDSO
support for gettimeofday(), clock_gettime() and clock_gettime64()
syscalls for 32- and 64-bit userspace.
The patch below adds the necessary glue code for glibc.
Signed-off-by: Helge Deller <deller@gmx.de>
Changes in v2:
- add vsyscalls for 64-bit too
The walk benchmarks don't measure anything useful - memory is not initialized
properly so doing a single walk in 32MB just measures reading the 4KB zero
page for reads and clear_page overhead for writes. The memset variants don't
even manage to do a walk in the 32MB region due to using incorrect pointer
increments... Neither is it clear why it is walking backwards since this
won't confuse modern prefetchers. If you fix the benchmark and print the
bandwidth, the results are identical for all sizes larger than ~1KB since it
is just testing memory bandwidth of a single 32MB block. This case is already
tested by the large benchmark, so overall it doesn't seem useful to keep these.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The previous version expanded $0 and $@ twice.
The new version defines a q no-op shell command. The Perl syntax
error is masked by the eval Perl function. The q { … } construct
is executed by the shell without errors because the q shell function
was defined, but treated as a non-expanding quoted string by Perl,
effectively hiding its context from the Perl interpreter. As before
the script is read by require instead of executed directly, to avoid
infinite recursion because the #! line contains /bin/sh.
Introduce the “fatal” function to produce diagnostics that are not
suppressed by “do”. Use “do” instead of “require” because it has
fewer requirements on the executed script than “require”.
Prefix relative paths with './' because “do” (and “require“ before)
searches for the script in @INC if the path is relative and does not
start with './'. Use $_ to make the trampoline shorter.
Add an Emacs mode marker to indentify the script as a Perl script.
The test is not a run-time check, so update the description.
Also use readelf -W for a more stable output format and fix
an LC_ALL typo.
This avoids garbled configure messages:
checking for s390-specific static PIE requirements (runtime check)... 0x0000000000000017 (JMPREL) 0x280
yes
Based on a -march=x86-64-v4 -mfpmath=sse build, with and without
--disable-multi-arch, running on a Zen 4 CPU. Also used different
-march=x8i6-64-v… settings.
Generation of the Perl script does not depend on Perl, so we can
always install it even if $(PERL) is not set during the build.
Change the malloc/mtrace.pl text substition not to rely on $(PERL).
Instead use PATH at run time to find the Perl interpreter. The Perl
interpreter cannot execute directly a script that starts with
“#! /bin/sh”: it always executes it with /bin/sh. There is no
perl command line switch to disable this behavior. Instead, use
the Perl require function to execute the script. The additional
shift calls remove the “.” shell arguments. Perl interprets the
“.” as a string concatenation operator, making the expression
syntactically valid.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
HWCAP value is overwritten at the first comparison of the LASX case.
The second comparison at LSX get incorrect result.
Change to use t0 to save HWCAP value, and use t1 to save comparison
result.
The _dl_sysdep_parse_arguments function contains initalization
of a large on-stack variable:
dl_parse_auxv_t auxv_values = { 0, };
This uses a non-inline version of memset on powerpc64le-linux-gnu,
so it must use the baseline memset.
A desired hugetlb page size can be encoded in the flags parameter of
system calls such as mmap() and shmget(). The Linux UAPI headers have
included explicit definitions for these encodings since v4.14.
This patch adds these definitions that are used along with MAP_HUGETLB
and SHM_HUGETLB flags as specified in the corresponding man pages. This
relieves programs from having to duplicate and/or compute the encodings
manually.
Additionally, the filter on these definitions in tst-mman-consts.py is
removed, as suggested by Florian. I then ran this tests successfully,
confirming the alignment with the kernel headers.
PASS: misc/tst-mman-consts
original exit status 0
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Tested-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Remove the definitions of HWCAP_IMPORTANT after removal of
LD_HWCAP_MASK / tunable glibc.cpu.hwcap_mask. There HWCAP_IMPORTANT
was used as default value.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Remove the environment variable LD_HWCAP_MASK and the tunable
glibc.cpu.hwcap_mask as those are not used anymore in common-code
after removal in elf/dl-cache.c:search_cache().
The only remaining user is sparc32 where it is used in
elf_machine_matches_host(). If sparc32 does not need it anymore,
we can get rid of it at all. Otherwise we could also move
LD_HWCAP_MASK / tunable glibc.cpu.hwcap_mask to be sparc32 specific.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Remove the definitions of _DL_PLATFORMS_COUNT as those are not used
anymore after removal in elf/dl-cache.c:search_cache().
Note: On x86, we can also get rid of the definitions
HWCAP_PLATFORMS_START and HWCAP_PLATFORMS_COUNT.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Remove the definitions of _DL_FIRST_PLATFORM as those were only used
in the _DL_HWCAP_PLATFORM definitions and in _dl_string_platform().
Both were removed.
Note: Removed on every architecture despite of powerpc, where
_dl_string_platform() is still used.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Remove the definitions of _DL_HWCAP_PLATFORM as those are not used
anymore after removal in elf/dl-cache.c:search_cache().
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Remove the platform strings in dl-procinfo.c where also
the implementation of _dl_string_platform() was removed.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Despite of powerpc where the returned integer is stored in tcb,
and the diagnostics output, there is no user anymore.
Thus this patch removes the diagnostics output and
_dl_string_platform for all other platforms.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The legacy hwcaps mechanism was removed with glibc 2.37:
See this commit series:
- d178c67535
x86_64: Remove platform directory library loading test
- 6099908fb8
elf: Remove legacy hwcaps support from the dynamic loader
- b78ff5a25d
elf: Remove legacy hwcaps support from ldconfig
- 4a7094119c
elf: Remove hwcap parameter from add_to_cache signature
- cfbf883db3
elf: Remove hwcap and bits_hwcap fields from struct cache_entry
- 78d9a1620b
Add NEWS entry for legacy hwcaps removal
- ab40f20364
elf: Remove _dl_string_hwcap
- e76369ed63
elf: Simplify output of hwcap subdirectories in ld.so help
According to Florian Weimer, this was an oversight and should also
have been removed.
As ldconfig does not generate ld.so.cache entries with hwcap/platform
bits in the hwcap-field anymore, this patch now skips those entries.
Thus currently only named-hwcap-entries and the default entries are
allowed.
For named-hwcap entries bit 62 is set and also the isa-level bits can
be set.
For the default entries the hwcap-field is 0.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
