Support added to identify Sparc M7/T7/S7/M8/T8 processor capability.
Performance tests run on Sparc S7 using new code and old niagara4 code.
Optimizations for memset also apply to bzero as they share code.
For memset/bzero, performance comparison with niagara4 code:
For memset nonzero data,
256-1023 bytes - 60-90% gain (in cache); 5% gain (out of cache)
1K+ bytes - 80-260% gain (in cache); 40-80% gain (out of cache)
For memset zero data (and bzero),
256-1023 bytes - 80-120% gain (in cache), 0% gain (out of cache)
1024+ bytes - 2-4x gain (in cache), 10-35% gain (out of cache)
Tested in sparcv9-*-* and sparc64-*-* targets in both multi and
non-multi arch configurations.
Patrick McGehearty <patrick.mcgehearty@oracle.com>
Adhemerval Zanella <adhemerval.zanella@linaro.org>
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile
(sysdeps_routines): Add memset-niagara7.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdes_rotuines):
Likewise.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memset-niagara7.S: New
file.
* sysdeps/sparc/sparc64/multiarch/memset-niagara7.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add __bzero_niagara7 and __memset_niagara7.
* sysdeps/sparc/sparc64/multiarch/ifunc-memset.h (IFUNC_SELECTOR):
Add niagara7 option.
* NEWS: Mention sparc m7 optimized memcpy, mempcpy, memmove, and
memset.
Support added to identify Sparc M7/T7/S7/M8/T8 processor capability.
Performance tests run on Sparc S7 using new code and old niagara4 code.
Optimizations for memcpy also apply to mempcpy and memmove
where they share code. Optimizations for memset also apply
to bzero as they share code.
For memcpy/mempcpy/memmove, performance comparison with niagara4 code:
Long word aligned data
0-127 bytes - minimal changes
128-1023 bytes - 7-30% gain
1024+ bytes - 1-7% gain (in cache); 30-100% gain (out of cache)
Word aligned data
0-127 bytes - 50%+ gain
128-1023 bytes - 10-200% gain
1024+ bytes - 0-15% gain (in cache); 5-50% gain (out of cache)
Unaligned data
0-127 bytes - 0-70%+ gain
128-447 bytes - 40-80%+ gain
448-511 bytes - 1-3% loss
512-4096 bytes - 2-3% gain (in cache); 0-20% gain (out of cache)
4096+ bytes - ± 3% (in cache); 20-50% gain (out of cache)
Tested in sparcv9-*-* and sparc64-*-* targets in both multi and
non-multi arch configurations.
Patrick McGehearty <patrick.mcgehearty@oracle.com>
Adhemerval Zanella <adhemerval.zanella@linaro.org>
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile
(sysdeps_routines): Add memcpy-memmove-niagara7 and memmove-ultra1.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdeps_routines):
Likewise.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-memmove-niagara7.S:
New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memmove-ultra1.S: Likewise.
* sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-memmove.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add __memcpy_niagara7, __mempcpy_niagara7,
and __memmove_niagara7.
* sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h (IFUNC_SELECTOR):
Add niagara7 option.
* sysdeps/sparc/sparc64/multiarch/memmove.c: New file.
* sysdeps/sparc/sparc64/multiarch/ifunc-memmove.h: Likewise.
* sysdeps/sparc/sparc64/multiarch/memcpy-memmove-niagara7.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/memmove-ultra1.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/rtld-memmove.c: Likewise.
This patch refactors the sparc64 ifunc selector to a C implementation.
Also, the generic symbol is moved to its own implementation file
add_n-generic.S).
Checked on sparc64-linux-gnu and sparcv9-linux-gnu.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdep_routines):
Add add_n-generic.
* sysdeps/sparc/sparc64/multiarch/add_n-generic.S: New file.
* sysdeps/sparc/sparc64/multiarch/add_n.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/add_n.S: Remove file.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch refactors the sparc64 ifunc selector to a C implementation.
Also, the generic symbol is moved to its own implementation file
submul_1-generic.S).
Checked on sparc64-linux-gnu and sparcv9-linux-gnu.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdep_routines):
Add submul_1-generic.
* sysdeps/sparc/sparc64/multiarch/submul_1-generic.S: New file.
* sysdeps/sparc/sparc64/multiarch/submul_1.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/submul_1.S: Remove file.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch refactors the sparc64 ifunc selector to a C implementation.
Also, the generic symbol is moved to its own implementation file
addmul_1-generic.S).
Checked on sparc64-linux-gnu and sparcv9-linux-gnu.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdep_routines):
Add addmul_1-generic.
* sysdeps/sparc/sparc64/multiarch/addmul_1-generic.S: New file.
* sysdeps/sparc/sparc64/multiarch/addmul_1.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/addmul_1.S: Remove file.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch refactors the sparc64 ifunc selector to a C implementation.
Also, the generic symbol is moved to its own implementation file
sub_n-generic.S).
Checked on sparc64-linux-gnu and sparcv9-linux-gnu.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdep_routines):
Add sub_n-generic.
* sysdeps/sparc/sparc64/multiarch/sub_n-generic.S: New file.
* sysdeps/sparc/sparc64/multiarch/sub_n.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/sub_n.S: Remove file.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch refactors the sparc64 ifunc selector to a C implementation.
Also, the generic symbol is moved to its own implementation file
mul_1-generic.S).
Checked on sparc64-linux-gnu and sparcv9-linux-gnu.
* sysdeps/sparc/sparc64/multiarch/Makefile (sysdep_routines):
Add mul_1-generic.
* sysdeps/sparc/sparc64/multiarch/mul_1-generic.S: New file.
* sysdeps/sparc/sparc64/multiarch/mul_1.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/mul_1.S: Remove file.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch refactor the SPARC64 ifunc selector to a C implementation.
The x86_64 implementation is used as default, which resulted in common
definitions (ifunc-init.h) used on both architectures. No functional
change is expected, including ifunc resolution rules.
Checked on sparc64-linux-gnu, sparcv9-linux-gnu and x86_64-linux-gnu.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-ultra1.S: New
file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c: Likewise.
* sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h: Likewise.
* sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/memcpy.c: Likewise.
* sysdeps/sparc/sparc64/multiarch/mempcpy.c: Likewise.
* sysdeps/sparc/sparc-ifunc.h (sparc_libc_ifunc_redirected): New
macro.
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile
[$(subdir) = string] (sysdep_routines): Add memcpy-ultra1.
* sysdeps/sparc/sparc64/multiarch/Makefile [$(subdir) = string]
(sysdep_routines): Add memcpy-ultra1.
* sysdeps/sparc/sparc64/multiarch/memcpy.S: Remove file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S: Likewise.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
[BZ #16150]
* sysdeps/sparc/sparc64/multiarch/add_n.S: Resolve to the correct generic
symbol in the non-vis3 case in static builds.
* sysdeps/sparc/sparc64/multiarch/addmul_1.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/mul_1.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/sub_n.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/submul_1.S: Likewise.
* math/Makefile: Recognize gmp-sysdep_routines.
* sysdeps/sparc/sparc64/multiarch/Makefile: Add VIS3 optimized GMP routines
to sysdeps.
* sysdeps/sparc/sparc64/multiarch/add_n-vis3.S: New file.
* sysdeps/sparc/sparc64/multiarch/add_n.S: New file.
* sysdeps/sparc/sparc64/multiarch/addmul_1-vis3.S: New file.
* sysdeps/sparc/sparc64/multiarch/addmul_1.S: New file.
* sysdeps/sparc/sparc64/multiarch/mul_1-vis3.S: New file.
* sysdeps/sparc/sparc64/multiarch/mul_1.S: New file.
* sysdeps/sparc/sparc64/multiarch/sub_n-vis3.S: New file.
* sysdeps/sparc/sparc64/multiarch/sub_n.S: New file.
* sysdeps/sparc/sparc64/multiarch/submul_1-vis3.S: New file.
* sysdeps/sparc/sparc64/multiarch/submul_1.S: New file.
* crypt/Makefile: Move test targets after toplevel Rules
inclusion. Grab any necessary sysdep routines when linking.
* crypt/md5.c (md5_process_block): Remove define, we will always
name it __md5_process_block.
(md5_finish_ctx): Update md5_process_block call.
(md5_stream): Likewise.
(md5_process_bytes): Likewise.
(md5_process_block): Rename to __md5_process_block and move to ...
* crypt/md5-block.c: ... here.
* crypt/sha256.c (sha256_process_block): Move to ...
* crypt/sha256-block.c: ... here.
* crypt/sha512.c (sha512_process_block): Move to ...
* crypt/sha512-block.c: ... here.
* locale/Makefile (CFLAGS-md5.c): Define to add crypt/ to include
path.
* sysdeps/sparc/sparc-ifunc.c (sparc_libc_ifunc): Define.
* sysdeps/sparc/sparc64/multiarch/Makefile
(libcrypt-sysdep_routines): Add crypto assembler sysdeps when in
crypt subdir.
(localedef-aux): Add md5 crypto assembler when in locale subdir.
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile: Mirror sparc64
multiarch changes.
* sysdeps/sparc/sparc64/multiarch/md5-block.c: New file.
* sysdeps/sparc/sparc64/multiarch/md5-crop.S: New file.
* sysdeps/sparc/sparc64/multiarch/sha256-block.c: New file.
* sysdeps/sparc/sparc64/multiarch/sha256-crop.S: New file.
* sysdeps/sparc/sparc64/multiarch/sha512-block.c: New file.
* sysdeps/sparc/sparc64/multiarch/sha512-crop.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/md5-block.c: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/md5-crop.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha256-block.c: New
file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha256-crop.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha512-block.c: New
file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha512-crop.S: New file.
* sysdeps/sparc/sparc64/multiarch/memcpy-niagara4.S: On 32-bit, clear
upper 32-bits of the length value in %o2 since we use branch-on-register
tests which consider the entire 64-bit register.
* sysdeps/sparc/sparc64/multiarch/memset-niagara4.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memset-niagara4.S: New
file.
* sysdeps/sparc/sparc64/multiarch/Makefile: Add to
sysdep_routines.
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile: Likewise.
* sysdeps/sparc/sparc64/multiarch/memset.S: Use Niagara-4 memset
and bzero when HWCAP_SPARC_CRYPTO is present.
* sysdeps/sparc/sparc64/multiarch/memcpy-niagara4.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-niagara4.S: New
file.
* sysdeps/sparc/sparc64/multiarch/Makefile: Add to
sysdep_routines.
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile: Likewise.
* sysdeps/sparc/sparc64/multiarch/memcpy.S: Use Niagara-4 memcpy
and mempcpy when HWCAP_SPARC_CRYPTO is set.
fmovd clears the current exception field in the %fsr, fsrc2
does not and therefore runs more efficiently on some cpus.
* sysdeps/sparc/sparc64/memcpy.S: Use fsrc2 to move 64-bit
values between float registers.
* sysdeps/sparc/sparc64/memset.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/memcpy-niagara2.S: Likewise.
* sysdeps/sparc/sparc64/multiarch/memcpy.S: Provide a hidden def to
the IFUNC routine in the libc case.
* sysdeps/sparc/sparc64/multiarch/memcpy.S: Likewise.
* sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-memset.c: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-memcpy.c: New file.
* sysdeps/sparc/sparc32/sparcv9/rtld-memset.c: New file.
* sysdeps/sparc/sparc32/sparcv9/rtld-memcpy.c: New file.
* sysdeps/sparc/sparc64/multiarch/rtld-memset.c: New file.
* sysdeps/sparc/sparc64/multiarch/rtld-memcpy.c: New file.
* sysdeps/sparc/sparc64/rtld-memset.c: New file.
* sysdeps/sparc/sparc64/rtld-memcpy.c: New file.
