x86-64: Add FMA multiarch functions to libm
This patch adds multiarch functions optimized with -mfma -mavx2 to libm.
e_pow-fma.c is compiled with $(config-cflags-nofma) due to PR 19003.
* sysdeps/x86_64/fpu/multiarch/Makefile (libm-sysdep_routines):
Add e_exp-fma, e_log-fma, e_pow-fma, s_atan-fma, e_asin-fma,
e_atan2-fma, s_sin-fma, s_tan-fma, mplog-fma, mpa-fma,
slowexp-fma, slowpow-fma, sincos32-fma, doasin-fma, dosincos-fma,
halfulp-fma, mpexp-fma, mpatan2-fma, mpatan-fma, mpsqrt-fma,
and mptan-fma.
(CFLAGS-doasin-fma.c): New.
(CFLAGS-dosincos-fma.c): Likewise.
(CFLAGS-e_asin-fma.c): Likewise.
(CFLAGS-e_atan2-fma.c): Likewise.
(CFLAGS-e_exp-fma.c): Likewise.
(CFLAGS-e_log-fma.c): Likewise.
(CFLAGS-e_pow-fma.c): Likewise.
(CFLAGS-halfulp-fma.c): Likewise.
(CFLAGS-mpa-fma.c): Likewise.
(CFLAGS-mpatan-fma.c): Likewise.
(CFLAGS-mpatan2-fma.c): Likewise.
(CFLAGS-mpexp-fma.c): Likewise.
(CFLAGS-mplog-fma.c): Likewise.
(CFLAGS-mpsqrt-fma.c): Likewise.
(CFLAGS-mptan-fma.c): Likewise.
(CFLAGS-s_atan-fma.c): Likewise.
(CFLAGS-sincos32-fma.c): Likewise.
(CFLAGS-slowexp-fma.c): Likewise.
(CFLAGS-slowpow-fma.c): Likewise.
(CFLAGS-s_sin-fma.c): Likewise.
(CFLAGS-s_tan-fma.c): Likewise.
* sysdeps/x86_64/fpu/multiarch/doasin-fma.c: New file.
* sysdeps/x86_64/fpu/multiarch/dosincos-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_asin-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_exp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_log-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_pow-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/halfulp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h: Likewise.
* sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpa-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpatan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpexp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mplog-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mptan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_atan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_sin-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_tan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/sincos32-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/slowexp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/slowpow-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_asin.c: Rewrite.
* sysdeps/x86_64/fpu/multiarch/e_atan2.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_exp.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_log.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_pow.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_atan.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_sin.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_tan.c: Likewise.
2017-08-07 15:19:59 +00:00
|
|
|
/* Common definition for ifunc selections optimized with AVX, AVX2/FMA
|
|
|
|
and FMA4.
|
2019-01-01 00:11:28 +00:00
|
|
|
Copyright (C) 2017-2019 Free Software Foundation, Inc.
|
x86-64: Add FMA multiarch functions to libm
This patch adds multiarch functions optimized with -mfma -mavx2 to libm.
e_pow-fma.c is compiled with $(config-cflags-nofma) due to PR 19003.
* sysdeps/x86_64/fpu/multiarch/Makefile (libm-sysdep_routines):
Add e_exp-fma, e_log-fma, e_pow-fma, s_atan-fma, e_asin-fma,
e_atan2-fma, s_sin-fma, s_tan-fma, mplog-fma, mpa-fma,
slowexp-fma, slowpow-fma, sincos32-fma, doasin-fma, dosincos-fma,
halfulp-fma, mpexp-fma, mpatan2-fma, mpatan-fma, mpsqrt-fma,
and mptan-fma.
(CFLAGS-doasin-fma.c): New.
(CFLAGS-dosincos-fma.c): Likewise.
(CFLAGS-e_asin-fma.c): Likewise.
(CFLAGS-e_atan2-fma.c): Likewise.
(CFLAGS-e_exp-fma.c): Likewise.
(CFLAGS-e_log-fma.c): Likewise.
(CFLAGS-e_pow-fma.c): Likewise.
(CFLAGS-halfulp-fma.c): Likewise.
(CFLAGS-mpa-fma.c): Likewise.
(CFLAGS-mpatan-fma.c): Likewise.
(CFLAGS-mpatan2-fma.c): Likewise.
(CFLAGS-mpexp-fma.c): Likewise.
(CFLAGS-mplog-fma.c): Likewise.
(CFLAGS-mpsqrt-fma.c): Likewise.
(CFLAGS-mptan-fma.c): Likewise.
(CFLAGS-s_atan-fma.c): Likewise.
(CFLAGS-sincos32-fma.c): Likewise.
(CFLAGS-slowexp-fma.c): Likewise.
(CFLAGS-slowpow-fma.c): Likewise.
(CFLAGS-s_sin-fma.c): Likewise.
(CFLAGS-s_tan-fma.c): Likewise.
* sysdeps/x86_64/fpu/multiarch/doasin-fma.c: New file.
* sysdeps/x86_64/fpu/multiarch/dosincos-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_asin-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_exp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_log-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_pow-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/halfulp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h: Likewise.
* sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpa-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpatan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpexp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mplog-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/mptan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_atan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_sin-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_tan-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/sincos32-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/slowexp-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/slowpow-fma.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_asin.c: Rewrite.
* sysdeps/x86_64/fpu/multiarch/e_atan2.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_exp.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_log.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/e_pow.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_atan.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_sin.c: Likewise.
* sysdeps/x86_64/fpu/multiarch/s_tan.c: Likewise.
2017-08-07 15:19:59 +00:00
|
|
|
This file is part of the GNU C Library.
|
|
|
|
|
|
|
|
The GNU C Library is free software; you can redistribute it and/or
|
|
|
|
modify it under the terms of the GNU Lesser General Public
|
|
|
|
License as published by the Free Software Foundation; either
|
|
|
|
version 2.1 of the License, or (at your option) any later version.
|
|
|
|
|
|
|
|
The GNU C Library is distributed in the hope that it will be useful,
|
|
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
|
|
Lesser General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU Lesser General Public
|
|
|
|
License along with the GNU C Library; if not, see
|
|
|
|
<http://www.gnu.org/licenses/>. */
|
|
|
|
|
|
|
|
#include <init-arch.h>
|
|
|
|
|
|
|
|
extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden;
|
|
|
|
extern __typeof (REDIRECT_NAME) OPTIMIZE (avx) attribute_hidden;
|
|
|
|
extern __typeof (REDIRECT_NAME) OPTIMIZE (fma) attribute_hidden;
|
|
|
|
extern __typeof (REDIRECT_NAME) OPTIMIZE (fma4) attribute_hidden;
|
|
|
|
|
|
|
|
static inline void *
|
|
|
|
IFUNC_SELECTOR (void)
|
|
|
|
{
|
|
|
|
const struct cpu_features* cpu_features = __get_cpu_features ();
|
|
|
|
|
|
|
|
if (CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
|
|
|
|
&& CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable))
|
|
|
|
return OPTIMIZE (fma);
|
|
|
|
|
|
|
|
if (CPU_FEATURES_ARCH_P (cpu_features, FMA4_Usable))
|
|
|
|
return OPTIMIZE (fma4);
|
|
|
|
|
|
|
|
if (CPU_FEATURES_ARCH_P (cpu_features, AVX_Usable))
|
|
|
|
return OPTIMIZE (avx);
|
|
|
|
|
|
|
|
return OPTIMIZE (sse2);
|
|
|
|
}
|