glibc/sysdeps/x86_64
Patrick McGehearty 6fd0a3c6a8 Improve __ieee754_exp() performance by greater than 5x on sparc/x86.
These changes will be active for all platforms that don't provide
their own exp() routines. They will also be active for ieee754
versions of ccos, ccosh, cosh, csin, csinh, sinh, exp10, gamma, and
erf.

Typical performance gains is typically around 5x when measured on
Sparc s7 for common values between exp(1) and exp(40).

Using the glibc perf tests on sparc,
      sparc (nsec)    x86 (nsec)
      old     new     old     new
max   17629   395    5173     144
min     399    54      15      13
mean   5317   200    1349      23

The extreme max times for the old (ieee754) exp are due to the
multiprecision computation in the old algorithm when the true value is
very near 0.5 ulp away from an value representable in double
precision. The new algorithm does not take special measures for those
cases. The current glibc exp perf tests overrepresent those values.
Informal testing suggests approximately one in 200 cases might
invoke the high cost computation. The performance advantage of the new
algorithm for other values is still large but not as large as indicated
by the chart above.

Glibc correctness tests for exp() and expf() were run. Within the
test suite 3 input values were found to cause 1 bit differences (ulp)
when "FE_TONEAREST" rounding mode is set. No differences in exp() were
seen for the tested values for the other rounding modes.
Typical example:
exp(-0x1.760cd2p+0)  (-1.46113312244415283203125)
 new code:    2.31973271630014299393707e-01   0x1.db14cd799387ap-3
 old code:    2.31973271630014271638132e-01   0x1.db14cd7993879p-3
    exp    =  2.31973271630014285508337 (high precision)
Old delta: off by 0.49 ulp
New delta: off by 0.51 ulp

In addition, because ieee754_exp() is used by other routines, cexp()
showed test results with very small imaginary input values where the
imaginary portion of the result was off by 3 ulp when in upward
rounding mode, but not in the other rounding modes.  For x86, tgamma
showed a few values where the ulp increased to 6 (max ulp for tgamma
is 5). Sparc tgamma did not show these failures.  I presume the tgamma
differences are due to compiler optimization differences within the
gamma function.The gamma function is known to be difficult to compute
accurately.

	* sysdeps/ieee754/dbl-64/e_exp.c: Include <math-svid-compat.h> and
	<errno.h>.  Include "eexp.tbl".
	(half): New constant.
	(one): Likewise.
	(__ieee754_exp): Rewrite.
	(__slowexp): Remove prototype.
	* sysdeps/ieee754/dbl-64/eexp.tbl: New file.
	* sysdeps/ieee754/dbl-64/slowexp.c: Remove file.
	* sysdeps/i386/fpu/slowexp.c: Likewise.
	* sysdeps/ia64/fpu/slowexp.c: Likewise.
	* sysdeps/m68k/m680x0/fpu/slowexp.c: Likewise.
	* sysdeps/x86_64/fpu/multiarch/slowexp-avx.c: Likewise.
	* sysdeps/x86_64/fpu/multiarch/slowexp-fma.c: Likewise.
	* sysdeps/x86_64/fpu/multiarch/slowexp-fma4.c: Likewise.
	* sysdeps/generic/math_private.h (__slowexp): Remove prototype.
	* sysdeps/ieee754/dbl-64/e_pow.c: Remove mention of slowexp.c in
	comment.
	* sysdeps/powerpc/power4/fpu/Makefile [$(subdir) = math]
	(CPPFLAGS-slowexp.c): Remove variable.
	* sysdeps/x86_64/fpu/multiarch/Makefile (libm-sysdep_routines):
	Remove slowexp-fma, slowexp-fma4 and slowexp-avx.
	(CFLAGS-slowexp-fma.c): Remove variable.
	(CFLAGS-slowexp-fma4.c): Likewise.
	(CFLAGS-slowexp-avx.c): Likewise.
	* sysdeps/x86_64/fpu/multiarch/e_exp-avx.c (__slowexp): Do not
	define as macro.
	* sysdeps/x86_64/fpu/multiarch/e_exp-fma.c (__slowexp): Likewise.
	* sysdeps/x86_64/fpu/multiarch/e_exp-fma4.c (__slowexp): Likewise.
	* math/Makefile (type-double-routines): Remove slowexp.
	* manual/probes.texi (slowexp_p6): Remove.
	(slowexp_p32): Likewise.
2017-12-19 17:27:31 +00:00
..
64 Move architecture shlib-versions files to Linux-specific directories. 2014-07-17 14:31:12 +00:00
fpu Improve __ieee754_exp() performance by greater than 5x on sparc/x86. 2017-12-19 17:27:31 +00:00
multiarch Remove remaining _HAVE_STRING_ARCH_* definitions (BZ #18858) 2017-09-06 14:35:23 -03:00
nptl x86: Add feature_1 to tcbhead_t [BZ #22563] 2017-12-19 02:45:34 -08:00
x32 Use libm_alias_float for x86_64. 2017-11-29 21:25:41 +00:00
____longjmp_chk.S
__longjmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
_mcount.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
abort-instr.h
add_n.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
addmul_1.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
atomic-machine.h Optimize generic spinlock code and use C11 like atomic macros. 2017-06-06 09:41:56 +02:00
backtrace.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
bsd-_setjmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
bsd-setjmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
bzero.S
configure Add --enable-static-pie configure option to build static PIE [BZ #19574] 2017-12-15 17:12:14 -08:00
configure.ac Add --enable-static-pie configure option to build static PIE [BZ #19574] 2017-12-15 17:12:14 -08:00
crti.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
crtn.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
dl-irel.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
dl-lookupcfg.h elf: Remove internal_function attribute 2017-08-31 16:59:37 +02:00
dl-machine.h x86-64: Use fxsave/xsave/xsavec in _dl_runtime_resolve [BZ #21265] 2017-10-20 11:00:34 -07:00
dl-procinfo.c Add sysdeps/x86/dl-procinfo.c 2017-04-10 12:01:45 -07:00
dl-runtime.c
dl-tls.c x86-64: Align the stack in __tls_get_addr [BZ #21609] 2017-07-06 04:43:20 -07:00
dl-tls.h x86-64: Align the stack in __tls_get_addr [BZ #21609] 2017-07-06 04:43:20 -07:00
dl-tlsdesc.h elf: Remove internal_function attribute 2017-08-31 16:59:37 +02:00
dl-tlsdesc.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
dl-trampoline.h x86-64: Use fxsave/xsave/xsavec in _dl_runtime_resolve [BZ #21265] 2017-10-20 11:00:34 -07:00
dl-trampoline.S x86-64: Use fxsave/xsave/xsavec in _dl_runtime_resolve [BZ #21265] 2017-10-20 11:00:34 -07:00
ffs.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
ffsll.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
hp-timing.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
htonl.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
ifuncmain8.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
ifuncmod8.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
Implies Add float128 support for x86_64, x86. 2017-06-26 22:02:24 +00:00
jmpbuf-offsets.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
jmpbuf-unwind.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
l10nflist.c
ldbl2mpn.c
ldsodefs.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
link-defines.sym
locale-defines.sym
localplt.data ld.so: Introduce struct dl_exception 2017-08-10 16:54:57 +02:00
lshift.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
machine-gmon.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
Makefile x86-64: Use fxsave/xsave/xsavec in _dl_runtime_resolve [BZ #21265] 2017-10-20 11:00:34 -07:00
memchr.S x86-64: Optimize memchr/rawmemchr/wmemchr with SSE2/AVX2 2017-06-09 05:13:31 -07:00
memcmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
memcopy.h X86-64: Add dummy memcopy.h and wordcopy.c 2016-06-09 04:38:34 -07:00
memcpy_chk.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
memcpy.S X86-64: Remove previous default/SSE2/AVX2 memcpy/memmove 2016-06-08 13:58:08 -07:00
memmove_chk.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
memmove.S x86-64: Use IFUNC memcpy and mempcpy in libc.a 2017-08-04 12:27:18 -07:00
mempcpy_chk.S x86_64: fix static build of __mempcpy_chk for compilers defaulting to PIC/PIE 2017-03-15 16:10:05 -07:00
mempcpy.S X86-64: Remove previous default/SSE2/AVX2 memcpy/memmove 2016-06-08 13:58:08 -07:00
memrchr.S x86_64: Remove redundant REX bytes from memrchr.S 2017-06-05 07:41:26 -07:00
memset_chk.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
memset.S x86: Remove __memset_zero_constant_len_parameter [BZ #21790] 2017-08-04 10:56:51 -07:00
memusage.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
mp_clz_tab.c
mul_1.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
preconfigure
preconfigure.ac
rawmemchr.S x86_64: Remove L(return_null) from rawmemchr.S 2017-05-20 06:13:38 -07:00
rshift.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
rtld-offsets.sym x86-64: Align the stack in __tls_get_addr [BZ #21609] 2017-07-06 04:43:20 -07:00
sched_cpucount.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
setjmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
stack-aliasing.h Clean up stack-coloring macros. 2014-06-20 19:50:16 -07:00
stackguard-macros.h
stackinfo.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
start.S x86-64: Check PIC instead of SHARED in start.S 2017-08-02 10:27:34 -07:00
stpcpy.S
strcasecmp_l-nonascii.c Use locale_t, not __locale_t, throughout glibc 2017-06-20 20:30:06 -04:00
strcasecmp_l.S
strcasecmp.S
strcat.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
strchr.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
strchrnul.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
strcmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
strcpy.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
strcspn.S x86-64: Implement strcspn/strpbrk/strspn IFUNC selectors in C 2017-06-15 08:59:05 -07:00
strlen.S x86-64: Update strlen.S to support wcslen/wcsnlen 2017-06-05 07:58:23 -07:00
strncase_l-nonascii.c Use locale_t, not __locale_t, throughout glibc 2017-06-20 20:30:06 -04:00
strncase_l.S
strncase.S
strncmp.S
strnlen.S
strpbrk.S x86-64: Implement strcspn/strpbrk/strspn IFUNC selectors in C 2017-06-15 08:59:05 -07:00
strrchr.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
strspn.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
sub_n.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
submul_1.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
sysdep.h x86: Add sysdeps/x86/sysdep.h 2017-11-01 05:37:26 -07:00
tls_get_addr.S x86-64: Align the stack in __tls_get_addr [BZ #21609] 2017-07-06 04:43:20 -07:00
tls-macros.h
tlsdesc.c elf: Remove internal_function attribute 2017-08-31 16:59:37 +02:00
tlsdesc.sym x86-64: Align the stack in __tls_get_addr [BZ #21609] 2017-07-06 04:43:20 -07:00
tst-audit3.c Modify several tests to use test-skeleton.c 2014-11-05 15:24:08 +05:30
tst-audit4-aux.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-audit4.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-audit5.c Modify several tests to use test-skeleton.c 2014-11-05 15:24:08 +05:30
tst-audit6.c Modify several tests to use test-skeleton.c 2015-07-15 15:10:23 +05:30
tst-audit7.c
tst-audit10-aux.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-audit10.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-audit.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-auditmod3a.c
tst-auditmod3b.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-auditmod4a.c
tst-auditmod4b.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-auditmod5a.c
tst-auditmod5b.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-auditmod6a.c
tst-auditmod6b.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-auditmod6c.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-auditmod7a.c
tst-auditmod7b.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-auditmod10a.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-auditmod10b.c Add missing header files throughout the testsuite. 2017-02-16 17:33:18 -05:00
tst-avx512-aux.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-avx512.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-avx512mod.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-avx-aux.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-avx.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-avxmod.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-mallocalign1.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-platform-1.c x86-64: Don't set GLRO(dl_platform) to NULL [BZ #22299] 2017-10-19 08:28:26 -07:00
tst-platformmod-1.c x86-64: Don't set GLRO(dl_platform) to NULL [BZ #22299] 2017-10-19 08:28:26 -07:00
tst-platformmod-2.c x86-64: Don't set GLRO(dl_platform) to NULL [BZ #22299] 2017-10-19 08:28:26 -07:00
tst-quad1.c Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-quad1pie.c
tst-quad2.c
tst-quad2pie.c
tst-quadmod1.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-quadmod1pie.S
tst-quadmod2.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-quadmod2pie.S
tst-split-dynreloc.c Fix dynamic linker issue with bind-now 2015-08-19 05:37:01 -07:00
tst-split-dynreloc.lds Fix dynamic linker issue with bind-now 2015-08-19 05:37:01 -07:00
tst-sse.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-ssemod.c x86-64: Verify that _dl_runtime_resolve preserves vector registers 2017-02-09 12:19:58 -08:00
tst-stack-align.h Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
tst-x86_64-1.c x86: Add x86_64 to x86-64 HWCAP [BZ #22093] 2017-09-11 08:18:32 -07:00
tst-x86_64mod-1.c x86: Add x86_64 to x86-64 HWCAP [BZ #22093] 2017-09-11 08:18:32 -07:00
Versions
wcschr.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
wcscmp.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
wcslen.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
wcsrchr.S Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
wmemset_chk.S x86-64: Optimize wmemset with SSE2/AVX2/AVX512 2017-06-05 11:09:59 -07:00
wmemset.S x86-64: Optimize wmemset with SSE2/AVX2/AVX512 2017-06-05 11:09:59 -07:00
wordcopy.c X86-64: Add dummy memcopy.h and wordcopy.c 2016-06-09 04:38:34 -07:00