Introduce new log symbol version that doesn't do SVID compatible error
handling. The standard errno and fp exception based error handling is
inline in the new code and does not have significant overhead.
The wrapper is disabled for sysdeps/ieee754/dbl-64 by using empty
w_log.c and enabled for targets with their own log implementation by
including math/w_log.c.
The compatibility symbol version still uses the wrapper with SVID error
handling around the new code. There is no new symbol version nor
compatibility code on !LIBM_SVID_COMPAT targets (e.g. riscv).
On targets where previously logl was an alias of log, now it points to
the compatibility symbol with the wrapper, because it still need the
SVID compatible error handling. This affects NO_LONG_DOUBLE (e.g. arm)
and LONG_DOUBLE_COMPAT (e.g. alpha) targets as well.
The __log_finite symbol is now an alias of log. Both __log_finite and
log set errno and thus not const functions.
The ia64 asm is changed so the compat and new symbol versions map to the
same address.
On x86_64 #include <math.h> was added before macro definitions that may
affect that header.
Tested with build-many-glibcs.py.
* math/Versions (GLIBC_2.29): Add log.
* math/w_log_compat.c (__log_compat): Change to versioned compat
symbol.
* math/w_log.c: New file.
* sysdeps/i386/fpu/w_log.c: New file.
* sysdeps/ia64/fpu/e_log.S: Update.
* sysdeps/ieee754/dbl-64/e_log.c (__ieee754_log): Rename to __log
and add necessary aliases.
* sysdeps/ieee754/dbl-64/w_log.c: New file.
* sysdeps/m68k/m680x0/fpu/w_log.c: New file.
* sysdeps/mach/hurd/i386/libm.abilist: Update.
* sysdeps/unix/sysv/linux/aarch64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/alpha/libm.abilist: Update.
* sysdeps/unix/sysv/linux/arm/libm.abilist: Update.
* sysdeps/unix/sysv/linux/hppa/libm.abilist: Update.
* sysdeps/unix/sysv/linux/i386/libm.abilist: Update.
* sysdeps/unix/sysv/linux/ia64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/m68k/coldfire/libm.abilist: Update.
* sysdeps/unix/sysv/linux/m68k/m680x0/libm.abilist: Update.
* sysdeps/unix/sysv/linux/microblaze/libm.abilist: Update.
* sysdeps/unix/sysv/linux/mips/mips32/libm.abilist: Update.
* sysdeps/unix/sysv/linux/mips/mips64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/nios2/libm.abilist: Update.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libm.abilist: Update.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libm.abilist: Update.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libm-le.abilist: Update.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/s390/s390-32/libm.abilist: Update.
* sysdeps/unix/sysv/linux/s390/s390-64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/sh/libm.abilist: Update.
* sysdeps/unix/sysv/linux/sparc/sparc32/libm.abilist: Update.
* sysdeps/unix/sysv/linux/sparc/sparc64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/x86_64/64/libm.abilist: Update.
* sysdeps/unix/sysv/linux/x86_64/x32/libm.abilist: Update.
* sysdeps/x86_64/fpu/multiarch/e_log-avx.c (__ieee754_log): Rename to
__log.
* sysdeps/x86_64/fpu/multiarch/e_log-fma.c (__ieee754_log): Likewise.
* sysdeps/x86_64/fpu/multiarch/e_log-fma4.c (__ieee754_log): Likewise.
* sysdeps/x86_64/fpu/multiarch/e_log.c (__ieee754_log): Likewise.
* sysdeps/x86_64/fpu/multiarch/w_log.c: New file.
Optimized log using carefully generated lookup table with 1/c and log(c)
values for small intervalls around 1. The log(c) is very near a double
precision value, it has about 62 bits precision. The algorithm is
log(2^k x) = k log(2) + log(c) + log(x/c), where the last term is
approximated by a polynomial of x/c - 1. Near 1 a single polynomial of
x - 1 is used.
There is separate code path when fma instruction is not available for
computing x/c - 1 precisely, in which case the table size is doubled.
The code uses __builtin_fma under __FP_FAST_FMA to ensure it is inlined
as an instruction.
With the default configuration settings the worst case error is 0.519 ULP
(and 0.520 without fma), the rodata size is 2192 bytes (4240 without fma).
The non-nearest rounding error is less than 1 ULP.
Improvements on Cortex-A72 compared to current glibc master:
log thruput: 3.28x in [0.01 11.1]
log latency: 2.23x in [0.01 11.1]
log thruput: 1.56x in [0.999 1.001]
log latency: 1.57x in [0.999 1.001]
Tested on
aarch64-linux-gnu (defined __FP_FAST_FMA)
arm-linux-gnueabihf (!defined __FP_FAST_FMA)
x86_64-linux-gnu (!defined __FP_FAST_FMA)
powerpc64le-linux-gnu (defined __FP_FAST_FMA)
targets.
* NEWS: Mention log improvement.
* math/Makefile (type-double-routines): Add e_log_data.
* sysdeps/i386/fpu/e_log_data.c: New file.
* sysdeps/ia64/fpu/e_log_data.c: New file.
* sysdeps/ieee754/dbl-64/e_log.c: Rewrite.
* sysdeps/ieee754/dbl-64/e_log_data.c: New file.
* sysdeps/ieee754/dbl-64/math_config.h (__log_data): Add.
* sysdeps/ieee754/dbl-64/ulog.h: Remove.
* sysdeps/ieee754/dbl-64/ulog.tbl: Remove.
* sysdeps/m68k/m680x0/fpu/e_log_data.c: New file.
Remove the slow paths from log. Like several other double precision math
functions, log is exactly rounded. This is not required from math functions
and causes major overheads as it requires multiple fallbacks using higher
precision arithmetic if a result is close to 0.5ULP. Ridiculous slowdowns
of up to 100000x have been reported when the highest precision path triggers.
Interestingly removing the slow paths makes hardly any difference in practice:
the worst case error is still ~0.502ULP, and exp(log(x)) shows identical results
before/after on many millions of random cases. All GLIBC math tests pass on
AArch64 and x64 with no change in ULP error. A simple test over a few hundred
million values shows log is now 18% faster on average.
* manual/probes.texi (slowlog): Delete documentation of removed probe.
(slowlog_inexact): Likewise
* sysdeps/ieee754/dbl-64/e_log.c (__ieee754_log): Remove slow paths.
* sysdeps/ieee754/dbl-64/ulog.h: Remove unused declarations.
According to ISO C Annex F, log (1) should be +0 in all rounding
modes, but some implementations in glibc wrongly return -0 in
round-downward mode (mapping to log1p (x - 1) is problematic because 1
- 1 is -0 in round-downward mode, and log1p (-0) is -0). This patch
fixes this. (It helps with some implementations of other functions
such as acosh, log2 and log10 that call out to log, but not enough to
enable all-rounding-modes testing for those functions without further
fixes to other implementations of them.)
Tested x86_64 and x86 and ulps updated accordingly, and did spot tests
for mips64 for the ldbl-128 fix, and i586 for the sysdeps/i386/fpu
implementations shadowed by those in sysdeps/i386/i686/fpu.
[BZ #16731]
* sysdeps/i386/fpu/e_log.S (__ieee754_log): Take absolute value
when x - 1 is zero.
* sysdeps/i386/fpu/e_logf.S (__ieee754_logf): Likewise.
* sysdeps/i386/fpu/e_logl.S (__ieee754_logl): Likewise.
* sysdeps/i386/i686/fpu/e_logl.S (__ieee754_logl): Likewise.
* sysdeps/ieee754/dbl-64/e_log.c (__ieee754_log): Return +0 when
argument is 1.
* sysdeps/ieee754/ldbl-128/e_logl.c (__ieee754_logl): Likewise.
* sysdeps/x86_64/fpu/e_logl.S: Take absolute value when x - 1 is
zero.
* math/libm-test.inc (log_test): Use ALL_RM_TEST.
* sysdeps/i386/fpu/libm-test-ulps: Update.
* sysdeps/x86_64/fpu/libm-test-ulps: Likewise.
Add systemtap probes to various slow paths in libm so that application
developers may use systemtap to find out if their applications are
hitting these slow paths. We have added probes for pow, exp, log,
tan, atan and atan2.
2001-03-12 Ulrich Drepper <drepper@redhat.com>
* sysdeps/ieee754/dbl-64/e_remainder.c: Fix handling of boundary
conditions.
* sysdeps/ieee754/dbl-64/e_pow.c: Fix handling of boundary
conditions.
* sysdeps/ieee754/dbl-64/s_sin.c (__sin): Handle Inf and NaN
correctly.
(__cos): Likewise.
* sysdeps/ieee754/dbl-64/e_asin.c (__ieee754_asin): Handle NaN
correctly.
(__ieee754_acos): Likewise.
redefinition.
* sysdeps/ieee754/dbl-64/endian.h: Define also one of BIG_ENDI and
LITTLE_ENDI.
* sysdeps/ieee754/dbl-64/MathLib.h (Init_Lib): Use void as
parameter list.
2001-03-11 Ulrich Drepper <drepper@redhat.com>
Last-bit accurate math library implementation by IBM Haifa.
Contributed by Abraham Ziv <ziv@il.ibm.com>, Moshe Olshansky
<olshansk@il.ibm.com>, Ealan Henis <ealan@il.ibm.com>, and
Anna Reitman <reitman@il.ibm.com>.
* math/Makefile (dbl-only-routines): New variable.
(libm-routines): Add $(dbl-only-routines).
* sysdeps/ieee754/dbl-64/e_acos.c: Empty, definition is in e_asin.c.
* sysdeps/ieee754/dbl-64/e_asin.c: Replaced with accurate asin
implementation.
* sysdeps/ieee754/dbl-64/e_atan2.c: Replaced with accurate atan2
implementation.
* sysdeps/ieee754/dbl-64/e_exp.c: Replaced with accurate exp
implementation.
* sysdeps/ieee754/dbl-64/e_lgamma_r.c: Don't use __kernel_sin and
__kernel_cos.
* sysdeps/ieee754/dbl-64/e_log.c: Replaced with accurate log
implementation.
* sysdeps/ieee754/dbl-64/e_remainder.c: Replaced with accurate
remainder implementation.
* sysdeps/ieee754/dbl-64/e_pow.c: Replaced with accurate pow
implementation.
* sysdeps/ieee754/dbl-64/e_sqrt.c: Replaced with accurate sqrt
implementation.
* sysdeps/ieee754/dbl-64/k_cos.c: Empty, definition is in s_sin.c.
* sysdeps/ieee754/dbl-64/k_sin.c: Empty, definition is in s_sin.c.
* sysdeps/ieee754/dbl-64/s_atan.c: Replaced with accurate atan
implementation.
* sysdeps/ieee754/dbl-64/s_cos.c: Empty, definition is in s_sin.c.
* sysdeps/ieee754/dbl-64/s_sin.c: Replaced with accurate sin/cos
implementation.
* sysdeps/ieee754/dbl-64/s_sincos.c: Rewritten to not use __kernel_sin
and __kernel_cos.
* sysdeps/ieee754/dbl-64/s_tan.c: Replaced with accurate tan
implementation.
* sysdeps/ieee754/dbl-64/Dist: Add new non-code files.
* sysdeps/ieee754/dbl-64/MathLib.h: New file.
* sysdeps/ieee754/dbl-64/asincos.tbl: New file.
* sysdeps/ieee754/dbl-64/atnat.h: New file.
* sysdeps/ieee754/dbl-64/atnat2.h: New file.
* sysdeps/ieee754/dbl-64/branred.c: New file.
* sysdeps/ieee754/dbl-64/branred.h: New file.
* sysdeps/ieee754/dbl-64/dla.h: New file.
* sysdeps/ieee754/dbl-64/doasin.c: New file.
* sysdeps/ieee754/dbl-64/doasin.h: New file.
* sysdeps/ieee754/dbl-64/dosincos.c: New file.
* sysdeps/ieee754/dbl-64/dosincos.h: New file.
* sysdeps/ieee754/dbl-64/endian.h: New file.
* sysdeps/ieee754/dbl-64/halfulp.c: New file.
* sysdeps/ieee754/dbl-64/mpa.c: New file.
* sysdeps/ieee754/dbl-64/mpa.h: New file.
* sysdeps/ieee754/dbl-64/mpa2.h: New file.
* sysdeps/ieee754/dbl-64/mpatan.c: New file.
* sysdeps/ieee754/dbl-64/mpatan.h: New file.
* sysdeps/ieee754/dbl-64/mpatan2.c: New file.
* sysdeps/ieee754/dbl-64/mpexp.c: New file.
* sysdeps/ieee754/dbl-64/mpexp.h: New file.
* sysdeps/ieee754/dbl-64/mplog.c: New file.
* sysdeps/ieee754/dbl-64/mplog.h: New file.
* sysdeps/ieee754/dbl-64/mpsqrt.c: New file.
* sysdeps/ieee754/dbl-64/mpsqrt.h: New file.
* sysdeps/ieee754/dbl-64/mptan.c: New file.
* sysdeps/ieee754/dbl-64/mydefs.h: New file.
* sysdeps/ieee754/dbl-64/powtwo.tbl: New file.
* sysdeps/ieee754/dbl-64/root.tbl: New file.
* sysdeps/ieee754/dbl-64/sincos.tbl: New file.
* sysdeps/ieee754/dbl-64/sincos32.c: New file.
* sysdeps/ieee754/dbl-64/sincos32.h: New file.
* sysdeps/ieee754/dbl-64/slowexp.c: New file.
* sysdeps/ieee754/dbl-64/slowpow.c: New file.
* sysdeps/ieee754/dbl-64/uasncs.h: New file.
* sysdeps/ieee754/dbl-64/uatan.tbl: New file.
* sysdeps/ieee754/dbl-64/uexp.h: New file.
* sysdeps/ieee754/dbl-64/uexp.tbl: New file.
* sysdeps/ieee754/dbl-64/ulog.h: New file.
* sysdeps/ieee754/dbl-64/ulog.tbl: New file.
* sysdeps/ieee754/dbl-64/upow.h: New file.
* sysdeps/ieee754/dbl-64/upow.tbl: New file.
* sysdeps/ieee754/dbl-64/urem.h: New file.
* sysdeps/ieee754/dbl-64/uroot.h: New file.
* sysdeps/ieee754/dbl-64/usncs.h: New file.
* sysdeps/ieee754/dbl-64/utan.h: New file.
* sysdeps/ieee754/dbl-64/utan.tbl: New file.
* sysdeps/i386/fpu/branred.c: New file.
* sysdeps/i386/fpu/doasin.c: New file.
* sysdeps/i386/fpu/dosincos.c: New file.
* sysdeps/i386/fpu/halfulp.c: New file.
* sysdeps/i386/fpu/mpa.c: New file.
* sysdeps/i386/fpu/mpatan.c: New file.
* sysdeps/i386/fpu/mpatan2.c: New file.
* sysdeps/i386/fpu/mpexp.c: New file.
* sysdeps/i386/fpu/mplog.c: New file.
* sysdeps/i386/fpu/mpsqrt.c: New file.
* sysdeps/i386/fpu/mptan.c: New file.
* sysdeps/i386/fpu/sincos32.c: New file.
* sysdeps/i386/fpu/slowexp.c: New file.
* sysdeps/i386/fpu/slowpow.c: New file.
* sysdeps/ia64/fpu/branred.c: New file.
* sysdeps/ia64/fpu/doasin.c: New file.
* sysdeps/ia64/fpu/dosincos.c: New file.
* sysdeps/ia64/fpu/halfulp.c: New file.
* sysdeps/ia64/fpu/mpa.c: New file.
* sysdeps/ia64/fpu/mpatan.c: New file.
* sysdeps/ia64/fpu/mpatan2.c: New file.
* sysdeps/ia64/fpu/mpexp.c: New file.
* sysdeps/ia64/fpu/mplog.c: New file.
* sysdeps/ia64/fpu/mpsqrt.c: New file.
* sysdeps/ia64/fpu/mptan.c: New file.
* sysdeps/ia64/fpu/sincos32.c: New file.
* sysdeps/ia64/fpu/slowexp.c: New file.
* sysdeps/ia64/fpu/slowpow.c: New file.
* sysdeps/m68k/fpu/branred.c: New file.
* sysdeps/m68k/fpu/doasin.c: New file.
* sysdeps/m68k/fpu/dosincos.c: New file.
* sysdeps/m68k/fpu/halfulp.c: New file.
* sysdeps/m68k/fpu/mpa.c: New file.
* sysdeps/m68k/fpu/mpatan.c: New file.
* sysdeps/m68k/fpu/mpatan2.c: New file.
* sysdeps/m68k/fpu/mpexp.c: New file.
* sysdeps/m68k/fpu/mplog.c: New file.
* sysdeps/m68k/fpu/mpsqrt.c: New file.
* sysdeps/m68k/fpu/mptan.c: New file.
* sysdeps/m68k/fpu/sincos32.c: New file.
* sysdeps/m68k/fpu/slowexp.c: New file.
* sysdeps/m68k/fpu/slowpow.c: New file.
* iconvdata/gconv-modules: Add a number of alias, mostly for IBM
codepages.
