This patch helps some math functions performance by adding the libc_fexxx
variant of inline functions to handle both FPU round and exception set/restore
and by using them on the libc_fexxx_ctx functions. It is based on already coded
fexxx family functions for PPC with fpu.
Here is the summary of performance improvements due this patch (measured on a
POWER7 machine):
Before:
cos(): ITERS:9.5895e+07: TOTAL:5116.03Mcy, MAX:77.6cy, MIN:49.792cy, 18744 calls/Mcy
exp(): ITERS:2.827e+07: TOTAL:5187.15Mcy, MAX:494.018cy, MIN:38.422cy, 5450.01 calls/Mcy
pow(): ITERS:6.1705e+07: TOTAL:5144.26Mcy, MAX:171.95cy, MIN:29.935cy, 11994.9 calls/Mcy
sin(): ITERS:8.6898e+07: TOTAL:5117.06Mcy, MAX:83.841cy, MIN:46.582cy, 16982 calls/Mcy
tan(): ITERS:2.9473e+07: TOTAL:5115.39Mcy, MAX:191.017cy, MIN:172.352cy, 5761.63 calls/Mcy
After:
cos(): ITERS:2.05265e+08: TOTAL:5111.37Mcy, MAX:78.754cy, MIN:24.196cy, 40158.5 calls/Mcy
exp(): ITERS:3.341e+07: TOTAL:5170.84Mcy, MAX:476.317cy, MIN:15.574cy, 6461.23 calls/Mcy
pow(): ITERS:7.6153e+07: TOTAL:5129.1Mcy, MAX:147.5cy, MIN:30.916cy, 14847.2 calls/Mcy
sin(): ITERS:1.58816e+08: TOTAL:5115.11Mcy, MAX:1490.39cy, MIN:22.341cy, 31048.4 calls/Mcy
tan(): ITERS:3.4964e+07: TOTAL:5114.18Mcy, MAX:177.422cy, MIN:146.115cy, 6836.68 calls/Mcy
Autoconf has been deprecating configure.in for quite a long time.
Rename all our configure.in and preconfigure.in files to .ac.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Add systemtap probes to various slow paths in libm so that application
developers may use systemtap to find out if their applications are
hitting these slow paths. We have added probes for pow, exp, log,
tan, atan and atan2.
http://sourceware.org/ml/libc-alpha/2013-07/msg00197.html
A rewrite to make this code correct for little-endian.
* sysdeps/ieee754/ldbl-128ibm/e_sqrtl.c (mynumber): Replace
union 32-bit int array member with 64-bit int array.
(t515, tm256): Double rather than long double.
(__ieee754_sqrtl): Rewrite using 64-bit arithmetic.
http://sourceware.org/ml/libc-alpha/2013-08/msg00085.html
Rid ourselves of ieee854.
* sysdeps/ieee754/ldbl-128ibm/ieee754.h (union ieee854_long_double):
Delete.
(IEEE854_LONG_DOUBLE_BIAS): Delete.
* sysdeps/ieee754/ldbl-128ibm/math_ldbl.h: Don't include ieee854
version of math_ldbl.h.
http://sourceware.org/ml/libc-alpha/2013-08/msg00084.html
Another batch of ieee854 macros and union replacement. These four
files also have bugs fixed with this patch. The fact that the two
doubles in an IBM long double may have different signs means that
negation and absolute value operations can't just twiddle one sign bit
as you can with ieee864 style extended double. fmodl, remainderl,
erfl and erfcl all had errors of this type. erfl also returned +1 for
large magnitude negative input where it should return -1. The hypotl
error is innocuous since the value adjusted twice is only used as a
flag. The e_hypotl.c tests for large "a" and small "b" are mutually
exclusive because we've already exited when x/y > 2**120. That allows
some further small simplifications.
[BZ #15734], [BZ #15735]
* sysdeps/ieee754/ldbl-128ibm/e_fmodl.c (__ieee754_fmodl): Rewrite
all uses of ieee875 long double macros and unions. Simplify test
for 0.0L. Correct |x|<|y| and |x|=|y| test. Use
ldbl_extract_mantissa value for ix,iy exponents. Properly
normalize after ldbl_extract_mantissa, and don't add hidden bit
already handled. Don't treat low word of ieee854 mantissa like
low word of IBM long double and mask off bit when testing for
zero.
* sysdeps/ieee754/ldbl-128ibm/e_hypotl.c (__ieee754_hypotl): Rewrite
all uses of ieee875 long double macros and unions. Simplify tests
for 0.0L and inf. Correct double adjustment of k. Delete dead code
adjusting ha,hb. Simplify code setting kld. Delete two600 and
two1022, instead use their values. Recognise that tests for large
"a" and small "b" are mutually exclusive. Rename vars. Comment.
* sysdeps/ieee754/ldbl-128ibm/e_remainderl.c (__ieee754_remainderl):
Rewrite all uses of ieee875 long double macros and unions. Simplify
test for 0.0L and nan. Correct negation.
* sysdeps/ieee754/ldbl-128ibm/s_erfl.c (__erfl): Rewrite all uses of
ieee875 long double macros and unions. Correct output for large
magnitude x. Correct absolute value calculation.
(__erfcl): Likewise.
* math/libm-test.inc: Add tests for errors discovered in IBM long
double versions of fmodl, remainderl, erfl and erfcl.
http://sourceware.org/ml/libc-alpha/2013-08/msg00083.html
Further replacement of ieee854 macros and unions. These files also
have some optimisations for comparison against 0.0L, infinity and nan.
Since the ABI specifies that the high double of an IBM long double
pair is the value rounded to double, a high double of 0.0 means the
low double must also be 0.0. The ABI also says that infinity and
nan are encoded in the high double, with the low double unspecified.
This means that tests for 0.0L, +/-Infinity and +/-NaN need only check
the high double.
* sysdeps/ieee754/ldbl-128ibm/e_atan2l.c (__ieee754_atan2l): Rewrite
all uses of ieee854 long double macros and unions. Simplify tests
for long doubles that are fully specified by the high double.
* sysdeps/ieee754/ldbl-128ibm/e_gammal_r.c (__ieee754_gammal_r):
Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_ilogbl.c (__ieee754_ilogbl): Likewise.
Remove dead code too.
* sysdeps/ieee754/ldbl-128ibm/e_jnl.c (__ieee754_jnl): Likewise.
(__ieee754_ynl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_log10l.c (__ieee754_log10l): Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_logl.c (__ieee754_logl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_powl.c (__ieee754_powl): Likewise.
Remove dead code too.
* sysdeps/ieee754/ldbl-128ibm/k_tanl.c (__kernel_tanl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_expm1l.c (__expm1l): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_frexpl.c (__frexpl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_isinf_nsl.c (__isinf_nsl): Likewise.
Simplify.
* sysdeps/ieee754/ldbl-128ibm/s_isinfl.c (___isinfl): Likewise.
Simplify.
* sysdeps/ieee754/ldbl-128ibm/s_log1pl.c (__log1pl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_modfl.c (__modfl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_nextafterl.c (__nextafterl): Likewise.
Comment on variable precision.
* sysdeps/ieee754/ldbl-128ibm/s_nexttoward.c (__nexttoward): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_nexttowardf.c (__nexttowardf):
Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_remquol.c (__remquol): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_scalblnl.c (__scalblnl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_scalbnl.c (__scalbnl): Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_tanhl.c (__tanhl): Likewise.
* sysdeps/powerpc/fpu/libm-test-ulps: Adjust tan_towardzero ulps.
http://sourceware.org/ml/libc-alpha/2013-08/msg00081.html
This is the first of a series of patches to ban ieee854_long_double
and the ieee854_long_double macros when using IBM long double. union
ieee854_long_double just isn't correct for IBM long double, especially
when little-endian, and pretending it is OK has allowed a number of
bugs to remain undetected in sysdeps/ieee754/ldbl-128ibm/.
This changes the few places in generic code that use it.
* stdio-common/printf_size.c (__printf_size): Don't use
union ieee854_long_double in fpnum union.
* stdio-common/printf_fphex.c (__printf_fphex): Likewise. Use
signbit macro to retrieve sign from long double.
* stdio-common/printf_fp.c (___printf_fp): Use signbit macro to
retrieve sign from long double.
* sysdeps/ieee754/ldbl-128ibm/printf_fphex.c: Adjust for fpnum change.
* sysdeps/ieee754/ldbl-128/printf_fphex.c: Likewise.
* sysdeps/ieee754/ldbl-96/printf_fphex.c: Likewise.
* sysdeps/x86_64/fpu/printf_fphex.c: Likewise.
* math/test-misc.c (main): Don't use union ieee854_long_double.
ports/
* sysdeps/ia64/fpu/printf_fphex.c: Adjust for fpnum change.
http://sourceware.org/ml/libc-alpha/2013-06/msg00919.html
I discovered a number of places where denormals and other corner cases
were being handled wrongly.
- printf_fphex.c: Testing for the low double exponent being zero is
unnecessary. If the difference in exponents is less than 53 then the
high double exponent must be nearing the low end of its range, and the
low double exponent hit rock bottom.
- ldbl2mpn.c: A denormal (ie. exponent of zero) value is treated as
if the exponent was one, so shift mantissa left by one. Code handling
normalisation of the low double mantissa lacked a test for shift count
greater than bits in type being shifted, and lacked anything to handle
the case where the difference in exponents is less than 53 as in
printf_fphex.c.
- math_ldbl.h (ldbl_extract_mantissa): Same as above, but worse, with
code testing for exponent > 1 for some reason, probably a typo for >= 1.
- math_ldbl.h (ldbl_insert_mantissa): Round the high double as per
mpn2ldbl.c (hi is odd or explicit mantissas non-zero) so that the
number we return won't change when applying ldbl_canonicalize().
Add missing overflow checks and normalisation of high mantissa.
Correct misleading comment: "The hidden bit of the lo mantissa is
zero" is not always true as can be seen from the code rounding the hi
mantissa. Also by inspection, lzcount can never be less than zero so
remove that test. Lastly, masking bitfields to their widths can be
left to the compiler.
- mpn2ldbl.c: The overflow checks here on rounding of high double were
just plain wrong. Incrementing the exponent must be accompanied by a
shift right of the mantissa to keep the value unchanged. Above notes
for ldbl_insert_mantissa are also relevant.
[BZ #15680]
* sysdeps/ieee754/ldbl-128ibm/e_rem_pio2l.c: Comment fix.
* sysdeps/ieee754/ldbl-128ibm/printf_fphex.c
(PRINT_FPHEX_LONG_DOUBLE): Tidy code by moving -53 into ediff
calculation. Remove unnecessary test for denormal exponent.
* sysdeps/ieee754/ldbl-128ibm/ldbl2mpn.c (__mpn_extract_long_double):
Correct handling of denormals. Avoid undefined shift behaviour.
Correct normalisation of low mantissa when low double is denormal.
* sysdeps/ieee754/ldbl-128ibm/math_ldbl.h
(ldbl_extract_mantissa): Likewise. Comment. Use uint64_t* for hi64.
(ldbl_insert_mantissa): Make both hi64 and lo64 parms uint64_t.
Correct normalisation of low mantissa. Test for overflow of high
mantissa and normalise.
(ldbl_nearbyint): Use more readable constant for two52.
* sysdeps/ieee754/ldbl-128ibm/mpn2ldbl.c
(__mpn_construct_long_double): Fix test for overflow of high
mantissa and correct normalisation. Avoid undefined shift.
http://sourceware.org/ml/libc-alpha/2013-07/msg00001.html
This patch starts the process of supporting powerpc64 little-endian
long double in glibc. IBM long double is an array of two ieee
doubles, so making union ibm_extended_long_double reflect this fact is
the correct way to access fields of the doubles.
* sysdeps/ieee754/ldbl-128ibm/ieee754.h
(union ibm_extended_long_double): Define as an array of ieee754_double.
(IBM_EXTENDED_LONG_DOUBLE_BIAS): Delete.
* sysdeps/ieee754/ldbl-128ibm/printf_fphex.c: Update all references
to ibm_extended_long_double and IBM_EXTENDED_LONG_DOUBLE_BIAS.
* sysdeps/ieee754/ldbl-128ibm/e_exp10l.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_expl.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/ldbl2mpn.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/math_ldbl.h: Likewise.
* sysdeps/ieee754/ldbl-128ibm/mpn2ldbl.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/s_nearbyintl.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/strtold_l.c: Likewise.
* sysdeps/ieee754/ldbl-128ibm/x2y2m1l.c: Likewise.
The compiler is smart enough to convert those into double for powerpc,
but if we put them as doubles, it adds overhead by performing those
operations in floating point mode.
The mantissa of mp_no is intended to take only integral values. This
is a relatively good choice for powerpc due to its 4 fpus, but not for
other architectures, which suffer due to this choice. This change
makes the default mantissa a long integer and allows powerpc to
override it. Additionally, some operations have been optimized for
integer manipulation, resulting in a significant improvement in
performance.
The patch increase the high value to check if expl overflows. Current
high mark value is not really correct, the algorithm accepts high values.
It also adds a correct wrapper function to check for overflow and underflow.
Using long throughout like powerpc does is beneficial since it reduces
the need to switch to 32-bit instructions. It gives a very minor
performance improvement.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_ceil.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_finite.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_floor.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_frexp.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_isinf.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_isnan.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_llround.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_logb.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_lround.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_modf.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_nearbyint.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_remquo.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_rint.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_round.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_scalbln.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_scalbn.c: New file.
* sysdeps/ieee754/ldbl-opt/wordsize-64/s_trunc.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/Implies: Add
ieee754/ldbl-opt/wordsize-64.
* sysdeps/powerpc/powerpc64/Implies: Add
ieee754/dbl-64/wordsize-64.
The mpexp code has an access into m1np:
for (i=n-1; i>0; i--,n--) { if (m1np[i][p]+m2>0) break; }
which could break for p >= 18 or i >= 7. Fortunately this code is
never called due to the way the exp function is implemented since
values having exponent less than -55 return 1.0. Make sure that if it
gets called in future, it is trapped.
With help from Joseph Myers.
* sysdeps/ieee754/ldbl-128/s_atanl.c (__atanl): Handle tiny and
very large arguments properly.
* math/libm-test.inc (atan_test): New tests.
(atan2_test): New tests.
* sysdeps/sparc/fpu/libm-test-ulps: Update.
* sysdeps/x86_64/fpu/libm-test-ulps: Update.
With help from Joseph Myers.
* sysdeps/ieee754/flt-32/e_j0f.c (__ieee754_y0f): Adjust tinyness
cutoff to 2**-13.
* sysdeps/ieee754/flt-32/e_j1f.c (__ieee754_y1f): Adjust tinyness
cutoff to 2**-25.
* sysdeps/ieee754/ldbl-128/e_j0l.c (U0): New constant.
( __ieee754_y0l): Avoid arithmetic underflow when 'x' is very
small.
* sysdeps/ieee754/ldbl-128/e_j1l.c (__ieee754_y1l): Likewise.
* math/libm-test.inc (y0_test): New tests.
(y1_test): New tests.
* sysdeps/i386/fpu/libm-test-ulps: Update.
* sysdeps/x86_64/fpu/libm-test-ulps: Update.
* sysdeps/sparc/fpu/libm-test-ulps: Update.
* sysdeps/i386/i686/fpu/multiarch/Makefile (sysdep_routines):
Add s_sinf-sse2, s_conf-sse2.
* sysdeps/i386/i686/fpu/multiarch/s_sinf-sse2.S: New file.
* sysdeps/i386/i686/fpu/multiarch/s_cosf-sse2.S: New file.
* sysdeps/i386/i686/fpu/multiarch/s_sinf.c: New file.
* sysdeps/i386/i686/fpu/multiarch/s_cosf.c: New file.
* sysdeps/ieee754/flt-32/s_sinf.c (SINF, SINF_FUNC): Add macros
for using routine as __sinf_ia32.
Use macro for function declaration and weak_alias.
* sysdeps/ieee754/flt-32/s_cosf.c (COSF, COSF_FUNC): Add macros
for using routine as __cosf_ia32.
Use macro for function declaration and weak_alias.
* sysdeps/i386/i686/fpu/multiarch/e_expf-sse2.S: Fix Copyright.
* sysdeps/i386/i686/fpu/multiarch/e_expf.c: Fix Copyright.
* sysdeps/x86_64/fpu/s_sinf.S: New file.
* sysdeps/x86_64/fpu/s_cosf.S: New file.
* sysdeps/x86_64/fpu/libm-test-ulps: Update.
* math/libm-test.inc (cos_test): Add more test cases.
(sin_test): Likewise.
(sincos_test): Likewise.
POSIX 2008 states that if the input for 'logb[f|l]' is a subnormal number
it should be treated as if it were normalized. This means the
implementation should calculate the log2 of the mantissa and add it to the
subnormal exponent (-126 for float and -1022 for double and IBM long
double). This patch takes care of that.
[BZ# 6794]
* sysdeps/ieee754/ldbl-96/s_ilogbl.c: Moved to ...
* sysdeps/ieee754/ldbl-96/e_ilogbl.c: ... here.
Rename __ilogbl to __ieee754_ilogbl and remove weak_alias.
* sysdeps/ieee754/ldbl-128/s_ilogbl.c: Moved to ...
* sysdeps/ieee754/ldbl-128/e_ilogbl.c: ... here.
Rename __ilogbl to __ieee754_ilogbl and remove weak_alias.
* sysdeps/ieee754/ldbl-64-128/s_ilogbl.c: Moved to ...
* sysdeps/ieee754/ldbl-64-128/e_ilogbl.c: ... here.
* sysdeps/sparc/sparc64/soft-fp/s_ilogbl.c: Moved to ...
* sysdeps/sparc/sparc64/soft-fp/e_ilogbl.c: ... here.
Rename __ilogbl to __ieee754_ilogbl and remove weak_alias.
[BZ #6794]
Following Joseph comments about bug 6794, here is a proposed fix. It turned out
to be a large fix mainly because I had to move some file along to follow libm
files/names conventions.
Basically I have added wrappers (w_ilogb.c, w_ilogbf.c, w_ilogbl.c) that now calls
the symbol '__ieee754_ilogb'. The wrappers checks for '__ieee754_ilogb' output and
set the errno and raise exceptions as expected.
The '__ieee754_ilogb' is implemented in sysdeps. I have moved the 's_ilogb[f|l]' files
to e_ilogb[f|l] and renamed the '__ilogb[f|l]' to '__ieee754_ilogb[f|l]'.
I also found out a bug in i386 and x86-64 assembly coded ilogb implementation where
it raises a FE_DIVBYZERO when argument is '0.0'. I corrected this issue as well.
Finally I added the errno and FE_INVALID tests for 0.0, NaN and +-InF argument. Tested
on i386, x86-64, ppc32 and ppc64.
* sysdeps/ieee754/ldbl-128/s_nearbyintl.c (__nearbyintl): Do not
manipulate bits before adding and subtracting TWO112[sx].
* sysdeps/ieee754/ldbl-128/s_rintl.c (__rintl): Likewise.
