TS 18661-1 generally defines libm functions taking sNaN arguments to
return qNaN and raise "invalid", even for the cases where a
corresponding qNaN argument would not result in a qNaN return. This
includes hypot with one argument being an infinity and the other being
an sNaN. This patch duly fixes hypot implementatations in glibc
(generic and powerpc) to ensure qNaN, computed by arithmetic on the
arguments, is returned in that case.
Various implementations do their checks for infinities and NaNs inline
by manipulating the representations of the arguments. For simplicity,
this patch just uses issignaling to check for sNaN arguments. This
could be inlined like the existing code (with due care about reversed
quiet NaN conventions, for implementations where that is relevant),
but given that all these checks are in cases where it's already known
at least one argument is not finite, which should be the uncommon
case, that doesn't seem worthwhile unless performance issues are
observed in practice.
Tested for x86_64, x86, mips64 and powerpc.
[BZ #20940]
* sysdeps/ieee754/dbl-64/e_hypot.c (__ieee754_hypot): Do not
return Inf for arguments Inf and sNaN.
* sysdeps/ieee754/flt-32/e_hypotf.c (__ieee754_hypotf): Likewise.
* sysdeps/ieee754/ldbl-128/e_hypotl.c (__ieee754_hypotl):
Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_hypotl.c (__ieee754_hypotl):
Likewise.
* sysdeps/ieee754/ldbl-96/e_hypotl.c (__ieee754_hypotl): Likewise.
* sysdeps/powerpc/fpu/e_hypot.c (TEST_INF_NAN): Do not return Inf
for arguments Inf and sNaN. When returning a NaN, compute it by
arithmetic on the arguments.
* sysdeps/powerpc/fpu/e_hypotf.c (TEST_INF_NAN): Likewise.
* math/libm-test.inc (pow_test_data): Add tests of sNaN arguments.
This runs the attached sed script against these files using
a regex which aggressively matches long double literals
when not obviously part of a comment.
Likewise, 5 digit or less integral constants are replaced
with integer constants, excepting the two cases of 0 used
in large tables, which are also the only integral values
of the form x.0*E0L encountered within these converted
files.
Likewise, -L(x) is transformed into L(-x).
Naturally, the script has a few minor hiccups which are
more clearly remedied via the attached fixup patch. Such
hiccups include, context-sensitive promotion to a real
type, and munging constants inside harder to detect
comment blocks.
Add a layer of macro indirection for long double files
which need to be built using another typename. Likewise,
add the L(num) macro used in a later patch to override
real constants.
These macros are only defined through the ldbl-128
math_ldbl.h header, thereby implicitly restricting
these macros to machines which back long double
with an IEEE binary128 format.
Likewise, appropriate changes are made for the few
files which indirectly include such ldbl-128 files.
These changes produce identical binaries for s390x,
aarch64, and ppc64.
Similar to various other bugs in this area, hypot functions can fail
to raise the underflow exception when the result is tiny and inexact
but one or more low bits of the intermediate result that is scaled
down (or, in the i386 case, converted from a wider evaluation format)
are zero. This patch forces the exception in a similar way to
previous fixes.
Note that this issue cannot arise for implementations of hypotf using
double (or wider) for intermediate evaluation (if hypotf should
underflow, that means the double square root is being computed of some
number of the form N*2^-298, for 0 < N < 2^46, which is exactly
represented as a double, and whatever the rounding mode such a square
root cannot have a mantissa with all zeroes after the initial 23
bits). Thus no changes are made to hypotf implementations in this
patch, only to hypot and hypotl.
Tested for x86_64, x86, mips64 and powerpc.
[BZ #18803]
* sysdeps/i386/fpu/e_hypot.S: Use DEFINE_DBL_MIN.
(MO): New macro.
(__ieee754_hypot) [PIC]: Load PIC register.
(__ieee754_hypot): Use DBL_NARROW_EVAL_UFLOW_NONNEG instead of
DBL_NARROW_EVAL.
* sysdeps/ieee754/dbl-64/e_hypot.c (__ieee754_hypot): Use
math_check_force_underflow_nonneg in case where result might be
tiny.
* sysdeps/ieee754/ldbl-128/e_hypotl.c (__ieee754_hypotl):
Likewise.
* sysdeps/ieee754/ldbl-128ibm/e_hypotl.c (__ieee754_hypotl):
Likewise.
* sysdeps/ieee754/ldbl-96/e_hypotl.c (__ieee754_hypotl): Likewise.
* sysdeps/powerpc/fpu/e_hypot.c (__ieee754_hypot): Likewise.
* math/auto-libm-test-in: Add more tests of hypot.
* math/auto-libm-test-out: Regenerated.
