Commit Graph

2 Commits

Author SHA1 Message Date
Neil Roberts
57a2441791 hex_float: Use max_digits10 for the float precision
CPPreference.com has this description of digits10:

“The value of std::numeric_limits<T>::digits10 is the number of
 base-10 digits that can be represented by the type T without change,
 that is, any number with this many significant decimal digits can be
 converted to a value of type T and back to decimal form, without
 change due to rounding or overflow.”

This means that any number with this many digits can be represented
accurately in the corresponding type. A change in any digit in a
number after that may or may not cause it a different bitwise
representation. Therefore this isn’t necessarily enough precision to
accurately represent the value in text. Instead we need max_digits10
which has the following description:

“The value of std::numeric_limits<T>::max_digits10 is the number of
 base-10 digits that are necessary to uniquely represent all distinct
 values of the type T, such as necessary for
 serialization/deserialization to text.”

The patch includes a test case in hex_float_test which tries to do a
round-robin conversion of a number that requires more than 6 decimal
places to be accurately represented. This would fail without the
patch.

Sadly this also breaks a bunch of other tests. Some of the tests in
hex_float_test use ldexp and then compare it with a value which is not
the same as the one returned by ldexp but instead is the value rounded
to 6 decimals. Others use values that are not evenly representable as
a binary floating fraction but then happened to generate the same
value when rounded to 6 decimals. Where the actual value didn’t seem
to matter these have been changed with different values that can be
represented as a binary fraction.
2018-04-03 12:53:10 -04:00
GregF
f28b106173 InsertExtractElim: Split out DeadInsertElim as separate pass 2018-01-30 08:52:14 -05:00