In several converters, a __GCONV_ILLEGAL_INPUT result gets overwritten
with __GCONV_FULL_OUTPUT. As a result, iconv (the function) returns
E2BIG instead of EILSEQ. The iconv program does not see the original
EILSEQ failure, does not recognize the invalid input, and may
incorrectly exit successfully.
To address this, a new __flags bit is used to indicate a sticky input
error state. All __GCONV_ILLEGAL_INPUT results are replaced with a
function call that sets this new __GCONV_ENCOUNTERED_ILLEGAL_INPUT and
returns __GCONV_ILLEGAL_INPUT. The iconv program checks for
__GCONV_ENCOUNTERED_ILLEGAL_INPUT and overrides the exit status.
The converter changes introducing __gconv_mark_illegal_input are
mostly mechanical, except for the res variable initialization in
iconvdata/iso-2022-jp.c: this error gets overwritten with __GCONV_OK
and other results in the following code. If res ==
__GCONV_ILLEGAL_INPUT afterwards, STANDARD_TO_LOOP_ERR_HANDLER below
will handle it.
The __gconv_mark_illegal_input changes do not alter the errno value
set by the iconv function. This is simpler to implement than
reviewing each __GCONV_FULL_OUTPUT result and adjust it not to
override a previous __GCONV_ILLEGAL_INPUT result. Doing it that way
would also change some E2BIG errors in to EILSEQ errors, so it had to
be done conditionally (under a flag set by the iconv program only), to
avoid confusing buffer management in other applications.
Reviewed-by: DJ Delorie <dj@redhat.com>
Only return __GCONV_INCOMPLETE_INPUT for a partial match when the end of
the input buffer is reached. Otherwise it is a non-match, and other
patterns should be tried.
In the future, this will result in a compilation failure if the
macros are unexpectedly undefined (due to header inclusion ordering
or header inclusion missing altogether).
Assembler sources are more difficult to convert. In many cases,
they are hand-optimized for the mangling and no-mangling variants,
which is why they are not converted.
sysdeps/s390/s390-32/__longjmp.c and sysdeps/s390/s390-64/__longjmp.c
are special: These are C sources, but most of the implementation is
in assembler, so the PTR_DEMANGLE macro has to be undefined in some
cases, to match the assembler style.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This allows us to define a generic no-op version of PTR_MANGLE and
PTR_DEMANGLE. In the future, we can use PTR_MANGLE and PTR_DEMANGLE
unconditionally in C sources, avoiding an unintended loss of hardening
due to missing include files or unlucky header inclusion ordering.
In i386 and x86_64, we can avoid a <tls.h> dependency in the C
code by using the computed constant from <tcb-offsets.h>. <sysdep.h>
no longer includes these definitions, so there is no cyclic dependency
anymore when computing the <tcb-offsets.h> constants.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
On 32-bit machines this has no affect. On 64-bit machines
{u}int_fast{16|32} are set as {u}int64_t which is often not
ideal. Particularly x86_64 this change both saves code size and
may save instruction cost.
Full xcheck passes on x86_64.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
Prior to the 2.20 release, the function was just changed to fail
unconditionally, in commit a1a6a401ab.
This commit removes the function completely, including gconv bits
which depend on it.
This changes the gconv ABI, which is not a public interface.
before calling it.
2005-12-27 David S. Miller <davem@sunset.davemloft.net>
* sysdeps/unix/sysv/linux/sparc/Versions: Add new errlist-compat
entry for up to 134 errnos.
2001-07-06 Paul Eggert <eggert@twinsun.com>
* manual/argp.texi: Remove ignored LGPL copyright notice; it's
not appropriate for documentation anyway.
* manual/libc-texinfo.sh: "Library General Public License" ->
"Lesser General Public License".
2001-07-06 Andreas Jaeger <aj@suse.de>
* All files under GPL/LGPL version 2: Place under LGPL version
2.1.
* iconv/gconv_trans.c: Correct a few bugs in the search loop. Remove
remainders of hash table.
* locale/categories.def: Remove remainders of transliteration
hash table.
* locale/langinfo.h: Likewise.
* locale/programs/ld-ctype.c: Likewise. Fix code to write out
transliteration tables.
* locale/gen-translit.pl: New file.
* locale/C-translit.h.in: New file.
* locale/C-ctype.c: Include C-translit.h. Initialize transliteration
data pointers with data from this file.
* locale/Makefile (distribute): Add C-translit.h.in, C-translit.h,
and gen-translit.pl.
Add rule to generate C-translit.h.
(profile_fixup): Don't define for __BOUNDED_POINTERS__.
* sysdeps/i386/dl-machine.h: Don't use regparm attribute for
__BOUNDED_POINTERS__.
(ELF_MACHINE_RUNTIME_TRAMPOLINE): Use non-regparm version
for __BOUNDED_POINTERS__.
* iconv/gconv_trans.c: Add #include <stdlib.h> to get malloc decl.
* elf/dl-runtime.c (fixup): Trampoline passes unbounded pointer.
(profile_fixup): Don't define for __BOUNDED_POINTERS__.
* sysdeps/i386/dl-machine.h: Don't use regparm attribute for
__BOUNDED_POINTERS__.
(ELF_MACHINE_RUNTIME_TRAMPOLINE): Use non-regparm version
for __BOUNDED_POINTERS__.
2000-06-19 Ulrich Drepper <drepper@redhat.com>
* iconv/gconv.h (__gconv_trans_fct): Add new parameter.
General namespace cleanup.
(struct __gconv_trans_data): Add next field.
(struct __gconv_step_data): Make __trans a pointer.
* iconv/gconv_conf.c: Split out code to find gconv directories from
__gconv_read_conf in new functions.
* iconv/gconv_int.h: Define new data structure and declare new
functions for handling of gconv directory list.
* iconv/gconv_open.c: Allow more than one error handling step being
used. Call function to load error handling module if it is none
of the builtin transformations.
* iconv/gconv_close.c: Add code to free transliteration data.
* iconv/gconv_trans.c: Add functions to load and unload modules
implementing transliteration etc.
* iconv/skeleton.c: Call all context functions now that more than
one module is allowed.
* iconv/loop.c (STANDARD_ERR_HANDLING): New macro.
* iconv/gconv_simple.c: Use STANDARD_ERR_HANDLING macro for places
where the full error handling using transliteration is needed.
* iconvdata/8bit-gap.c: Likewise.
* iconvdata/8bit-generic.c: Likewise.
* iconvdata/ansi_x3.110.c: Likewise.
* iconvdata/big5.c: Likewise.
* iconvdata/big5hkscs.c: Likewise.
* iconvdata/euc-cn.c: Likewise.
* iconvdata/euc-jp.c: Likewise.
* iconvdata/euc-kr.c: Likewise.
* iconvdata/euc-tw.c: Likewise.
* iconvdata/gbgbk.c: Likewise.
* iconvdata/gbk.c: Likewise.
* iconvdata/iso-2022-cn.c: Likewise.
* iconvdata/iso-2022-jp.c: Likewise.
* iconvdata/iso-2022-kr.c: Likewise.
* iconvdata/iso646.c: Likewise.
* iconvdata/iso8859-1.c: Likewise.
* iconvdata/iso_6937-2.c: Likewise.
* iconvdata/iso_6937.c: Likewise.
* iconvdata/johab.c: Likewise.
* iconvdata/sjis.c: Likewise.
* iconvdata/t.61.c: Likewise.
* iconvdata/uhc.c: Likewise.
* iconvdata/unicode.c: Likewise.
* iconvdata/utf-16.c: Likewise.
* libio/iofwide.c: Reset __trans member of __gconv_trans_data
structure correctly after last change.
* wcsmbs/btowc.c: Likewise.
* wcsmbs/mbrtowc.c: Likewise.
* wcsmbs/mbsnrtowcs.c: Likewise.
* wcsmbs/mbsrtowcs.c: Likewise.
* wcsmbs/wcrtomb.c: Likewise.
* wcsmbs/wcsnrtombs.c: Likewise.
* wcsmbs/wcsrtombs.c: Likewise.
* wcsmbs/wctob.c: Likewise.
* localedata/Makefile: Set -Wno-format for some files since gcc does
not know all the format specifiers.
2000-06-18 Ulrich Drepper <drepper@redhat.com>
* locale/loadlocale.c (_nl_unload_locale): Remove a bit of
unneeded code.
* locale/lc-time.c (_nl_init_era_entries): Likewise.
2000-06-17 Ulrich Drepper <drepper@redhat.com>
* iconv/gconv_trans.c: Implement handling if translit_ignore.
* locale/langinfo.h: Add entries for translit_ignore information.
* locale/categories.def: Add entries for new LC_CTYPE elements.
* locale/C-ctype.c: Add initializers for new fields. Use NULL
pointer instead of "" where possible.
* locale/programs/ld-ctype.c: Write out translit_ignore information.
* intl/Depend: Add localedata.
* intl/tst-gettext.c: Call setlocale for LC_CTYPE.
* intl/tst-gettext.sh: Set LOCPATH to localedata build dir.
* locale/langinfo.h: Add entries for default_missing information.
* locale/C-ctype.c: Add initializers for new fields.
* iconv/gconv_trans.c: If nothing matched, try to use default_missing
information.
* locale/categories.h: Add entries for all LC_CTYPE values.
* locale/programs/ld-ctype.c (ctype_output): Write out default_missing
information.
* localedata/tst-trans.c: Write out an error message if class is
not found.
