Bugfix 27256 has introduced another issue:
In conversion from ISO-2022-JP-3 encoding, it is possible
to force iconv to emit extra NUL character on internal state reset.
To do this, it is sufficient to feed iconv with escape sequence
which switches active character set.
The simplified check 'data->__statep->__count != ASCII_set'
introduced by the aforementioned bugfix picks that case and
behaves as if '\0' character has been queued thus emitting it.
To eliminate this issue, these steps are taken:
* Restore original condition
'(data->__statep->__count & ~7) != ASCII_set'.
It is necessary since bits 0-2 may contain
number of buffered input characters.
* Check that queued character is not NUL.
Similar step is taken for main conversion loop.
Bundled test case follows following logic:
* Try to convert ISO-2022-JP-3 escape sequence
switching active character set
* Reset internal state by providing NULL as input buffer
* Ensure that nothing has been converted.
Signed-off-by: Nikita Popov <npv1310@gmail.com>
(cherry picked from commit ff012870b2)
The conversion loop to the internal encoding does not follow
the interface contract that __GCONV_FULL_OUTPUT is only returned
after the internal wchar_t buffer has been filled completely. This
is enforced by the first of the two asserts in iconv/skeleton.c:
/* We must run out of output buffer space in this
rerun. */
assert (outbuf == outerr);
assert (nstatus == __GCONV_FULL_OUTPUT);
This commit solves this issue by queuing a second wide character
which cannot be written immediately in the state variable, like
other converters already do (e.g., BIG5-HKSCS or TSCII).
Reported-by: Tavis Ormandy <taviso@gmail.com>
(cherry picked from commit 7d88c6142c)
2004-08-02 Jakub Jelinek <jakub@redhat.com>
* iconvdata/ibm932.c (BODY): Avoid binary search for ch >= 0xffff.
Always treat high as highest number in range + 1.
* iconvdata/ibm943.c (BODY): Likewise.
2004-07-31 Bruno Haible <bruno@clisp.org>
* iconvdata/JISX0213.TXT: Updated to JISX0213 plane 1 version 2004.
* iconvdata/jisx0213.c (__jisx0213_to_ucs_main,
__jisx0213_to_ucs_pagestart, __jisx0213_from_ucs_level1,
__jisx0213_from_ucs_level2): Regenerated.
* iconvdata/jisx0213.h (jisx0213_added_in_2004_p): New function.
* iconvdata/iso-2022-jp-3.c (JISX0213_1_2000_set): Renamed from
JISX0213_1_set.
(JISX0213_1_2004_set): New enum value.
(BODY for FROM_LOOP): Treat ESC $ ( Q like ESC $ ( O.
(BODY for TO_LOOP): For JISX 0213 plane 1 characters, emit ESC $ ( O
when possible, ESC $ ( Q when needed.
* iconvdata/testdata/EUC-JISX0213: Add the 10 new characters.
* iconvdata/testdata/EUC-JISX0213..UTF8: Update.
* iconvdata/testdata/SHIFT_JISX0213: Add the 10 new characters.
* iconvdata/testdata/SHIFT_JISX0213..UTF8: Update.
* iconvdata/testdata/ISO-2022-JP-3: Add the 10 new JISX0213 characters.
* iconvdata/testdata/ISO-2022-JP-3..UTF8: Update.
2004-07-22 Bruno Haible <bruno@clisp.org>
* iconvdata/gconv-modules (ISO-8859-7): Add alias ISO_8859-7:2003.
* malloc/hooks.c (realloc_check): If mremap succeeds actually use
the result.
* sysdeps/i386/i486/bits/string.h: Add early clobber for "a"
output parameter. Reported by Denis Zaitsev <zzz@cd-club.ru>.
* Versions.def (libthread_db): Add GLIBC_2.3.
2002-07-09 Jakub Jelinek <jakub@redhat.com>
* libio/freopen.c (freopen): Reset jump tables, use mmap stdio for the
new stream if possible.
* libio/freopen64.c (freopen64): Likewise.
* libio/Makefile (tests): Add tst-freopen.
* libio/tst-freopen.c: New file.
2002-07-03 Jakub Jelinek <jakub@redhat.com>
* libio/stdio.h (obstack_printf): Add format attribute.
(obstack_vprintf): Likewise.
2002-07-04 Jakub Jelinek <jakub@redhat.com>
* posix/regex_internal.h (re_token_t): Shrink the structure to 8
resp. 16 bytes on 32-bit resp. 64-bit platforms.
(re_charset_t, re_string_t): Reorder structure members for 64-bit
architectures.
2002-07-04 Jakub Jelinek <jakub@redhat.com>
* posix/regcomp.c (re_compile_internal): Move clearing of dfa...
(init_dfa): ...here.
2002-07-05 Isamu Hasegawa <isamu@yamato.ibm.com>
* posix/regcomp.c (parse_sub_exp): Add nodes OP_OPEN_SUBEXP and
OP_CLOSE_SUBEXP at both ends of sub expressions instead of SUBEXP
nodes in parse trees.
(calc_first): Fix the Definition of `firsts' of OP_OPEN_SUBEXP
and OP_CLOSE_SUBEXP, and remove the definition of SUBEXP.
(calc_epsdest): Treat OP_OPEN_SUBEXP and OP_CLOSE_SUBEXP as the
nodes which can epsilon transit.
* posix/regex_internal.h: Add OP_OPEN_SUBEXP and OP_CLOSE_SUBEXP
to IS_EPSILON_NODE macro.
* posix/regexec.c (update_regs): New function to update registers
by refering OP_OPEN_SUBEXP and OP_CLOSE_SUBEXP nodes.
(proceed_next_node): Use the entities of the nodes to determine
whether the node can transit to.
(set_regs): Use update_regs function to update registers.
2002-07-03 Jakub Jelinek <jakub@redhat.com>
* posix/bug-regex4.c (main): Remove duplicate setlocale.
* posix/bug-regex6.c: New file.
* posix/Makefile (tests): Add bug-regex6.
(bug-regex6-ENV): Set LOCPATH.
2002-07-10 Ulrich Drepper <drepper@redhat.com>
2002-05-26 Bruno Haible <bruno@clisp.org>
* iconv/loop.c (STANDARD_FROM_LOOP_ERR_HANDLER): New macro.
(STANDARD_TO_LOOP_ERR_HANDLER): Renamed from STANDARD_ERR_HANDLER.
All callers changed.
* iconv/gconv_simple.c (ascii_internal_loop): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(utf8_internal_loop): Likewise.
(ucs2_internal_loop): Likewise.
(internal_ucs2_loop): Perform error handling like in
STANDARD_FROM_LOOP_ERR_HANDLER.
* iconvdata/unicode.c (BODY for TO_LOOP): Perform error handling like
in STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
* iconvdata/utf-16.c (BODY for TO_LOOP): Perform error handling like
in STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
* iconvdata/utf-32.c (BODY for TO_LOOP): Perform error handling like
in STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
* iconvdata/big5.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
* iconvdata/iso-2022-jp.c (BODY for FROM_LOOP): Likewise.
* iconvdata/8bit-gap.c (BODY for FROM_LOOP): Likewise.
* iconvdata/8bit-generic.c (BODY for FROM_LOOP): Likewise.
* iconvdata/ansi_x3.110.c (BODY for FROM_LOOP): Likewise.
* iconvdata/armscii-8.c (BODY for FROM_LOOP): Likewise.
* iconvdata/cp1255.c (BODY for FROM_LOOP): Likewise.
* iconvdata/cp1258.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-cn.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-jisx0213.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-jp.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-kr.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-tw.c (BODY for FROM_LOOP): Likewise.
* iconvdata/big5hkscs.c (BODY for FROM_LOOP): Likewise.
* iconvdata/gb18030.c (BODY for FROM_LOOP): Likewise.
* iconvdata/gbk.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-cn-ext.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-cn.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-jp-3.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-kr.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso646.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso_6937-2.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso_6937.c (BODY for FROM_LOOP): Likewise.
* iconvdata/johab.c (BODY for FROM_LOOP): Likewise.
* iconvdata/shift_jisx0213.c (BODY for FROM_LOOP): Likewise.
* iconvdata/sjis.c (BODY for FROM_LOOP): Likewise.
* iconvdata/t.61.c (BODY for FROM_LOOP): Likewise.
* iconvdata/uhc.c (BODY for FROM_LOOP): Likewise.
* iconvdata/utf-7.c (BODY for FROM_LOOP): Likewise.
* iconvdata/gbbig5.c (BODY for FROM_LOOP): Likewise. When ignoring
an error, still set result = __GCONV_ILLEGAL_INPUT.
(BODY for TO_LOOP): Likewise.
* iconvdata/ibm930.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm932.c: Include <dlfcn.h> and <stdint.h>.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm933.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm935.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm937.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm939.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm943.c: Include <dlfcn.h> and <stdint.h>.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/gbgbk.c (BODY for FROM_LOOP): Update.
* iconvdata/iso8859-1.c (BODY for TO_LOOP): Update.
* iconvdata/tcvn5712-1.c (BODY for TO_LOOP): Update.
2002-06-28 Kaz Kojima <kkojima@rr.iij4u.or.jp>
* sysdeps/sh/dl-machine.h (elf_machine_load_address): Use local
labels in assembler instructions.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/testdata/EUC-JISX0213: New file.
* iconvdata/testdata/EUC-JISX0213..UTF8: New file.
* iconvdata/testdata/SHIFT_JISX0213: New file.
* iconvdata/testdata/SHIFT_JISX0213..UTF8: New file.
* iconvdata/testdata/ISO-2022-JP-3: New file.
* iconvdata/testdata/ISO-2022-JP-3..UTF8: New file.
* iconvdata/TESTS: Add EUC-JISX0213, SHIFT_JISX0213, ISO-2022-JP-3.
* iconvdata/run-iconv-test.sh: Skip the charmap test if the charmap
contains comments indicating precomposed characters.
2002-04-21 Bruno Haible <bruno@clisp.org>
* iconvdata/iso-2022-jp-3.c: New file.
* iconvdata/gconv-modules (ISO-2022-JP-3): New module.
* iconvdata/Makefile (modules): Add ISO-2022-JP-3.
(LDFLAGS-ISO-2022-JP-3.so): New variable.
(ISO-2022-JP-3.so): Depend on libJIS.so and libJISX0213.so.
(distribute): Add iso-2022-jp-3.c.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/tst-table-to.c (main): Restore handling of
transliterations, broken by last patch.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/TESTS: Add IBM922, IBM1124, IBM1129.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/euc-jisx0213.c (BODY for FROM_LOOP): Don't advance inptr
when there is not enough room in the output buffer.
* iconvdata/shift_jisx0213.c (BODY for FROM_LOOP): Likewise.
2002-04-20 Bruno Haible <bruno@clisp.org>
* iconv/skeleton.c ({FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO}): New
macros.
(RESET_INPUT_BUFFER): Use them instead of {MIN,MAX}_NEEDED_{FROM,TO}.
Also make this macro work if FROM_DIRECTION evaluates to 0.
(gconv_init): Use the new macros instead of {MIN,MAX}_NEEDED_{FROM,TO}.
(FUNCTION_NAME): Likewise, in the handling of unaligned accesses and
of consume_incomplete.
* iconvdata/cp1255.c: Define {FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO}
instead of {MIN,MAX}_NEEDED_{FROM,TO}. Set FROM_LOOP_MAX_NEEDED_TO to
4, not 8.
* iconvdata/cp1258.c: Likewise.
* iconvdata/ibm930.c: Define {FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO}
instead of {MIN,MAX}_NEEDED_{FROM,TO}. Set TO_LOOP_MAX_NEEDED_TO to 3,
not 2.
* iconvdata/ibm933.c: Likewise.
* iconvdata/ibm935.c: Likewise.
* iconvdata/ibm937.c: Likewise.
* iconvdata/ibm939.c: Likewise.
* iconvdata/iso-2022-cn.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}. Set TO_LOOP_MAX_NEEDED_TO to 6, not 4.
* iconvdata/iso-2022-cn-ext.c: Likewise.
* iconvdata/iso-2022-jp.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}.
* iconvdata/tcvn5712-1.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}. Set FROM_LOOP_MAX_NEEDED_FROM to 1, not 2.
* iconvdata/euc-jisx0213.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}. Set FROM_LOOP_MAX_NEEDED_TO to 8, not 4.
* iconvdata/shift_jisx0213.c: Likewise.
2002-05-14 Franz Sirl <Franz.Sirl-kernel@lauterbach.com>
* sysdeps/powerpc/Makefile (sysdeps_routines): Add divdi3, cmpdi2,
shdi3, fixdfdi, fixsfdi in csu dir.
(shared-only-routines): Likewise.
* sysdeps/wordsize-32/cmpdi2.c: New file.
* sysdeps/wordsize-32/shdi3.c: New file.
* sysdeps/wordsize-32/fixdfdi.c: New file.
* sysdeps/wordsize-32/fixsfdi.c: New file.
* sysdeps/powerpc/Versions (__divdi3, __moddi3, __udivdi3, __umoddi3,
__ashldi3, __ashrdi3, __lshrdi3, __cmpdi2, __ucmpdi2, __fixdfdi,
__fixsfdi, __fixunsdfdi, __fixunssfdi): Export at GLIBC_2.0.
* sysdeps/unix/sysv/linux/semctl.c (__old_semctl, __new_semctl):
Only use va_arg if the argument will be used [PR libc/3259].
2002-05-06 Michael A. Schmidt <mmaass@us.ibm.com>
* catgets/catgets.c (catopen): Free buffer if internal catalog
open fails.
2002-05-14 Ulrich Drepper <drepper@redhat.com>
* debug/xtrace.sh: Make more robust. For realtime tracking use
non-buffering methods.
* debug/pcprofiledump.c: Implement -u option.
Patch by Roger Luethi <rl@hellgate.ch>.
