All the changes are in comments or '#error' messages.
Applying this commit results in bit-identical rebuild of iconvdata/*.so
Reviewed-by: Florian Weimer <fw@deneb.enyo.de>
And use a packed structure instead. The compiler generates optimized
unaligned code if the architecture supports it.
Checked on x86_64-linux-gnu and i686-linux-gnu.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
I (and maybe one or two others) added a (C) to the copyright notice
regardless of the contribution checklist[1] not mentioning it. Fix all
these instances so that the notice reads as "Copyright The GNU Toolchain
Authors" across the source code.
[1] https://sourceware.org/glibc/wiki/Contribution%20checklist
Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Bugfix 27256 has introduced another issue:
In conversion from ISO-2022-JP-3 encoding, it is possible
to force iconv to emit extra NUL character on internal state reset.
To do this, it is sufficient to feed iconv with escape sequence
which switches active character set.
The simplified check 'data->__statep->__count != ASCII_set'
introduced by the aforementioned bugfix picks that case and
behaves as if '\0' character has been queued thus emitting it.
To eliminate this issue, these steps are taken:
* Restore original condition
'(data->__statep->__count & ~7) != ASCII_set'.
It is necessary since bits 0-2 may contain
number of buffered input characters.
* Check that queued character is not NUL.
Similar step is taken for main conversion loop.
Bundled test case follows following logic:
* Try to convert ISO-2022-JP-3 escape sequence
switching active character set
* Reset internal state by providing NULL as input buffer
* Ensure that nothing has been converted.
Signed-off-by: Nikita Popov <npv1310@gmail.com>
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The conversion loop to the internal encoding does not follow
the interface contract that __GCONV_FULL_OUTPUT is only returned
after the internal wchar_t buffer has been filled completely. This
is enforced by the first of the two asserts in iconv/skeleton.c:
/* We must run out of output buffer space in this
rerun. */
assert (outbuf == outerr);
assert (nstatus == __GCONV_FULL_OUTPUT);
This commit solves this issue by queuing a second wide character
which cannot be written immediately in the state variable, like
other converters already do (e.g., BIG5-HKSCS or TSCII).
Reported-by: Tavis Ormandy <taviso@gmail.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
2004-08-02 Jakub Jelinek <jakub@redhat.com>
* iconvdata/ibm932.c (BODY): Avoid binary search for ch >= 0xffff.
Always treat high as highest number in range + 1.
* iconvdata/ibm943.c (BODY): Likewise.
2004-07-31 Bruno Haible <bruno@clisp.org>
* iconvdata/JISX0213.TXT: Updated to JISX0213 plane 1 version 2004.
* iconvdata/jisx0213.c (__jisx0213_to_ucs_main,
__jisx0213_to_ucs_pagestart, __jisx0213_from_ucs_level1,
__jisx0213_from_ucs_level2): Regenerated.
* iconvdata/jisx0213.h (jisx0213_added_in_2004_p): New function.
* iconvdata/iso-2022-jp-3.c (JISX0213_1_2000_set): Renamed from
JISX0213_1_set.
(JISX0213_1_2004_set): New enum value.
(BODY for FROM_LOOP): Treat ESC $ ( Q like ESC $ ( O.
(BODY for TO_LOOP): For JISX 0213 plane 1 characters, emit ESC $ ( O
when possible, ESC $ ( Q when needed.
* iconvdata/testdata/EUC-JISX0213: Add the 10 new characters.
* iconvdata/testdata/EUC-JISX0213..UTF8: Update.
* iconvdata/testdata/SHIFT_JISX0213: Add the 10 new characters.
* iconvdata/testdata/SHIFT_JISX0213..UTF8: Update.
* iconvdata/testdata/ISO-2022-JP-3: Add the 10 new JISX0213 characters.
* iconvdata/testdata/ISO-2022-JP-3..UTF8: Update.
2004-07-22 Bruno Haible <bruno@clisp.org>
* iconvdata/gconv-modules (ISO-8859-7): Add alias ISO_8859-7:2003.
* malloc/hooks.c (realloc_check): If mremap succeeds actually use
the result.
* sysdeps/i386/i486/bits/string.h: Add early clobber for "a"
output parameter. Reported by Denis Zaitsev <zzz@cd-club.ru>.
* Versions.def (libthread_db): Add GLIBC_2.3.
2002-07-09 Jakub Jelinek <jakub@redhat.com>
* libio/freopen.c (freopen): Reset jump tables, use mmap stdio for the
new stream if possible.
* libio/freopen64.c (freopen64): Likewise.
* libio/Makefile (tests): Add tst-freopen.
* libio/tst-freopen.c: New file.
2002-07-03 Jakub Jelinek <jakub@redhat.com>
* libio/stdio.h (obstack_printf): Add format attribute.
(obstack_vprintf): Likewise.
2002-07-04 Jakub Jelinek <jakub@redhat.com>
* posix/regex_internal.h (re_token_t): Shrink the structure to 8
resp. 16 bytes on 32-bit resp. 64-bit platforms.
(re_charset_t, re_string_t): Reorder structure members for 64-bit
architectures.
2002-07-04 Jakub Jelinek <jakub@redhat.com>
* posix/regcomp.c (re_compile_internal): Move clearing of dfa...
(init_dfa): ...here.
2002-07-05 Isamu Hasegawa <isamu@yamato.ibm.com>
* posix/regcomp.c (parse_sub_exp): Add nodes OP_OPEN_SUBEXP and
OP_CLOSE_SUBEXP at both ends of sub expressions instead of SUBEXP
nodes in parse trees.
(calc_first): Fix the Definition of `firsts' of OP_OPEN_SUBEXP
and OP_CLOSE_SUBEXP, and remove the definition of SUBEXP.
(calc_epsdest): Treat OP_OPEN_SUBEXP and OP_CLOSE_SUBEXP as the
nodes which can epsilon transit.
* posix/regex_internal.h: Add OP_OPEN_SUBEXP and OP_CLOSE_SUBEXP
to IS_EPSILON_NODE macro.
* posix/regexec.c (update_regs): New function to update registers
by refering OP_OPEN_SUBEXP and OP_CLOSE_SUBEXP nodes.
(proceed_next_node): Use the entities of the nodes to determine
whether the node can transit to.
(set_regs): Use update_regs function to update registers.
2002-07-03 Jakub Jelinek <jakub@redhat.com>
* posix/bug-regex4.c (main): Remove duplicate setlocale.
* posix/bug-regex6.c: New file.
* posix/Makefile (tests): Add bug-regex6.
(bug-regex6-ENV): Set LOCPATH.
2002-07-10 Ulrich Drepper <drepper@redhat.com>
2002-05-26 Bruno Haible <bruno@clisp.org>
* iconv/loop.c (STANDARD_FROM_LOOP_ERR_HANDLER): New macro.
(STANDARD_TO_LOOP_ERR_HANDLER): Renamed from STANDARD_ERR_HANDLER.
All callers changed.
* iconv/gconv_simple.c (ascii_internal_loop): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(utf8_internal_loop): Likewise.
(ucs2_internal_loop): Likewise.
(internal_ucs2_loop): Perform error handling like in
STANDARD_FROM_LOOP_ERR_HANDLER.
* iconvdata/unicode.c (BODY for TO_LOOP): Perform error handling like
in STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
* iconvdata/utf-16.c (BODY for TO_LOOP): Perform error handling like
in STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
* iconvdata/utf-32.c (BODY for TO_LOOP): Perform error handling like
in STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
* iconvdata/big5.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
* iconvdata/iso-2022-jp.c (BODY for FROM_LOOP): Likewise.
* iconvdata/8bit-gap.c (BODY for FROM_LOOP): Likewise.
* iconvdata/8bit-generic.c (BODY for FROM_LOOP): Likewise.
* iconvdata/ansi_x3.110.c (BODY for FROM_LOOP): Likewise.
* iconvdata/armscii-8.c (BODY for FROM_LOOP): Likewise.
* iconvdata/cp1255.c (BODY for FROM_LOOP): Likewise.
* iconvdata/cp1258.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-cn.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-jisx0213.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-jp.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-kr.c (BODY for FROM_LOOP): Likewise.
* iconvdata/euc-tw.c (BODY for FROM_LOOP): Likewise.
* iconvdata/big5hkscs.c (BODY for FROM_LOOP): Likewise.
* iconvdata/gb18030.c (BODY for FROM_LOOP): Likewise.
* iconvdata/gbk.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-cn-ext.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-cn.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-jp-3.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso-2022-kr.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso646.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso_6937-2.c (BODY for FROM_LOOP): Likewise.
* iconvdata/iso_6937.c (BODY for FROM_LOOP): Likewise.
* iconvdata/johab.c (BODY for FROM_LOOP): Likewise.
* iconvdata/shift_jisx0213.c (BODY for FROM_LOOP): Likewise.
* iconvdata/sjis.c (BODY for FROM_LOOP): Likewise.
* iconvdata/t.61.c (BODY for FROM_LOOP): Likewise.
* iconvdata/uhc.c (BODY for FROM_LOOP): Likewise.
* iconvdata/utf-7.c (BODY for FROM_LOOP): Likewise.
* iconvdata/gbbig5.c (BODY for FROM_LOOP): Likewise. When ignoring
an error, still set result = __GCONV_ILLEGAL_INPUT.
(BODY for TO_LOOP): Likewise.
* iconvdata/ibm930.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm932.c: Include <dlfcn.h> and <stdint.h>.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm933.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm935.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm937.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm939.c (BODY for FROM_LOOP): For error handling use
STANDARD_FROM_LOOP_ERR_HANDLER.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/ibm943.c: Include <dlfcn.h> and <stdint.h>.
(BODY for FROM_LOOP): Use STANDARD_FROM_LOOP_ERR_HANDLER for error
handling.
(BODY for TO_LOOP): Here use STANDARD_TO_LOOP_ERR_HANDLER.
* iconvdata/gbgbk.c (BODY for FROM_LOOP): Update.
* iconvdata/iso8859-1.c (BODY for TO_LOOP): Update.
* iconvdata/tcvn5712-1.c (BODY for TO_LOOP): Update.
2002-06-28 Kaz Kojima <kkojima@rr.iij4u.or.jp>
* sysdeps/sh/dl-machine.h (elf_machine_load_address): Use local
labels in assembler instructions.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/testdata/EUC-JISX0213: New file.
* iconvdata/testdata/EUC-JISX0213..UTF8: New file.
* iconvdata/testdata/SHIFT_JISX0213: New file.
* iconvdata/testdata/SHIFT_JISX0213..UTF8: New file.
* iconvdata/testdata/ISO-2022-JP-3: New file.
* iconvdata/testdata/ISO-2022-JP-3..UTF8: New file.
* iconvdata/TESTS: Add EUC-JISX0213, SHIFT_JISX0213, ISO-2022-JP-3.
* iconvdata/run-iconv-test.sh: Skip the charmap test if the charmap
contains comments indicating precomposed characters.
2002-04-21 Bruno Haible <bruno@clisp.org>
* iconvdata/iso-2022-jp-3.c: New file.
* iconvdata/gconv-modules (ISO-2022-JP-3): New module.
* iconvdata/Makefile (modules): Add ISO-2022-JP-3.
(LDFLAGS-ISO-2022-JP-3.so): New variable.
(ISO-2022-JP-3.so): Depend on libJIS.so and libJISX0213.so.
(distribute): Add iso-2022-jp-3.c.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/tst-table-to.c (main): Restore handling of
transliterations, broken by last patch.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/TESTS: Add IBM922, IBM1124, IBM1129.
2002-05-12 Bruno Haible <bruno@clisp.org>
* iconvdata/euc-jisx0213.c (BODY for FROM_LOOP): Don't advance inptr
when there is not enough room in the output buffer.
* iconvdata/shift_jisx0213.c (BODY for FROM_LOOP): Likewise.
2002-04-20 Bruno Haible <bruno@clisp.org>
* iconv/skeleton.c ({FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO}): New
macros.
(RESET_INPUT_BUFFER): Use them instead of {MIN,MAX}_NEEDED_{FROM,TO}.
Also make this macro work if FROM_DIRECTION evaluates to 0.
(gconv_init): Use the new macros instead of {MIN,MAX}_NEEDED_{FROM,TO}.
(FUNCTION_NAME): Likewise, in the handling of unaligned accesses and
of consume_incomplete.
* iconvdata/cp1255.c: Define {FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO}
instead of {MIN,MAX}_NEEDED_{FROM,TO}. Set FROM_LOOP_MAX_NEEDED_TO to
4, not 8.
* iconvdata/cp1258.c: Likewise.
* iconvdata/ibm930.c: Define {FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO}
instead of {MIN,MAX}_NEEDED_{FROM,TO}. Set TO_LOOP_MAX_NEEDED_TO to 3,
not 2.
* iconvdata/ibm933.c: Likewise.
* iconvdata/ibm935.c: Likewise.
* iconvdata/ibm937.c: Likewise.
* iconvdata/ibm939.c: Likewise.
* iconvdata/iso-2022-cn.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}. Set TO_LOOP_MAX_NEEDED_TO to 6, not 4.
* iconvdata/iso-2022-cn-ext.c: Likewise.
* iconvdata/iso-2022-jp.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}.
* iconvdata/tcvn5712-1.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}. Set FROM_LOOP_MAX_NEEDED_FROM to 1, not 2.
* iconvdata/euc-jisx0213.c: Define
{FROM,TO}_LOOP_{MIN,MAX}_NEEDED_{FROM,TO} instead of
{MIN,MAX}_NEEDED_{FROM,TO}. Set FROM_LOOP_MAX_NEEDED_TO to 8, not 4.
* iconvdata/shift_jisx0213.c: Likewise.
2002-05-14 Franz Sirl <Franz.Sirl-kernel@lauterbach.com>
* sysdeps/powerpc/Makefile (sysdeps_routines): Add divdi3, cmpdi2,
shdi3, fixdfdi, fixsfdi in csu dir.
(shared-only-routines): Likewise.
* sysdeps/wordsize-32/cmpdi2.c: New file.
* sysdeps/wordsize-32/shdi3.c: New file.
* sysdeps/wordsize-32/fixdfdi.c: New file.
* sysdeps/wordsize-32/fixsfdi.c: New file.
* sysdeps/powerpc/Versions (__divdi3, __moddi3, __udivdi3, __umoddi3,
__ashldi3, __ashrdi3, __lshrdi3, __cmpdi2, __ucmpdi2, __fixdfdi,
__fixsfdi, __fixunsdfdi, __fixunssfdi): Export at GLIBC_2.0.
* sysdeps/unix/sysv/linux/semctl.c (__old_semctl, __new_semctl):
Only use va_arg if the argument will be used [PR libc/3259].
2002-05-06 Michael A. Schmidt <mmaass@us.ibm.com>
* catgets/catgets.c (catopen): Free buffer if internal catalog
open fails.
2002-05-14 Ulrich Drepper <drepper@redhat.com>
* debug/xtrace.sh: Make more robust. For realtime tracking use
non-buffering methods.
* debug/pcprofiledump.c: Implement -u option.
Patch by Roger Luethi <rl@hellgate.ch>.
