This commit replaces string manipulation during `iconv_open' and iconv_prog
option parsing with a structured, flag based conversion specification. In
doing so, it alters the internal `__gconv_open' interface and accordingly
adjusts its uses.
This change fixes several hangs in the iconv program and therefore includes
a new test to exercise iconv_prog options that originally led to these hangs.
It also includes a new regression test for option handling in the iconv
function.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Commit ba7b4d294b ("Complete the
removal of __gconv_translit_find") added a declaration of the
GLIBC_PRIVATE function, __gconv_transliterate, to the installed
header <gconv.h>. It should have been added to the internal
<gconv_int.h> header.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
In iconv/gconv_conf.c, __gconv_get_path unnecessarily obtains a lock when
populating the array pointed to by __gconv_path_elem. The locking is not
necessary because all calls to __gconv_read_conf (which in turn calls
__gconv_get_path) are serialized using __libc_once.
This patch:
- removes all locking in __gconv_get_path;
- replaces all explicitly serialized __gconv_read_conf calls with calls to
__gconv_load_conf, a new wrapper that is serialized internally;
- adds a new test, iconv/tst-iconv_mt.c, to exercise iconv initialization,
usage, and cleanup in a multi-threaded program;
- indents __gconv_get_path correctly, removing tab characters (which makes
the patch look a little bigger than it really is).
After removing the unnecessary locking, it was confirmed that the test case
fails if the relevant __libc_once is removed. Additionally, four localedata
and iconvdata tests also fail. This gives confidence that the testsuite
sufficiently guards against some regressions relating to multi-threading
with iconv.
Tested on x86_64 and i686.
The variables __gconv_path_elem, __gconv_max_path_elem_len and function
__gconv_get_path declared in, as well as the type path_elem and macro
GCONV_NCHAR_GOAL defined in gconv_int.h are all used in only one iconv
compilation unit each. In addition, the extern declaration of the variable
__gconv_nmodules refers to a variable that does not exist any more.
Considering this, these symbols do not need to be exposed via a header file.
This patch removes the extern declarations from the header file and moves
the definitions to the compilation units where they are used.
Prior to the 2.20 release, the function was just changed to fail
unconditionally, in commit a1a6a401ab.
This commit removes the function completely, including gconv bits
which depend on it.
This changes the gconv ABI, which is not a public interface.
* iconv/gconv_cache.c (__gconv_lookup_cache): Return __GCONV_NULCONV
if from and to charsets are the same.
* iconv/gconv_db.c (__gconv_find_transform): Likewise.
* intl/dcigettext.c (_nl_find_msg): Return NULL even if __gconv_open
returns __GCONV_NOCONV, but not for __GCONV_NULCONV.
2007-07-17 Jakub Jelinek <jakub@redhat.com>
* wcsmbs/wchar.h: Only define wint_t if __need_wint_t.
Don't define wint_t when __need_mbstate_t unless it
is necessary.
(__mbstate_t): Use __WINT_TYPE__ rather than wint_t
in the typedef if possible.
* wctype/wctype.h (wint_t): Define by including
wchar.h with __need_wint_t instead of including stddef.h
with __need_wint_t and as fallback definining it ourselves.
* iconv/gconv.h (__need_wint_t): Define before including
wchar.h.
* sysdeps/gnu/_G_config.h: Don't include gconv.h if not _LIBC
or _GLIBCPP_USE_WCHAR_T.
(__need_wchar_t): Don't define
if not _LIBC or _GLIBCPP_USE_WCHAR_T.
(__need_wint_t): Don't define before including stddef.h,
define before including wchar.h only if _LIBC or
_GLIBCPP_USE_WCHAR_T.
(_G_iconv_t): Don't define if not _LIBC or _GLIBCPP_USE_WCHAR_T.
* sysdeps/mach/hurd/_G_config.h: Likewise.
* sysdeps/generic/_G_config.h: Likewise.
* libio/libio.h (__wunderflow, __wuflow, __woverflow): Only
prototype if _LIBC or _GLIBCPP_USE_WCHAR_T.
(_IO_getwc_unlocked, _IO_putwc_unlocked): Only define
if _LIBC or _GLIBCPP_USE_WCHAR_T.
* nscd/nscd_initgroups.c: Mark __gr_map_handle as hidden.
* nscd/nscd_getgr_r.c: Likewise.
* include/rpc/pmap_rmt.h: Mark all functions as hidden.
* nscd/nscd_getai.c (__hst_map_handle): Mark as hidden.
* nscd/nscd_gethst_r.c (__hst_map_handle): Likewise.
* stdlib/longlong.h (__clz_tab): Mark as hidden.
* iconv/gconv_int.h: Mark __gconv_lock as hidden.
* iconv/gconv_db.c: Don't define lock as static. Rename to
__gconv_lock and export from the file.
* iconv/gconv_int.h: Declare __gconv_lock.
* libio/iofclose.c [_LIBC] (_IO_new_fclose): Lock gconv lock
before __gconv_release_step calls.
Patch by Shunichi Sagawa <s-sagawa@jp.fujitsu.com>.
* iconv/gconv_simple.c (internal_ucs4_loop): Fix typo in last change.
* iconv/gconv_db.c
2002-11-30 Bruno Haible <bruno@clisp.org>
* iconv/gconv.h (__gconv_btowc_fct): New typedef.
(struct __gconv_step): New field __btowc_fct.
* wcsmbs/btowc.c (__btowc): Use the __btowc_fct shortcut if possible.
* iconv/gconv_int.h (__BUILTIN_TRANSFORM): Renamed from
__BUILTIN_TRANS.
(__gconv_btwoc_ascii): New declaration.
* iconv/gconv_simple.c (BUILTIN_TRANSFORMATION): Add BtowcFct argument.
(__gconv_btwoc_ascii): New function.
* iconv/gconv_builtin.h: Add BtowcFct argument to all
BUILTIN_TRANSFORMATION invocations.
* iconv/gconv_conf.c (BUILTIN_TRANSFORMATION): Add BtowcFct argument.
* iconv/iconvconfig.c (BUILTIN_TRANSFORMATION): Likewise.
* iconv/gconv_builtin.c (map): New field btowc_fct.
(BUILTIN_TRANSFORMATION): Add BtowcFct argument. Use it to initialize
btowc_fct field.
(__gconv_get_builtin_trans): Initialize __btowc_fct field.
* iconv/gconv_cache.c (find_module): Initialize __btowc_fct field.
* iconv/gconv_db.c (gen_steps, increment_counter): Likewise.
* wcsmbs/wcsmbsload.c (to_wc, to_mb): Likewise.
* iconv/skeleton.c: Document STORE_REST and FROM_ONEBYTE.
(gconv_init): Initialize __btowc_fct field.
Undefine EXTRA_LOOP_ARGS and FROM_ONEBYTE at the end.
* iconv/loop.c: Document ONEBYTE_BODY.
(gconv_btowc, FROM_ONEBYTE): Define if ONEBYTE_BODY is defined.
Undefine ONEBYTE_BODY at the end.
* iconvdata/8bit-generic.c (ONEBYTE_BODY): New macro.
* iconvdata/8bit-gap.c (NONNUL): New macro.
(BODY for FROM_LOOP): Use it.
(ONEBYTE_BODY): New macro.
* iconvdata/isiri-3342.c (HAS_HOLES): Set to 1.
(NONNUL): New macro.
* iconvdata/ansi_x3.110.c (ONEBYTE_BODY): New macro.
* iconvdata/armscii-8.c (ONEBYTE_BODY): New macro.
* iconvdata/cp1255.c (ONEBYTE_BODY): New macro.
* iconvdata/cp1258.c (ONEBYTE_BODY): New macro.
* iconvdata/tcvn5712-1.c (ONEBYTE_BODY): New macro.
* iconvdata/big5.c (ONEBYTE_BODY): New macro.
* iconvdata/big5hkscs.c (ONEBYTE_BODY): New macro.
* iconvdata/euc-cn.c (ONEBYTE_BODY): New macro.
* iconvdata/euc-jp.c (ONEBYTE_BODY): New macro.
* iconvdata/euc-jisx0213.c (ONEBYTE_BODY): New macro.
* iconvdata/euc-kr.c (ONEBYTE_BODY): New macro.
* iconvdata/euc-tw.c (ONEBYTE_BODY): New macro.
* iconvdata/gbk.c (ONEBYTE_BODY): New macro.
* iconvdata/gb18030.c (ONEBYTE_BODY): New macro.
* iconvdata/ibm932.c: Include <stdbool.h>.
(TRUE, FALSE): Remove macros.
(BODY for FROM_LOOP): Remove unused variable rp1.
(ONEBYTE_BODY): New macro.
(BODY for TO_LOOP): Use bool.
* iconvdata/ibm932.h (__ibm932sb_to_ucs4_idx): Remove array.
* iconvdata/ibm943.c: Include <stdbool.h>.
(TRUE, FALSE): Remove macros.
(BODY for FROM_LOOP): Remove unused variable rp1.
(ONEBYTE_BODY): New macro.
(BODY for TO_LOOP): Use bool.
* iconvdata/ibm943.h (__ibm943sb_to_ucs4_idx): Remove array.
* iconvdata/iso8859-1.c (ONEBYTE_BODY): New macro.
* iconvdata/iso_6937-2.c (ONEBYTE_BODY): New macro.
* iconvdata/iso_6937.c (ONEBYTE_BODY): New macro.
* iconvdata/johab.c (ONEBYTE_BODY): New macro.
* iconvdata/sjis.c (ONEBYTE_BODY): New macro.
* iconvdata/shift_jisx0213.c (ONEBYTE_BODY): New macro.
* iconvdata/t.61.c (ONEBYTE_BODY): New macro.
* iconvdata/uhc.c (ONEBYTE_BODY): New macro.
* iconvdata/gbbig5.c: Tweak comment.
* iconv/gconv_int.h: Include <stdlib.h> for alloca decl.
(norm_add_slashes): alloca -> __alloca
* sysdeps/generic/readelflib.c (process_elf_file): Make LOADADDR
p_vaddr/p_offset when computing LOADADDR.
2002-08-04 Ulrich Drepper <drepper@redhat.com>
* stdio-common/psignal.c: Declare _sys_siglist_internal. Use USEINT
to access _sys_siglist.
* string/strsignal.c: Likewise.
* sysdeps/generic/siglist.c: Add _sys_siglist_internal alias.
* sysdeps/gnu/siglist.c: Likewise.
* sysdeps/unix/siglist.c: Likewise.
* sysdeps/unix/sysv/linux/arm/siglist.c: Likewise.
* libio/fileops.c: Add missing INTUSEs for _IO_file_jumps.
* libio/wfileops.c: Add missing INTUSE for _IO_file_close.
* intl/dcigettext.c: Define _nl_default_dirname_internal as hidden
alias and use it.
* intl/bindtextdom.c: Use _nl_default_dirname_internal.
* include/netinet/in.h: Add declaration of in6addr_loopback_internal.
* inet/in6_addr.c: Add INTVARDEF for in6addr_loopback.
* sysdeps/posix/getaddrinfo.c: Use INTUSE for in6addr_loopback access.
* include/time.h: Add libc_hidden_proto for __gmtime_r.
* time/gmtime.c (__gmtime_r): Add libc_hidden_def.
* iconv/Versions: Replace __gconv_alias_db, __gconv_modules_db,
and __gconv_cache with __gconv_get_alias_db, __gconv_get_modules_db,
and __gconv_get_cache respectively.
* iconv/gconv_cache.c (gconv_cache): Renamed for __gconv_cache and
defined static. Change all users.
(__gconv_get_cache): New function.
* iconv/gconv_db.c (__gconv_get_modules_db): New function.
(__gconv_get_alias_db): New function.
* iconv/gconv_int.h (__gconv_alias_db): Declare as hidden.
(__conv_modules_db): Likewise.
Add prototypes for __gconv_get_cache, __gconv_get_modules_db,
and __gconv_get_alias_db.
* iconv/iconv_prog.c: Use the new functions instead of accessing the
variables.
* include/stdlib.h: Add prototype and libc_hidden_proto for
__default_morecore.
* sysdeps/generic/morecore.c: Include <stdlib.h>.
* malloc/obstack.c: Remove fputs macro.
* malloc/mtrace.c: Remove fopen macro.
* iconv/gconv_cache.c (find_module): Don't allocate room for the
filename. Use alloca, we don't need it beyond this function.
(__gconv_release_cache): New function.
* iconv/gconv_db.c (__gconv_close_transform): Call
__gconv_release_cache after the steps are handled.
* iconv/gconv_dl.c (__gconv_find_shlib): Allocate file name in the
record as well.
* iconv/gconv_int.h: Add prototype fpr __gconv_release_cache.
2001-07-22 Ulrich Drepper <drepper@redhat.com>
* iconv/gconv_builtin.c (struct builtin_map): Remove init and end
elements.
(BUILTIN_TRANSFORMATION): Remove Init and End parameters.
(__gconv_get_builtin_trans): Initialize __init_fct and __end_fct to
NULL.
* iconv/gconv_builtin.h: Remove NULL parameters for Init and End in
all BUILTIN_TRANSFORMATION calls.
* iconv/gconv_conf.c (BUILTIN_TRANSFORMATION): Remove Init and End
parameters.
* iconv/gconv_simple.c: Likewise.
* iconv/gconv_db.c (gen_steps): Internal converters don't have
initializers, move the code accordingly.
* iconv/gconv_conf.c (__gconv_read_conf): Don't read configuration
file if STATIC_GCONV is defined.
* iconv/gconv_conf.c (__gconv_path_envvar): New global variable.
(__gconv_get_path): Use it instead of call getenv.
(__gconv_read_conf): First see whether cache can be used. If yes,
don't do any work here.
* iconv/gconv_db.c (__gconv_release_step): Renamed from release_step
and exported. Change callers.
(__gconv_find_transform): First call __gconv_lookup_cache and only
continue if it signals no cache available. Remove some unnecessary
tests.
* iconv/gconv_int.h: Declare __gconv_path_envvar, __gconv_lookup_cache,
__gconv_release_step, and __gconv_loaded_cache.
* iconv/gconv_cache.c: New file.
* iconv/iconvconfig.c: New file.
* iconv/iconvconfig.h: New file.
* iconv/strtab.c: New file.
* iconv/Makefile: Add rules to build new files and programs.
2001-07-06 Paul Eggert <eggert@twinsun.com>
* manual/argp.texi: Remove ignored LGPL copyright notice; it's
not appropriate for documentation anyway.
* manual/libc-texinfo.sh: "Library General Public License" ->
"Lesser General Public License".
2001-07-06 Andreas Jaeger <aj@suse.de>
* All files under GPL/LGPL version 2: Place under LGPL version
2.1.
* iconv/gconv_int.h (strict gconv_module): Remove all members
associated with regular expressions. Use a simple string as the
from name.
* iconv/gconv_db.c: Remove code handling regular expressions.
* iconv/gconv_conf.c: Likewise.
* iconv/iconv_prog.c: Likewise.
* iconv/gconv_builtin.h: Adjust for change in gconv_conf.c.
2000-06-19 Ulrich Drepper <drepper@redhat.com>
* iconv/gconv.h (__gconv_trans_fct): Add new parameter.
General namespace cleanup.
(struct __gconv_trans_data): Add next field.
(struct __gconv_step_data): Make __trans a pointer.
* iconv/gconv_conf.c: Split out code to find gconv directories from
__gconv_read_conf in new functions.
* iconv/gconv_int.h: Define new data structure and declare new
functions for handling of gconv directory list.
* iconv/gconv_open.c: Allow more than one error handling step being
used. Call function to load error handling module if it is none
of the builtin transformations.
* iconv/gconv_close.c: Add code to free transliteration data.
* iconv/gconv_trans.c: Add functions to load and unload modules
implementing transliteration etc.
* iconv/skeleton.c: Call all context functions now that more than
one module is allowed.
* iconv/loop.c (STANDARD_ERR_HANDLING): New macro.
* iconv/gconv_simple.c: Use STANDARD_ERR_HANDLING macro for places
where the full error handling using transliteration is needed.
* iconvdata/8bit-gap.c: Likewise.
* iconvdata/8bit-generic.c: Likewise.
* iconvdata/ansi_x3.110.c: Likewise.
* iconvdata/big5.c: Likewise.
* iconvdata/big5hkscs.c: Likewise.
* iconvdata/euc-cn.c: Likewise.
* iconvdata/euc-jp.c: Likewise.
* iconvdata/euc-kr.c: Likewise.
* iconvdata/euc-tw.c: Likewise.
* iconvdata/gbgbk.c: Likewise.
* iconvdata/gbk.c: Likewise.
* iconvdata/iso-2022-cn.c: Likewise.
* iconvdata/iso-2022-jp.c: Likewise.
* iconvdata/iso-2022-kr.c: Likewise.
* iconvdata/iso646.c: Likewise.
* iconvdata/iso8859-1.c: Likewise.
* iconvdata/iso_6937-2.c: Likewise.
* iconvdata/iso_6937.c: Likewise.
* iconvdata/johab.c: Likewise.
* iconvdata/sjis.c: Likewise.
* iconvdata/t.61.c: Likewise.
* iconvdata/uhc.c: Likewise.
* iconvdata/unicode.c: Likewise.
* iconvdata/utf-16.c: Likewise.
* libio/iofwide.c: Reset __trans member of __gconv_trans_data
structure correctly after last change.
* wcsmbs/btowc.c: Likewise.
* wcsmbs/mbrtowc.c: Likewise.
* wcsmbs/mbsnrtowcs.c: Likewise.
* wcsmbs/mbsrtowcs.c: Likewise.
* wcsmbs/wcrtomb.c: Likewise.
* wcsmbs/wcsnrtombs.c: Likewise.
* wcsmbs/wcsrtombs.c: Likewise.
* wcsmbs/wctob.c: Likewise.
* localedata/Makefile: Set -Wno-format for some files since gcc does
not know all the format specifiers.
2000-06-18 Ulrich Drepper <drepper@redhat.com>
* locale/loadlocale.c (_nl_unload_locale): Remove a bit of
unneeded code.
* locale/lc-time.c (_nl_init_era_entries): Likewise.
2000-06-16 Ulrich Drepper <drepper@redhat.com>
* iconv/gconv_int.h (norm_add_slashes): Optionally add given suffix.
* iconv/gconv_open.c: Remove error handling specification from `from'
character set name.
* intl/loadmsgcat.c (_nl_load_domain): Call norm_add_slashes with
new parameter to always enable transliteration.
* locale/localeinfo.h (LIMAGIC): Bump number because of incompatible
change.
(struct locale_data): Add new members use_translit and options.
* locale/findlocale.c (_nl_find_locale): Set use_translit flag is
character set name contained modifier TRANSLIT.
* locale/loadlocale.c (_nl_load_locale): Initialize new use_translit
and options fields.
(_nl_unload_locale): Free options string if necessary.
* wcsmbs/wcsmbsload.c (__wcsmbs_load_conv): Enable translation if
the locale names suggested this.
* locale/C-address.c: Add two new initialilzers to adjust data
structure for new format.
* locale/C-collate.c: Likewise.
* locale/C-ctype.c: Likewise.
* locale/C-identification.c: Likewise.
* locale/C-measurement.c: Likewise.
* locale/C-messages.c: Likewise.
* locale/C-monetary.c: Likewise.
* locale/C-name.c: Likewise.
* locale/C-numeric.c: Likewise.
* locale/C-paper.c: Likewise.
* locale/C-telephone.c: Likewise.
* locale/C-time.c: Likewise.
* locale/setlocale.c: Add some more __builtin_expect.
