I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
During testing of localedef running in a minimal container
there were several error cases which were hard to diagnose
since they appeared as strerror (errno) values printed by the
higher level functions. This change adds three new verbose
messages for potential failure paths. The new messages give
the user the opportunity to use -v and display additional
information about why localedef might be failing. I found
these messages useful myself while writing a localedef
container test for --no-hard-links.
Since the changes cleanup the code that handle codeset
normalization we add tst-localedef-path-norm which contains
many sub-tests to verify the correct expected normalization of
codeset strings both when installing to default paths (the
only time normalization is enabled) and installing to absolute
paths. During the refactoring I created at least one
buffer-overflow which valgrind caught, but these tests did not
catch because the exec in the container had a very clean heap
with zero-initialized memory. However, between valgrind and
the tests the results are clean.
The new tst-localedef-path-norm passes without regression on
x86_64.
Change-Id: I28b9f680711ff00252a2cb15625b774cc58ecb9d
Downstream distributions need consistent sets of hardlinks in
order for rpm to operate effectively. This means that even if
locales are built with a high level of parallelism that the
resulting files need to have consistent hardlink counts. The only
way to achieve this is with a post-install hardlink pass using a
program like 'hardlink' (shipped in Fedora).
If the downstream distro wants to post-process the hardlinks then
the time spent in localedef looking up sibling directories and
processing hardlinks is wasted effort.
To optimize the build and install pass we add a --no-hard-links
option to localedef to avoid doing the hardlink optimziation for
size.
Tested on x86_64 with 'make localedata/install-locale-files'
before and after. Without the patch we have files with 100+
hardlink counts. After the patch and running with --no-hard-links
all link counts are 1. This patch also alters the convenience
target 'make localedata/install-locale-files' to use the new
option.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
From localedef --help:
Output control:
...
--no-warnings=<warnings> Comma-separated list of warnings to disable;
supported warnings are: ascii, intcurrsym
...
--warnings=<warnings> Comma-separated list of warnings to enable;
supported warnings are: ascii, intcurrsym
Locales using SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII
compatible. In order to build locales using these character maps, and
have localedef exit with a status of 0, we add new option to localedef
to disable or enable specific warnings. The options are --no-warnings
and --warnings, to disable and enable specific warnings respectively.
The options take a comma-separated list of warning names. The warning
names are taken directly from the generated warning. When a warning
that can be disabled is issued it will print something like this: foo is
not defined [--no-warnings=foo]
For the initial implementation we add two controllable warnings; first
'ascii' which is used by the localedata installation makefile target to
install SHIFT_JIS and SHIFT_JISX0213-using locales without error; second
'intcurrsym' which allows a program to use a non-standard international
currency symbol without triggering a warning. The 'intcurrsym' is
useful in the future if country codes are added that are not in our
current ISO 4217 list, and the user wants to avoid the warning. Having
at least two warnings to control gives an example for how the changes
can be extended to more warnings if required in the future.
These changes allow ja_JP.SHIFT_JIS and ja_JP.SHIFT_JISX0213 to be
compiled without warnings using --no-warnings=ascii. The
localedata/Makefile $(INSTALL-SUPPORTED-LOCALES) target is adjusted to
automatically add `--no-warnings=ascii` for such charmaps, and likewise
localedata/gen-locale.sh is adjusted with similar logic.
v2: Bring verbose, be_quiet, and all warning control booleans into
record-status.c, and compile this object file to be used by locale,
iconv, and localedef. Any users include record-status.h.
v3: Fix an instance of boolean coercion in set_warning().
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
The error and warning handling in localedef, locale, and iconv
is a bit of a mess.
We use ugly constructs like this:
WITH_CUR_LOCALE (error (1, errno, gettext ("\
cannot read character map directory `%s'"), directory));
to issue errors, and read error_message_count directly from the
error API to detect errors. The problem with that is that the
code also uses error to print warnings, and informative messages.
All of this leads to problems where just having warnings will
produce an exit status as-if errors had been seen.
To fix this situation I have adopted the following high-level
changes:
* All errors are counted distinctly.
* All warnings are counted distinctly.
* All informative messages are not counted.
* Increasing verbosity cannot generate *more* errors, and
it previously did for errors conditional on verbose,
this is now fixed.
* Increasing verbosity *can* generate *more* warnings.
* Making the output quiet cannot generate *fewer* errors,
and it previously did for errors conditional on be_quiet,
this is now fixed.
* Each of error, warning, and informative message has it's
own function to call defined in record-status.h, and they
are: record_error, record_warning, and record_verbose.
* The record_error function always records an error, but
conditional on be_quiet may not print it.
* The record_warning function always records a warning,
but conditional on be_quiet may not print it.
* The record_verbose function only prints the verbose
message if verbose is true and be_quiet is false.
This has allowed the following fix:
* Previously any warnings were being treated as errors
because they incremented error_message_count, but now
we properly return an exit status of 1 if there are
warnings but output was generated.
All of this allows localedef to correctly decide if errors,
or warnings were present, and produce the correct exit code.
The locale and iconv programs now also use record-status.h
and we have removed the WITH_CUR_LOCALE hack, and instead
have internal push_locale/pop_locale functions centralized
in the record routines.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
The localedef --list-archive option claims that it can
accept a [file] argument and list the contents of that
archive. The support was never implemented. This patch
adds that support and allows --list-archive to work as
expected. You can now use localedef to list the contents
of arbitrary locale archives by using:
./localedef --list-archive file
These prototypes are duplicated in many places. Add a dedicated
header for holding prototypes for program-specific functions to
avoid that.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
name to find codeset name (if any) and normalize it. If the
normalized name differs, look up only that in the archive.
* locale/programs/locarchive.c (add_locale_to_archive): If the name
contains a codeset, normalize the codeset store only the normalized
name in the archive. If not, add an alias containing the locale's
normalized codeset name. Apply codeset name normalization when
matching entries in the alias file.
* locale/programs/locarchive.c (delete_locales_from_archive): Don't
decrement HEAD->namehash_used here.
(add_locale): Only need to insert name string when name_offset != 0.
* locale/programs/localedef.c (options): Add -A/--alias-file.
(alias_file): New variable.
(parse_opt): Grok -A, set that.
* locale/programs/localedef.h: Declare it.
* locale/programs/locarchive.c (insert_name): New function, broken out
of ...
(add_locale_to_archive): ... here. Call that.
(add_alias): New function.
(add_locale): New static function, add_locale_to_archive renamed.
(add_locale_to_archive): Call that and use add_alias to add an alias
for the name with codeset if the given name lacks it.
(enlarge_archive): Call add_locale instead of add_locale_to_archive.
* locale/Makefile (routines): Add loadarchive.
* locale/loadarchive.c: New file, started from code by Ulrich Drepper.
(_nl_load_locale_from_archive): New function.
* locale/localeinfo.h: Declare it.
* locale/findlocale.c (_nl_find_locale): If using default locale path,
try _nl_load_locale_from_archive first.
* locale/loadlocale.c (_nl_intern_locale_data): New function,
broken out of _nl_load_locale.
(_nl_load_locale): Call that.
* locale/localeinfo.h: Declare it.
(struct locale_data): Replace member `mmaped' with `alloc', an enum.
(struct locale_data): Remove unused member `options'.
* locale/findlocale.c (_nl_remove_locale): Update uses.
* locale/loadlocale.c (_nl_load_locale, _nl_unload_locale): Likewise.
* locale/C-collate.c: Update initializer.
* locale/C-identification.c: Likewise.
* locale/C-measurement.c: Likewise.
* locale/C-telephone.c: Likewise.
* locale/C-address.c: Likewise.
* locale/C-name.c: Likewise.
* locale/C-paper.c: Likewise.
* locale/C-time.c: Likewise.
* locale/C-numeric.c: Likewise.
* locale/C-monetary.c: Likewise.
* locale/C-messages.c : Likewise.
* locale/C-ctype.c: Likewise.
* locale/hashval.h [! LONGBITS]: Include <limits.h> here and
use CHAR_BIT instead of BITSPERBYTE.
* locale/localeinfo.h (_nl_find_locale, _nl_load_locale,
_nl_unload_locale): Add `internal_function attribute_hidden' to decls.
* locale/findlocale.c (_nl_find_locale): Add internal_function to defn.
(_nl_remove_locale): Likewise.
* locale/loadlocale.c (_nl_load_locale, _nl_unload_locale): Likewise.
* locale/findlocale.c (_nl_default_locale_path): New variable.
(_nl_find_locale): If LOCALE_PATH is null, default to that.
* locale/localeinfo.h: Declare it.
* locale/setlocale.c (setlocale): Use _nl_default_locale_path
in place of LOCALEDIR. If no LOCPATH, pass null to _nl_find_locale.
* locale/newlocale.c (__newlocale): Likewise.
* misc/err.c (vwarnx, vwarn): Fix typos in libc_hidden_def uses.
* inet/rexec.c (rexec_af): Add libc_hidden_def.
* sysdeps/generic/morecore.c: Likewise.
* signal/allocrtsig.c (__libc_current_sigrtmin): Likewise.
(__libc_current_sigrtmax): Likewise.
2002-08-08 Roland McGrath <roland@redhat.com>
* locale/loadlocale.c (_nl_load_locale): Don't use MAP_INHERIT.
* catgets/open_catalog.c (__open_catalog): Likewise.
* locale/programs/locarchive.c (INITIAL_NUM_NAMES): Renamed
from typo INITIAL_NUM_NANES.
(create_archive): Update use.
2002-04-18 Ulrich Drepper <drepper@redhat.com>
* locale/programs/locfile.c (to_archive): New variable. To collect
data which has to be added to the locale archive.
(write_all_categories): Take new third parameter with locale name.
Unless no_archive flag set add new locale data to the archive
(write_locale_data): Unless no-archive flag set store generated data
in to_archive data structure instead of generation output file.
Add new parameter with locale category index.
* locale/programs/locfile.h: Add new parameters in declaractions of
write_all_categories and write_locale_data.
* locale/programs/localedef.c: Recognize --no-archive, --list-archive,
--add-to-archive, and --delete-from-archive options. Pass extra
parameter to write_all_categories.
* locale/programs/localedef.h: Add prototypes for functions in
locarchive.c.
* locale/locarchive.h: New file.
* locale/programs/locarchive.c: New file.
* locale/Makefile (distribute): Add programs/locarchive.c and
locarchive.h.
(localedef-modules): Add md5 and locarchive.
Add vpath to crypt subdir for md5.c.
* locale/programs/ld-address.c: Pass locale category ID as new second
parameter to write_locale_data.
* locale/programs/ld-collate.c: Likewise.
* locale/programs/ld-ctype.c: Likewise.
* locale/programs/ld-identification.c: Likewise.
* locale/programs/ld-measurement.c: Likewise.
* locale/programs/ld-messages.c: Likewise.
* locale/programs/ld-monetary.c: Likewise.
* locale/programs/ld-name.c: Likewise.
* locale/programs/ld-numeric.c: Likewise.
* locale/programs/ld-paper.c: Likewise.
* locale/programs/ld-telephone.c: Likewise.
* locale/programs/ld-time.c: Likewise.
* locale/simple-hash.c: Move compute_hashval function from here...
* locale/hashval.h: ...to here. New file.
* locale/simple-hash.h: Add prototype for compute_hashval.
* include/libintl.h: Minor cleanups.
* elf/reldep7.c: New file.
* elf/reldep7mod1.c: New file.
* elf/reldep7mod2.c: New file.
* elf/Makefile: Add rules to build and run reldep7.
2001-07-06 Paul Eggert <eggert@twinsun.com>
* manual/argp.texi: Remove ignored LGPL copyright notice; it's
not appropriate for documentation anyway.
* manual/libc-texinfo.sh: "Library General Public License" ->
"Lesser General Public License".
2001-07-06 Andreas Jaeger <aj@suse.de>
* All files under GPL/LGPL version 2: Place under LGPL version
2.1.
2000-07-23 Bruno Haible <haible@clisp.cons.org>
* wctype/wchar-lookup.h: New file.
* wctype/iswctype.c: Include "wchar-lookup.h".
(__iswctype): Support alternate locale format with 3-level tables.
* wctype/iswctype_l.c (__iswctype_l): Likewise.
* wctype/towctrans.c (__towctrans): Likewise.
* wctype/towctrans_l.c (__towctrans_l): Likewise.
* wctype/wcfuncs.c: Include "wchar-lookup.h".
(__ctype32_wctype, __ctype32_wctrans): Declare external.
(__iswalnum, __iswalpha, __iswcntrl, __iswdigit, __iswlower,
__iswgraph, __iswprint, __iswpunct, __iswspace, __iswupper,
__iswxdigit, towlower, towupper): Support alternate locale format
with 3-level tables.
* wctype/wcextra.c (iswblank): Likewise.
* wctype/wcfuncs_l.c: Include "wchar-lookup.h".
(__iswalnum_l, __iswalpha_l, __iswcntrl_l, __iswdigit_l, __iswlower_l,
__iswgraph_l, __iswprint_l, __iswpunct_l, __iswspace_l, __iswupper_l,
__iswxdigit_l, __towlower_l, __towupper_l): Support alternate locale
format with 3-level tables.
* wctype/wcextra_l.c (__iswblank_l): Likewise.
* wctype/wctype.c (__wctype): Likewise. In the alternate locale
format, return a 3-level table pointer.
* wctype/wctype_l.c (__wctype_l): Likewise.
* wctype/wctrans.c (wctrans): Likewise.
* wctype/wctype.h (__ISwupper, __ISwlower, __ISwalpha, __ISwdigit,
__ISwxdigit, __ISwspace, __ISwprint, __ISwgraph, __ISwblank,
__ISwcntrl, __ISwpunct, __ISwalnum): New enum values.
(iswctype): Remove macro definition.
* wcsmbs/wcwidth.h: Include "wchar-lookup.h".
(internal_wcwidth): Support alternate locale format with 3-level
tables.
* locale/langinfo.h (_NL_CTYPE_CLASS_OFFSET, _NL_CTYPE_MAP_OFFSET):
New nl_items.
* locale/categories.def (_NL_CTYPE_CLASS_OFFSET, _NL_CTYPE_MAP_OFFSET):
Define them as being type "word".
* locale/C-ctype.c (_nl_C_LC_CTYPE): Add initializers for them.
* ctype/ctype-info.c (__ctype32_wctype, __ctype32_wctrans,
__ctype32_width): New exported variables.
* locale/lc-ctype.c (_nl_postload_ctype): Initialize them in the
alternate locale format. Don't initialize __ctype_names and
__ctype_width in the alternate locale format.
* locale/programs/localedef.h (oldstyle_tables): New declaration.
* locale/programs/localedef.c (oldstyle_tables): New variable.
(OPT_OLDSTYLE): New macro.
(options): Add --old-style option.
(parse_opt): Handle --old-style option.
* locale/programs/ld-ctype.c (locale_ctype_t): Add class_offset,
map_offset, class_3level, map_3level, width_3level members.
(ctype_output): Support for alternate locale format: Computation of
nelems changes. _NL_CTYPE_TOUPPER32, _NL_CTYPE_TOLOWER32 and
_NL_CTYPE_CLASS32 only 256 characters. _NL_CTYPE_NAMES empty.
New fields _NL_CTYPE_CLASS_OFFSET, _NL_CTYPE_MAP_OFFSET. Field
_NL_CTYPE_WIDTH now contains the three-level table. Extra elems
now contain both class and map tables.
(struct wctype_table): New type.
(wctype_table_init, wctype_table_add, wctype_table_finalize): New
functions.
(struct wcwidth_table): New type.
(wcwidth_table_init, wcwidth_table_add, wcwidth_table_finalize): New
functions.
(struct wctrans_table): New type.
(wctrans_table_init, wctrans_table_add, wctrans_table_finalize): New
functions.
(allocate_arrays): Support for alternate locale format: Set
plane_size and plane_cnt to 0. Restrict ctype->ctype32_b to the first
256 characters. Compute ctype->class_3level. Restrict ctype->map32[idx]
to the first 256 characters. Compute ctype->map_3level. Set
ctype->class_offset and ctype->map_offset. Compute ctype->width_3level
instead of ctype->width.
* locale/programs/ld-collate.c (collate_output): Don't start with empty
extrapool and indirectpool obstacks since we need the offsets to be
nonzero.
(collate_read): Call load_locale, not find_locale.
* locale/programs/ld-ctype.c (ctype_finish): If LC_CTYPE category
wasn't defined in the file also initialize repertoire if possible.
* locale/programs/ld-time.c (time_finish): Fix message string.
* locale/programs/linereader.c: Cast parameters of lr_error to
correct type to prevnet warning.
* locale/programs/localedef.c (load_locale): New file.
* locale/programs/localedef.h: Add its prototype.
* locale/programs/repertoire.c (repertoire_new_char): Add missing
parameters to lr_error call.
* localedata/Makefile: Enable running tests again.
* localedata/tests/test2.def: Adjust syntax to new specification.
* localedata/tests/test3.def: Likewise.
* localedata/tst-trans.sh: Redirect output of program into file.
* string/strcoll.c: Fix many error in new implementation to make it
pass (at least) the test suite.
* locale/Makefile: Don't link localedef statically anymore.
* locale/ld-collate.c (struct element_t): Add field is_character and
use it to distinguish real character from collating elements and
symbols.
* locale/programs/ld-time.c: Likewise.
1999-09-12 Ulrich Drepper <drepper@cygnus.com>
* locale/programs/ld-address.c: Fix handling of non-existing
definitions for this category. Correctly ignore content of this
category is this is necessary.
* locale/programs/ld-collate.c: Likewise.
* locale/programs/ld-ctype.c: Likewise.
* locale/programs/ld-identification.c: Likewise.
* locale/programs/ld-measurement.c: Likewise.
* locale/programs/ld-messages.c: Likewise.
* locale/programs/ld-monetary.c: Likewise.
* locale/programs/ld-name.c: Likewise.
* locale/programs/ld-numeric.c: Likewise.
* locale/programs/ld-paper.c: Likewise.
* locale/programs/ld-telephone.c: Likewise.
* locale/programs/ld-time.c: Likewise.
* locale/programs/locfile.h (handle_copy): Take extra parameter
with result pointer. Fill in name of locale from which to copy.
Correctly read token after `END'.
* locale/programs/localedef.c (main): Correct handling copy.
(add_to_readlist): Take extra parameter which says whether we
are supposed to read the file or not.
(find_locale): Call add_to_readlist with extra parameter set to 0.
* locale/programs/localedef.h (struct localedef_t): Use __LC_LAST
instead of wrong number in array definitions.
(add_to_readlist): Update prototype.
* locale/programs/locfile.c (locfile_read): Mark categories not
available in source file as read.
(write_all_categories): Fix typo.
