The array lr->buf contains characters, which can be signed. A 0xff
byte in the input could be incorrectly reported as EOF. More
importantly, get_string in linereader.c converts a signed input byte
to a Unicode code point using ADDWC ((uint32_t) ch), under the
assumption that this decodes the ISO-8859-1 input encoding. If char
is signed, this does not give the correct result. This means that
ISO-8859-1 input files for localedef are not actually supported,
contrary to the comment in get_string. This is a happy accident because
we can therefore change the file encoding to UTF-8 without impacting
backwards compatibility.
While at it, remove the \32 check for MS-DOS end-of-file character (^Z).
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
This propagates the recent http->https URL changes.
Since I used gperf 3.1 to regenerate, this is also a minor
internal-to-localedef API change.
URL problem reported by Joseph Myers in:
https://www.sourceware.org/ml/libc-alpha/2019-09/msg00143.html
* locale/programs/charmap-kw.h, locale/programs/locfile-kw.h:
Regenerate with gperf 3.1.
* locale/programs/linereader.h (kw_hash_fct_t):
* locale/programs/repertoire.c (repertoiremap_hash):
2nd arg is now size_t not unsigned, for compatibility with gperf 3.1.
The error and warning handling in localedef, locale, and iconv
is a bit of a mess.
We use ugly constructs like this:
WITH_CUR_LOCALE (error (1, errno, gettext ("\
cannot read character map directory `%s'"), directory));
to issue errors, and read error_message_count directly from the
error API to detect errors. The problem with that is that the
code also uses error to print warnings, and informative messages.
All of this leads to problems where just having warnings will
produce an exit status as-if errors had been seen.
To fix this situation I have adopted the following high-level
changes:
* All errors are counted distinctly.
* All warnings are counted distinctly.
* All informative messages are not counted.
* Increasing verbosity cannot generate *more* errors, and
it previously did for errors conditional on verbose,
this is now fixed.
* Increasing verbosity *can* generate *more* warnings.
* Making the output quiet cannot generate *fewer* errors,
and it previously did for errors conditional on be_quiet,
this is now fixed.
* Each of error, warning, and informative message has it's
own function to call defined in record-status.h, and they
are: record_error, record_warning, and record_verbose.
* The record_error function always records an error, but
conditional on be_quiet may not print it.
* The record_warning function always records a warning,
but conditional on be_quiet may not print it.
* The record_verbose function only prints the verbose
message if verbose is true and be_quiet is false.
This has allowed the following fix:
* Previously any warnings were being treated as errors
because they incremented error_message_count, but now
we properly return an exit status of 1 if there are
warnings but output was generated.
All of this allows localedef to correctly decide if errors,
or warnings were present, and produce the correct exit code.
The locale and iconv programs now also use record-status.h
and we have removed the WITH_CUR_LOCALE hack, and instead
have internal push_locale/pop_locale functions centralized
in the record routines.
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
2001-07-06 Paul Eggert <eggert@twinsun.com>
* manual/argp.texi: Remove ignored LGPL copyright notice; it's
not appropriate for documentation anyway.
* manual/libc-texinfo.sh: "Library General Public License" ->
"Lesser General Public License".
2001-07-06 Andreas Jaeger <aj@suse.de>
* All files under GPL/LGPL version 2: Place under LGPL version
2.1.
2001-02-09 Ulrich Drepper <drepper@redhat.com>
* locale/programs/linereader.c (get_ident): Stop loop if EOF. Use
lr_ungetc to push back last read character.
* locale/programs/linereader.h (lr_ungetc): Don't push back is
character is EOF.
(lr_ignore_rest): Don't warn about garbage if it is really the end
of the file.
* manual/Makefile: Use ifnottext and not ifinfo to protect Top node
definition.
2001-02-04 Ulrich Drepper <drepper@redhat.com>
* iconv/Makefile (iconv_prog-modules): Define. Add vpath to find
files in locale/programs. Add CFLAGS definition to allow compiling
localedef files.
* iconv/dummy-repertoire.c: New file.
* iconv/iconv_charmap.c: New file.
* iconv/iconv_prog.h: New file.
* iconv/iconv_prog.c: Make verbose and omit_invalid global.
(main): If parameter for -f and -t contain slashes try first to resolve
the strings as filenames of charmap files. Use them for conversion
in this case.
* iconvdata/run-iconv-test.sh: If charmaps exist also run tests with
iconv getting charmap names as parameters.
* locale/programs/linereader.c (lr_token): Take extra parameters
verbose and pass it to get_string.
(get_string): Take extra parameters verbose.
* locale/programs/charmap.c (parse_charmap): Take extra parameters
verbose and be_quiet. Change all callers of lr_token and
parse_charmap.
* locale/programs/charmap.h: Likewise.
* locale/programs/ld-address.c: Likewise.
* locale/programs/ld-collate.c: Likewise.
* locale/programs/ld-ctype.c: Likewise.
* locale/programs/ld-identification.c: Likewise.
* locale/programs/ld-measurement.c: Likewise.
* locale/programs/ld-messages.c: Likewise.
* locale/programs/ld-monetary.c: Likewise.
* locale/programs/ld-name.c: Likewise.
* locale/programs/ld-numeric.c: Likewise.
* locale/programs/ld-paper.c: Likewise.
* locale/programs/ld-telephone.c: Likewise.
* locale/programs/ld-time.c: Likewise.
* locale/programs/linereader.c: Likewise.
* locale/programs/linereader.h: Likewise.
* locale/programs/localedef.c: Likewise.
* locale/programs/locfile.c: Likewise.
* locale/programs/locfile.h: Likewise.
* locale/programs/repertoire.c: Likewise.
Thu Oct 31 00:01:39 1996 Ulrich Drepper <drepper@cygnus.com>
* signal/Makefile (routines): Add sigwait.
* signal/signal.h: Add prototype for sigwait.
* sysdeps/posix/sigwait.c: New file. Implementation of sigwait
function from POSIX.1c.
* sysdeps/stub/sigwait.c: New file. Stub version of sigwait.
Wed Oct 30 02:01:17 1996 Richard Henderson <rth@tamu.edu>
* sunrpc/xdr_float.c (xdr_float): Handle sizeof(float)!=sizeof(long),
but don't bother going farther than sizeof(float)==sizeof(int).
(xdr_double): Handle little-endian machines! Handle sizeof(double)
!= 2*sizeof(long), though again don't bother with more than int.
Thu Oct 29 16:09:42 1996 Craig Metz <cmetz@inner.net>
* sysdeps/posix/getaddrinfo.c: Use buffer limits for inet_ntop
function.
Tue Oct 29 12:37:22 1996 Ulrich Drepper <drepper@cygnus.com>
* Makerules: Create symbolic links for linking in $(libdir).
(make-link): Use absolute path for destination if this is not in
the same directory.
* elf/rtld.c (dl_main): When verifying don't check the name of
the dynamic linker.
* shlib-versions: Change entries for Hurd specific libs from
*-*-gnu* to *-*-gnu?* so that i586-pc-linux-gnu does not match
these entries.
* assert/assert.h: Reformat copyright.
Change reference to ANSI into reference to ISO C.
* ctype/ctype.h: Likewise.
* errno.h: Likewise.
* limits.h: Likewise.
* math/math.h: Likewise.
* setjmp/setjmp.h: Likewise.
* stdio/stdio.h: Likewise.
* libio/stdio.h: Likewise.
* stdlib/stdlib.h: Likewise.
* string/string.h: Likewise.
* time/time.h: Likewise.
* string/argz.h: Use __const is definitions.
* elf/dlfcn.h: Use __const and __P. Reformat copyright.
* misc/err.h: Likewise.
* wctype/wctype.h (wctrans_t): Use __const instead of const.
* Makeconfig ($(common-objpfx)soversions.mk): Generate list of
sonames for versioned libraries.
* Makefile: Remove code to generate libc-version.h.
Generate gnu/lib-names.h with info from soversions.mk.
* features.h: Define __GLIBC__ and __GLIBC_MINOR__.
* dirent/tst-seekdir.c: Initialize save3.
* grp/testgrp.c: Initialize my_group.
* grp/fgetgrent_r.c: Change interface to follow POSIX.1c.
* grp/grp.h: Likewise.
* nss/getXXbyYY.c: Likewise.
* nss/getXXbyYY_r.c: Likewise.
* nss/getXXent.c: Likewise.
* nss/getXXent_r.c: Likewise.
* pwd/fgetpwent_r.c: Likewise.
* pwd/pwd.h: Likewise.
* shadow/fgetspent_r.c: Likewise.
* shadow/sgetspent.c: Likewise.
* shadow/sgetspent_r.c: Likewise.
* grp/fgetgrent.c: Adapt for change in interface of fgetgrent_r.
* pwd/fgetpwent.c: Likewise, for fgetpwent_r.c.
* shadow/fgetspent.c: Likewise, for fgetpwent_r.c.
* resolv/netdb.h: Adapt prototypes for reentrant functions to
follow POSIX.1c.
* sunrpc/rpc/netdb.h: Likewise,
* shadow/shadow.h: Likewise.
* inet/getnetgrent_r.c: Follow change in pwd/grp function interface.
* sysdeps/unix/getlogin_r.c: Return ERANGE when buffer is too small.
* inet/herrno.c: Don't define __h_errno. Only h_errno otherwise the
ELF aliasing creates strange situations.
* sysdeps/unix/sysv/linux/errnos.H: Define __set_errno as inline
function.
* sysdeps/unix/sysv/linux/i386/sysdep.S: Don't define __errno.
* sysdeps/unix/sysv/linux/m68k/sysdep.S: Likewise.
* libio/libio.h: Don't declare _IO_flockfile and _IO_funlockfile
weak.
* locale/programs/charmap.c: Add casts to prevent warnings.
* locale/programs/linereader.h: Likewise.
* locale/programs/ld-collate.c: Likewise.
* locale/programs/stringtrans.c: Likewise.
Change types for various variables to prevent warnings.
* locale/programs/ld-ctype.c: Likewise.
* locale/programs/linereader.h (lr_ungetc): Likewise.
* locale/programs/charset.h (struct charset): Use `unsigned int'
as type for width_default.
* posix/regex.c: Change type of `this_reg' variables.
* stdio-common/Makefile: Use -Wno-format for tstdiomisc.c.
* stdio-common/bug5.c: De-ANSI-fy. Use correct types for
variables.
* stdio-common/printf_fp.c: Initialize to_shift.
* stdio-common/test_rdwr.c: Add cast.
* stdio-common/vfprintf.c: Add casts and use correct types to
prevent warnings.
* stdio-common/vfscanf.c: Initialize str and strptr.
* sysdeps/libm-ieee754/e_jnf.c: Use correct types to prevent warnings.
* sysdeps/libm-ieee754/e_pow.c: Likewise.
* sysdeps/libm-ieee754/e_powf.c: Likewise.
* sysdeps/libm-ieee754/e_rem_pio2f.c: Likewise.
* time/test-tz.c: Likewise.
* manual/creature.texi: Document _REENTRANT and _THREAD_SAFE.
* manual/libc.texinfo: Prevent makeinfo failure by avoiding
libc.cp index. This must be fixed.
* manual/nss.texi: Adapt for correct POSIX.1c interface of
reentrant functions.
* manual/users.texi: Document netgroup functions.
* po/es.po: Updated.
* po/fr.po: Updated.
* posix/fnmatch.c: Change to match libit version.
* posix/unistd.h: Change prototype for ttyname_r to match POSIX.1c.
* sysdep/posix/ttyname_r.c: Likewise.
* stdlib/atexit.h (__new_exitfn): Add internal locking.
* stdlib/exit.c: De-ANSI-fy. Handle new ef_us value for flavor.
* stdlib/exit.h: De-ANSI-fy. Define new ef_us value for flavor.
* stdlib/random.c (__srandom): Add internal locking.
(__initstate): Likewise.
(__setstate): Likewise.
(__random): Likewise.
Mon Oct 28 22:28:37 1996 NIIBE Yutaka <gniibe@mri.co.jp>
* sysdeps/generic/crypt-entry.c (crypt_r): Use __set_errno.
(crypt): Likewise.
* resolv/gethnamaddr.c (gethostbyname2): Likewise.
* sysdeps/generic/uname.c: Likewise.
* sysdeps/posix/rename.c: Likewise.
* sysdeps/stub/setrlimit.c: Likewise.
* nss/nss_db/db-netgrp.c (_nss_db_setnetgrent): Fix typo.
Sun Oct 27 11:12:50 1996 Andreas Schwab <schwab@issan.informatik.uni-dortmund.de>
* locale/programs/ld-collate.c (collate_order_elem): Fix format
string.
(collate_element_to): Cast field width argument to `int' for
format string.
(collate_symbol): Likewise.
(collate_order_elem): Likewise.
(collate_weight_bsymbol): Likewise.
(collate_simple_weight): Likewise.
* locale/programs/ld-time.c (STRARR_ELEM): Fix format string.
* locale/programs/ld-ctype.c (ctype_class_newP): Add missing
argument for format string.
(ctype_map_newP): Likewise.
(set_class_defaults): Fix format string.
* locale/programs/localedef.c (construct_output_path): Putting an
explicit \0 into the format string does not work, use %c.
Sat Oct 26 20:38:36 1996 Richard Henderson <rth@tamu.edu>
* Makerules: Install all shared libraries in $(slibdir).
* login/Makefile: Build libutil.so in others pass after
libc.so is created.
* misc/mntent.h: Include <paths.h> for _PATH_MNTTAB & _PATH_MOUNTED.
* string/stratcliff.c: Allocate 3 pages instead of one, then use
mprotect so that we know that the adjacent pages are inaccessible.
* resource/sys/resource.h: Move all structures and enums to ...
* sysdeps/generic/resourcebits.h: ... here ...
* sysdeps/unix/bsd/sun/sunos4/resourcebits.h: ... and here.
* sysdeps/unix/sysv/linux/alpha/resourcebits.h: Remove.
* sysdeps/unix/sysv/linux/i386/resourcebits.h: Remove.
* sysdeps/unix/sysv/linux/m68k/resourcebits.h: Remove.
* sysdeps/unix/sysv/linux/mips/resourcebits.h: Remove.
* sysdeps/unix/sysv/linux/resourcebits.h: New file. Use kernel
header for RLIMIT_* definitions. The members of struct rlimit
are longs.
Thu Oct 24 17:43:34 1996 Andreas Schwab <schwab@issan.informatik.uni-dortmund.de>
* MakeTAGS (sysdep-dirs): Fix typo.
Wed Oct 23 03:45:22 1996 Ulrich Drepper <drepper@cygnus.com>
* Makefile (headers): Don't mention libc-version.h.
(install-others): ...but here.
* time/strptime.c: Recognize %s, %u, %g, and %G format.
nothing is found. This guarantees all subsequent calls behave
* sysdeps/unix/sysv/linux/syscalls.list: Change function name for
* io/getwd.c (getwd) [! PATH_MAX]: Don't assume that the user's
buffer is any longer than the amount necessary to hold the
filename; the Hurd getcwd uses the *entire* contents of the
buffer, however long it is specified to be.
* posix/getconf.c: De-ANSI-fy. Recognize POSIX.2 constant names.
since these do not depend on the platform.
* intl/Makefile (copysrc): Add missing > in sed cmd.
Sat Mar 23 17:52:49 1996 Ulrich Drepper <drepper@gnu.ai.mit.edu>
* Makeconfig: Rename Makefile variable nlsdir to i18ndir and
change value to $(datadir)/i18n. `nls' is not an appropriate
name.
* Makefile (subdirs): Add new subdir wctype.
* ctype/ctype-info.c: Add new global variable __ctype_names
and initialize from _nl_C_LC_CTYPE.
* ctype/ctype.h: In P1003.3b/D11 `alnum' is a separate character
class. Use bit 11.
[_ISbit]: Protect definition of bitmasks because they are also
used in wctype.h.
* libio/genops.c (_IO_sputbackc, _IO_sungetc): Clear EOF flag
after successfully pushing back a character.
Fundamental changes in locale implementation. Almost nothing
from the old code is used anymore.
* locale/charmap.c, locale/collate.c, locale/config.h,
locale/ctypedump.c, locale/hash.h, locale/keyword.gperf,
locale/keyword.h, locale/loadlocale.c, locale/locale-ctype.c,
locale/locale.c locale/localeconv.c, locale/localedef.c,
locale/localedef.h, locale/locfile-hash.c, locale/locfile-lex.c,
locale/locfile-parse.c, locale/messages.c, locale/monetary.c,
locale/numeric.c, locale/setlocale.c, locale/token.h,
locale/xmalloc.c: Removed.
* locale/Makefile: Update for new locale implementation with
program source code distributed in subdir.
* locale/categories.def, locale/iso-4217.def: Updated file
for new locale implementation.
* locale/langinfo.h: Updated for new locale implementation.
(ERA_D_T_FMT, ERA_T_FMT): New official values according to
P1003.2b/D11.
(_NL_COLLATE_NRULES, _NL_COLLATE_RULES, _NL_COLLATE_HASH_SIZE,
_NL_COLLATE_HASH_LAYERS, _NL_COLLATE_TABLE_EB,
_NL_COLLATE_TABLE_EL, _NL_COLLATE_UNDEFINED, _NL_COLLATE_EXTRA_EB,
_NL_COLLATE_EXTRA_EL, _NL_CTYPE_NAMES_EB, _NL_CTYPE_NAMES_EL,
_NL_CTYPE_HASH_SIZE, _NL_CTYPE_HASH_LAYERS, _NL_CTYPE_CLASS_NAMES,
_NL_CTYPE_MAP_NAMES, _NL_CTYPE_WIDTH): New internal values for
extended LC_CTYPE and LC_COLLATE implementation.
* locale/simple-hash.c, locale/simple-hash.h, locale/xmalloc.c,
locale/xstrdup.c: Helper functions for locale related programs.
* locale/C-collate.c, locale/C-ctype.c,
locale/C-messages.c, locale/C-monetary.c,
locale/C-numeric.c, locale/C-time.c,
locale/lc-collate.c, locale/lc-ctype.c,
locale/lc-messages.c, locale/lc-monetary.c,
locale/lc-numeric.c, locale/lc-time.c: New implementation of locale
functions, and new generated "C" locale data.
* locale/loadlocale.c: Now handles word fields in locale binary
automatically by changing the endianess if necessary.
* locale/localeinfo.h (LIMAGIC): Changed magic number because
of incompatible changes.
(locale_data): Changed definition to allow word as a value type.
(coll_sort_rule): Values for collation sorting mode.
(_NL_CURRENT_WORD): New macro to access word value of locale entry.
(__collate_table, __collate_extra): Declare new global variables
for collation tables.
* locale/programs/charmap-kw.gperf, locale/programs/charmap-kw.h,
locale/programs/charmap.c, locale/programs/charset.c,
locale/programs/charset.h, locale/programs/config.h,
locale/programs/ctypedump.c, locale/programs/ld-collate.c,
locale/programs/ld-ctype.c, locale/programs/ld-messages.c,
locale/programs/ld-monetary.c, locale/programs/ld-numeric.c,
locale/programs/ld-time.c, locale/programs/linereader.c,
locale/programs/linereader.h, locale/programs/locale.c,
locale/programs/localedef.c, locale/programs/locales.h,
locale/programs/locfile-kw.gperf, locale/programs/locfile-kw.h,
locale/programs/locfile-token.h, locale/programs/locfile.c,
locale/programs/locfile.h, locale/programs/stringtrans.c,
locale/programs/stringtrans.h: Implementation of locale related
programs.
* locale/weight.h: Functions to access collation tables.
* posix/unistd.h: Define _POSIX2_LOCALEDEF.
* stdio-common/printf_fp.c: Fix bug with printing certain numbers
< 10^-1. Reported by Bill Metzenthen.
* stdio-common/tfformat.c: Add new test for above bug.
* string/strcoll.c, string/strxfrm.c: Real implementation of
string collation according to ISO C.
* wctype/Makefile, wctype/cname-lookup.h, wctype/iswctype.c,
wctype/test_wctype.c, wctype/towctrans.c, wctype/wcfuncs.c,
wctype/wctrans.c, wctype/wctype.c, wctype/wctype.h: New files.
Implementation of wide character classes and mapping.
