Commit Graph

505 Commits

Author SHA1 Message Date
lijianglin
e1d3312015 add GB18030-2022 charmap and test the entire GB18030 charmap [BZ #30243]
support GB18030-2022 after add and change some transcoding relationship
of GB18030-2022.Details are as follows:
add 25 transcoding relationship
  UE81E 0x82359037
  UE826 0x82359038
  UE82B 0x82359039
  UE82C 0x82359130
  UE832 0x82359131
  UE843 0x82359132
  UE854 0x82359133
  UE864 0x82359134
  UE78D 0x84318236
  UE78F 0x84318237
  UE78E 0x84318238
  UE790 0x84318239
  UE791 0x84318330
  UE792 0x84318331
  UE793 0x84318332
  UE794 0x84318333
  UE795 0x84318334
  UE796 0x84318335
  UE816 0xfe51
  UE817 0xfe52
  UE818 0xfe53
  UE831 0xfe6c
  UE83B 0xfe76
  UE855 0xfe91
change 6 transcoding relationship
  U20087 0x95329031
  U20089 0x95329033
  U200CC 0x95329730
  U215D7 0x9536b937
  U2298F 0x9630ba35
  U241FE 0x9635b630
Test the entire GB18030 charmap, not only the Unicode BMP part.

Co-authored-by: yangyanchao <yangyanchao6@huawei.com>
Co-authored-by: liqingqing <liqingqing3@huawei.com>
Co-authored-by: Bruno Haible <bruno@clisp.org>
Reviewed-by: Andreas Schwab <schwab@suse.de>
Reviewed-by: Mike FABIAN <mfabian@redhat.com>
2023-08-29 19:02:30 +02:00
Paul Pluzhnikov
f58a8c1c15 Fix misspellings in iconv/ and iconvdata/ -- BZ 25337
All the changes are in comments or '#error' messages.
Applying this commit results in bit-identical rebuild of iconvdata/*.so

Reviewed-by: Florian Weimer <fw@deneb.enyo.de>
2023-05-27 16:37:14 +00:00
Adhemerval Zanella Netto
91fc5b9990 Remove --with-default-link configure option
Now that there is no need to use a special linker script to hardening
internal data structures, remove the --with-default-link configure
option and associated definitions.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2023-03-27 13:57:55 -03:00
Adhemerval Zanella
5729e0e9af iconv: Remove _STRING_ARCH_unaligned usage for get/set macros
And use a packed structure instead.  The compiler generates optimized
unaligned code if the architecture supports it.

Checked on x86_64-linux-gnu and i686-linux-gnu.

Reviewed-by: Wilco Dijkstra  <Wilco.Dijkstra@arm.com>
2023-02-17 15:56:54 -03:00
Wilco Dijkstra
32c7acd464 Replace rawmemchr (s, '\0') with strchr
Almost all uses of rawmemchr find the end of a string.  Since most targets use
a generic implementation, replacing it with strchr is better since that is
optimized by compilers into strlen (s) + s.  Also fix the generic rawmemchr
implementation to use a cast to unsigned char in the if statement.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2023-02-06 16:16:19 +00:00
Joseph Myers
6d7e8eda9b Update copyright dates with scripts/update-copyrights 2023-01-06 21:14:39 +00:00
наб
049781285d iconvdata/tst-table-charmap.sh: remove handling of old, borrowed format
This "Old POSIX/DKUUG borrowed format" handling is original to the file
and doesn't seem to have ever been used, i.e. id/t-t-c doesn't seem to
have ever been called with argv[1] == POSIX.

Upcoming is a POSIX charmap, which would inadvertently trigger this.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Florian Weimer <fweimer@redhat.com>
2022-11-09 13:48:46 +01:00
Tom Honermann
598f790fb1 gconv: Correct Big5-HKSCS conversion to preserve all state bits. [BZ #25744]
This patch corrects the Big5-HKSCS converter to preserve the lowest 3 bits of
the mbstate_t __count data member when the converter encounters an incomplete
multibyte character.

This fixes BZ #25744.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-07-06 09:27:13 -03:00
Sam James
7df596a58c grep: egrep -> grep -E, fgrep -> grep -F
Newer versions of GNU grep (after grep 3.7, not inclusive) will warn on
'egrep' and 'fgrep' invocations.

Convert usages within the tree to their expanded non-aliased counterparts
to avoid irritating warnings during ./configure and the test suite.

Signed-off-by: Sam James <sam@gentoo.org>
Reviewed-by: Fangrui Song <maskray@google.com>
2022-06-05 12:09:02 -07:00
Noah Goldstein
535e935a28 Replace {u}int_fast{16|32} with {u}int32_t
On 32-bit machines this has no affect. On 64-bit machines
{u}int_fast{16|32} are set as {u}int64_t which is often not
ideal. Particularly x86_64 this change both saves code size and
may save instruction cost.

Full xcheck passes on x86_64.
2022-04-13 21:23:04 -05:00
Adhemerval Zanella
11ba44f3a7 iconvdata: Fix enum type on UTF-7
Checked on x86_64-linux-gnu.
2022-03-23 15:35:35 -03:00
Max Gautier
9df157b4ed iconv: Add UTF-7-IMAP variant in utf-7.c
UTF-7-IMAP differs from UTF-7 in the followings ways (see RFC 3501[1]
for reference) :

- The shift character is '&' instead of '+'
- There is no "optional direct characters" and the "direct characters"
  set is different
- There is no implicit shift back to US-ASCII from BASE64, all BASE64
  sequences MUST be terminated with '-'

[1]: https://datatracker.ietf.org/doc/html/rfc3501#section-5.1.3

Signed-off-by: Max Gautier <mg@max.gautier.name>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-03-21 13:23:57 -03:00
Max Gautier
ef7b963280 iconv: make utf-7.c able to use variants
Add infrastructure in utf-7.c to handle variants. The approach comes from
iso646.c
The variant is defined at gconv_init time and is passed as a
supplementary variable.

Signed-off-by: Max Gautier <mg@max.gautier.name>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-03-21 13:23:56 -03:00
Max Gautier
09abb567a9 iconv: Better mapping to RFC for UTF-7
- Direct use of characters instead of arcane arrays
- isxbase64 is not the Modified BASE64 alphabet, but the characters who
  needs to trigger an explicit shift back to US-ASCII. Make that clearer

Signed-off-by: Max Gautier <mg@max.gautier.name>
Reviewed-by: Adhemerval Zanellla  <adhemerval.zanella@linaro.org>
2022-03-21 09:25:52 -03:00
Max Gautier
cd08388523 iconv: Always encode "optional direct" UTF-7 characters
Signed-off-by: Max Gautier <mg@max.gautier.name>
Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2022-03-21 09:25:43 -03:00
Paul Eggert
581c785bf3 Update copyright dates with scripts/update-copyrights
I used these shell commands:

../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")

and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.

I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah.  I don't
know why I run into these diagnostics whereas others evidently do not.

remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
2022-01-01 11:40:24 -08:00
Siddhesh Poyarekar
72e4a717bd Fix The GNU ToolChain Authors copyright notice
I (and maybe one or two others) added a (C) to the copyright notice
regardless of the contribution checklist[1] not mentioning it.  Fix all
these instances so that the notice reads as "Copyright The GNU Toolchain
Authors" across the source code.

[1] https://sourceware.org/glibc/wiki/Contribution%20checklist

Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-12-17 07:59:45 +05:30
Nikita Popov
ff012870b2 gconv: Do not emit spurious NUL character in ISO-2022-JP-3 (bug 28524)
Bugfix 27256 has introduced another issue:
In conversion from ISO-2022-JP-3 encoding, it is possible
to force iconv to emit extra NUL character on internal state reset.
To do this, it is sufficient to feed iconv with escape sequence
which switches active character set.
The simplified check 'data->__statep->__count != ASCII_set'
introduced by the aforementioned bugfix picks that case and
behaves as if '\0' character has been queued thus emitting it.

To eliminate this issue, these steps are taken:
* Restore original condition
'(data->__statep->__count & ~7) != ASCII_set'.
It is necessary since bits 0-2 may contain
number of buffered input characters.
* Check that queued character is not NUL.
Similar step is taken for main conversion loop.

Bundled test case follows following logic:
* Try to convert ISO-2022-JP-3 escape sequence
switching active character set
* Reset internal state by providing NULL as input buffer
* Ensure that nothing has been converted.

Signed-off-by: Nikita Popov <npv1310@gmail.com>
2021-11-04 19:59:42 +01:00
Siddhesh Poyarekar
30891f35fa Remove "Contributed by" lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date.  Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.

Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions.  These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.

The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively.  These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:

https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dc
https://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2021-09-03 22:06:44 +05:30
Siddhesh Poyarekar
2d2d9f2b48 Move malloc hooks into a compat DSO
Remove all malloc hook uses from core malloc functions and move it
into a new library libc_malloc_debug.so.  With this, the hooks now no
longer have any effect on the core library.

libc_malloc_debug.so is a malloc interposer that needs to be preloaded
to get hooks functionality back so that the debugging features that
depend on the hooks, i.e. malloc-check, mcheck and mtrace work again.
Without the preloaded DSO these debugging features will be nops.
These features will be ported away from hooks in subsequent patches.

Similarly, legacy applications that need hooks functionality need to
preload libc_malloc_debug.so.

The symbols exported by libc_malloc_debug.so are maintained at exactly
the same version as libc.so.

Finally, static binaries will no longer be able to use malloc
debugging features since they cannot preload the debugging DSO.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
2021-07-22 18:37:59 +05:30
Siddhesh Poyarekar
06a1b79407 Reinstate gconv-modules as the default configuration file
Reinstate gconv-modules as the main file so that the configuration
files in gconv-modules.d/ become add-on configuration.  With this, the
effective user visible change is that GCONV_PATH can now have
supplementary configuration in GCONV_PATH/gconv-modules.d/ in addition
to the main GCONV_PATH/gconv-modules file.
2021-06-14 18:38:09 +05:30
Siddhesh Poyarekar
5a5b481365 iconvdata: Split out non-essential gconv module configuration
Split module configuration so that only the bare minimum charsets,
i.e. ANSI_X3.110, ISO8859-15, ISO8859-1, CP1252, UNICODE, UTF-16,
UTF-32 and UTF-7 are configured in gconv-modules.conf.  The remaining
module configurations are now in gconv-modules-extra.conf.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-06-09 09:47:16 +05:30
Siddhesh Poyarekar
fc5bfade69 iconvdata: Move gconv-modules configuration to gconv-modules.conf
Move all gconv-modules configuration files to gconv-modules.conf.
That is, the S390 extensions now become gconv-modules-s390.conf.  Move
both configuration files into gconv-modules.d.

Now GCONV_PATH/gconv-modules is read only for backward compatibility
for third-party gconv modules directories.

Reviewed-by: DJ Delorie <dj@redhat.com>
2021-06-09 09:47:16 +05:30
Florian Weimer
6f1c701026 dlfcn: Cleanups after -ldl is no longer required
This commit removes the ELF constructor and internal variables from
dlfcn/dlfcn.c.  The file now serves the same purpose as
nptl/libpthread-compat.c, so it is renamed to dlfcn/libdl-compat.c.
The use of libdl-shared-only-routines ensures that libdl.a is empty.

This commit adjusts the test suite not to use $(libdl).  The libdl.so
symbolic link is no longer installed.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>
2021-06-03 09:11:45 +02:00
Florian Weimer
f17164bd51 localedata: Use U+00AF MACRON in more EBCDIC charsets [BZ #27882]
This updates IBM256, IBM277, IBM278, IBM280, IBM284, IBM297, IBM424
in the same way that IBM273 was updated for bug 23290.

IBM256 and IBM424 still have holes after this change, so HAS_HOLES
is not updated.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
2021-05-18 07:21:45 +02:00
Florian Weimer
7d88c6142c gconv: Fix assertion failure in ISO-2022-JP-3 module (bug 27256)
The conversion loop to the internal encoding does not follow
the interface contract that __GCONV_FULL_OUTPUT is only returned
after the internal wchar_t buffer has been filled completely.  This
is enforced by the first of the two asserts in iconv/skeleton.c:

	      /* We must run out of output buffer space in this
		 rerun.  */
	      assert (outbuf == outerr);
	      assert (nstatus == __GCONV_FULL_OUTPUT);

This commit solves this issue by queuing a second wide character
which cannot be written immediately in the state variable, like
other converters already do (e.g., BIG5-HKSCS or TSCII).

Reported-by: Tavis Ormandy <taviso@gmail.com>
2021-01-27 14:02:47 +01:00
Paul Eggert
2b778ceb40 Update copyright dates with scripts/update-copyrights
I used these shell commands:

../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")

and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
2021-01-02 12:17:34 -08:00
Andreas Schwab
ee7a3144c9 Fix buffer overrun in EUC-KR conversion module (bz #24973)
The byte 0xfe as input to the EUC-KR conversion denotes a user-defined
area and is not allowed.  The from_euc_kr function used to skip two bytes
when told to skip over the unknown designation, potentially running over
the buffer end.
2020-12-21 09:01:49 +05:30
Dmitry V. Levin
14ef9c185b treewide: fix incorrect spelling of indices in comments
Replace 'indeces' with 'indices', the most annoying of these typos were
those found in elf.h which is a public header file copied to other
projects.
2020-12-11 02:00:00 +00:00
Arjun Shankar
9a99c68214 iconv: Accept redundant shift sequences in IBM1364 [BZ #26224]
The IBM1364, IBM1371, IBM1388, IBM1390 and IBM1399 character sets
share converter logic (iconvdata/ibm1364.c) which would reject
redundant shift sequences when processing input in these character
sets.  This led to a hang in the iconv program (CVE-2020-27618).

This commit adjusts the converter to ignore redundant shift sequences
and adds test cases for iconv_prog hangs that would be triggered upon
their rejection.  This brings the implementation in line with other
converters that also ignore redundant shift sequences (e.g. IBM930
etc., fixed in commit 692de4b396).

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2020-11-04 12:19:38 +01:00
Carlos O'Donell
c580e6466d Reset converter state after second wchar_t output (Bug 25734)
An input BIG5-HKSCS character may be converted into at most 2 wchar_t
characters. After outputting the second whcar_t character (which was
saved in the converter state) we must reset the state. If we fail
to reset the state we will be stuck continually copying that
character to the output even if we have further input to consider.

We add a new test case that covers the 4 BIG5-HKSCS characters
that may become 2 wchar_t characters.

Reviewed-by: Tom Honermann <tom@honermann.net>
2020-04-15 22:31:40 -04:00
Joseph Myers
d614a75396 Update copyright dates with scripts/update-copyrights. 2020-01-01 00:14:33 +00:00
Paul Eggert
5a82c74822 Prefer https to http for gnu.org and fsf.org URLs
Also, change sources.redhat.com to sourceware.org.
This patch was automatically generated by running the following shell
script, which uses GNU sed, and which avoids modifying files imported
from upstream:

sed -ri '
  s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g
  s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g
' \
  $(find $(git ls-files) -prune -type f \
      ! -name '*.po' \
      ! -name 'ChangeLog*' \
      ! -path COPYING ! -path COPYING.LIB \
      ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \
      ! -path manual/texinfo.tex ! -path scripts/config.guess \
      ! -path scripts/config.sub ! -path scripts/install-sh \
      ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \
      ! -path INSTALL ! -path  locale/programs/charmap-kw.h \
      ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \
      ! '(' -name configure \
            -execdir test -f configure.ac -o -f configure.in ';' ')' \
      ! '(' -name preconfigure \
            -execdir test -f preconfigure.ac ';' ')' \
      -print)

and then by running 'make dist-prepare' to regenerate files built
from the altered files, and then executing the following to cleanup:

  chmod a+x sysdeps/unix/sysv/linux/riscv/configure
  # Omit irrelevant whitespace and comment-only changes,
  # perhaps from a slightly-different Autoconf version.
  git checkout -f \
    sysdeps/csky/configure \
    sysdeps/hppa/configure \
    sysdeps/riscv/configure \
    sysdeps/unix/sysv/linux/csky/configure
  # Omit changes that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines
  git checkout -f \
    sysdeps/powerpc/powerpc64/ppc-mcount.S \
    sysdeps/unix/sysv/linux/s390/s390-64/syscall.S
  # Omit change that caused a pre-commit check to fail like this:
  # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline
  git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S
2019-09-07 02:43:31 -07:00
Joseph Myers
c4f50205e1 Add some spaces before '('.
This patch fixes various places where a space should have been present
before '(' in accordance with the GNU Coding Standards.  Most but not
all of the fixes in this patch are for calls to sizeof (but it's not
exhaustive regarding such calls that should be fixed).

Tested for x86_64, and with build-many-glibcs.py.

	* benchtests/bench-strcpy.c (do_test): Use space before '('.
	* benchtests/bench-string.h (cmdline_process_function): Likewise.
	* benchtests/bench-strlen.c (do_test): Likewise.
	(test_main): Likewise.
	* catgets/gencat.c (read_old): Likewise.
	* elf/cache.c (load_aux_cache): Likewise.
	* iconvdata/bug-iconv8.c (do_test): Likewise.
	* math/test-tgmath-ret.c (do_test): Likewise.
	* nis/nis_call.c (rec_dirsearch): Likewise.
	* nis/nis_findserv.c (__nis_findfastest_with_timeout): Likewise.
	* nptl/tst-audit-threads.c (do_test): Likewise.
	* nptl/tst-cancel4-common.h (set_socket_buffer): Likewise.
	* nss/nss_test1.c (init): Likewise.
	* nss/test-netdb.c (test_hosts): Likewise.
	* posix/execvpe.c (maybe_script_execute): Likewise.
	* stdio-common/tst-fmemopen4.c (do_test): Likewise.
	* stdio-common/tst-printf.c (do_test): Likewise.
	* stdio-common/vfscanf-internal.c (__vfscanf_internal): Likewise.
	* stdlib/fmtmsg.c (NKEYWORDS): Likewise.
	* stdlib/qsort.c (STACK_SIZE): Likewise.
	* stdlib/test-canon.c (do_test): Likewise.
	* stdlib/tst-swapcontext1.c (do_test): Likewise.
	* string/memcmp.c (OPSIZ): Likewise.
	* string/test-strcpy.c (do_test): Likewise.
	(do_random_tests): Likewise.
	* string/test-strlen.c (do_test): Likewise.
	(test_main): Likewise.
	* string/test-strrchr.c (do_test): Likewise.
	(do_random_tests): Likewise.
	* string/tester.c (test_memrchr): Likewise.
	(test_memchr): Likewise.
	* sysdeps/generic/memcopy.h (OPSIZ): Likewise.
	* sysdeps/generic/unwind-dw2.c (execute_stack_op): Likewise.
	* sysdeps/generic/unwind-pe.h (read_sleb128): Likewise.
	(read_encoded_value_with_base): Likewise.
	* sysdeps/hppa/dl-machine.h (elf_machine_runtime_setup): Likewise.
	* sysdeps/hppa/fpu/feupdateenv.c (__feupdateenv): Likewise.
	* sysdeps/ia64/fpu/sfp-machine.h (TI_BITS): Likewise.
	* sysdeps/mach/hurd/spawni.c (__spawni): Likewise.
	* sysdeps/posix/spawni.c (maybe_script_execute): Likewise.
	* sysdeps/powerpc/fpu/tst-setcontext-fpscr.c (query_auxv):
	Likewise.
	* sysdeps/unix/sysv/linux/aarch64/bits/procfs.h (ELF_NGREG):
	Likewise.
	* sysdeps/unix/sysv/linux/arm/bits/procfs.h (ELF_NGREG): Likewise.
	* sysdeps/unix/sysv/linux/arm/ioperm.c (init_iosys): Likewise.
	* sysdeps/unix/sysv/linux/csky/bits/procfs.h (ELF_NGREG):
	Likewise.
	* sysdeps/unix/sysv/linux/m68k/bits/procfs.h (ELF_NGREG):
	Likewise.
	* sysdeps/unix/sysv/linux/nios2/bits/procfs.h (ELF_NGREG):
	Likewise.
	* sysdeps/unix/sysv/linux/spawni.c (maybe_script_execute):
	Likewise.
	* sysdeps/unix/sysv/linux/x86/bits/procfs.h (ELF_NGREG): Likewise.
	* sysdeps/unix/sysv/linux/x86/bits/sigcontext.h
	(FP_XSTATE_MAGIC2_SIZE): Likewise.
	* sysdeps/x86/fpu/sfp-machine.h (TI_BITS): Likewise.
	* time/test_time.c (main): Likewise.
2019-02-27 13:55:45 +00:00
Joseph Myers
107562682c Adjust wording of two fall-through comments.
In two places in glibc, -Wextra produces implicit-fallthrough warnings
where there are comments about the fall-through but their wording
doesn't match one of the forms expected by the default
implicit-fallthrough level.  This patch adjusts those two places to
have a comment in a form that is accepted, so avoiding the warning
(this seems preferable to only being able to use a looser level of the
warning that allows any comment at all as evidence of deliberate
fall-through).

Tested for x86_64.

	* iconvdata/cns11643.h (ucs4_to_cns11643): Adjust fall-through
	comment wording.
	* nis/nis_call.c (__do_niscall3): Likewise.
2019-02-13 18:37:54 +00:00
Joseph Myers
04277e02d7 Update copyright dates with scripts/update-copyrights.
* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.
2019-01-01 00:11:28 +00:00
Andreas Schwab
21526a507d Fix out-of-bounds access in IBM-1390 converter (bug 23448)
The IBM-1390 converter can consume/produce two UCS4 characters in each
loop.
2018-07-24 16:45:46 +02:00
Florian Weimer
14beef7575 localedata: Make IBM273 compatible with ISO-8859-1 [BZ #23290]
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-06-14 22:34:10 +02:00
Andreas Schwab
326e74e7c1 Add aliases to recognize normalized charset names (bug 19527) 2018-04-16 17:30:56 +02:00
Joseph Myers
688903eb3e Update copyright dates with scripts/update-copyrights.
* All files with FSF copyright notices: Update copyright dates
	using scripts/update-copyrights.
	* locale/programs/charmap-kw.h: Regenerated.
	* locale/programs/locfile-kw.h: Likewise.
2018-01-01 00:32:25 +00:00
Mike FABIAN
799c8d6905 Add new codepage charmaps/IBM858 [BZ #21084]
This code page is identical to code page 850 except that X'D5'
has been changed from LI61 (dotless i) to SC20 (euro symbol).

The code points from /x01 to /x1f in the /localedata/charmaps/IBM858
file have the same mapping as those in localedata/charmaps/ANSI_X3.4-1968.
That means they disagree with with

ftp://ftp.software.ibm.com/software/globalization/gcoc/attachments/CP00858.txt

in that range.
For example, localedata/charmaps/IBM858 and localedata/charmaps/ANSI_X3.4-1968 have:

   “<U0001>     /x01         START OF HEADING (SOH)”

whereas CP00858.txt has:

   “01 SS000000        Smiling Face”

That means that CP00858.txt is not really ASCII-compatible and to make
it ASCII-compatible we deviate fro CP00858.txt in the code points from /x01
to /x1f.

	[BZ #21084]
	* benchtests/strcoll-inputs/filelist#en_US.UTF-8: Add IBM858 and ibm858.c.
	* iconvdata/Makefile: Add IBM858.
	* iconvdata/gconv-modules: Add IBM858.
	* iconvdata/ibm858.c: New file.
	* iconvdata/tst-tables.sh: Add IBM858
	* localedata/charmaps/IBM858: New file.
2017-09-14 15:50:57 +02:00
Joseph Myers
3cefdd7310 Increase some test timeouts.
This patch increases the timeouts for some tests that I've seen timing
out on slow systems in my 2.26 release testing.  (In the case of
tst-tsearch.c, increasing the timeout means removing a setting of 10
that was put there before the default timeout was increased to 20
seconds, so putting the default into effect.)

	* iconvdata/tst-loading.c (TIMEOUT): Define to 30.
	* misc/tst-tsearch.c (TIMEOUT): Remove.
	* nptl/tst-create-detached.c (TIMEOUT): Define to 100.
	* nptl/tst-robust-fork.c (TIMEOUT): Likewise.
	* nptl/tst-rwlock19.c (TIMEOUT): Likewise.
	* string/tst-cmp.c (TIMEOUT): Define to 600.
2017-07-06 17:01:03 +00:00
Zack Weinberg
2bfdaeddaa Rename cppflags-iterator.mk to libof-iterator.mk, remove extra-modules.mk.
cppflags-iterator.mk no longer has anything to do with CPPFLAGS; all
it does is set libof-$(foo) for a list of files.  extra-modules.mk
does the same thing, but with a different input variable, and doesn't
let the caller control the module.  Therefore, this patch gives
cppflags-iterator.mk a better name, removes extra-modules.mk, and
updates all uses of both.

	* extra-modules.mk: Delete file.
	* cppflags-iterator.mk: Rename to ...
	* libof-iterator.mk: ...this.  Adjust comments.

	* Makerules, extra-lib.mk, benchtests/Makefile, elf/Makefile
	* elf/rtld-Rules, iconv/Makefile, locale/Makefile, malloc/Makefile
	* nscd/Makefile, sunrpc/Makefile, sysdeps/s390/Makefile:
	Use libof-iterator.mk instead of cppflags-iterator.mk or
	extra-modules.mk.

	* benchtests/strcoll-inputs/filelist#en_US.UTF-8: Remove
	extra-modules.mk and cppflags-iterator.mk, add libof-iterator.mk.
2017-05-09 07:06:29 -04:00
Florian Weimer
2d6ab5df3b Document and fix --enable-bind-now [BZ #21015] 2017-03-02 14:44:28 +01:00
Zack Weinberg
9090848d06 Narrowing the visibility of libc-internal.h even further.
posix/wordexp-test.c used libc-internal.h for PTR_ALIGN_DOWN; similar
to what was done with libc-diag.h, I have split the definitions of
cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and PTR_ALIGN_DOWN
to a new header, libc-pointer-arith.h.

It then occurred to me that the remaining declarations in libc-internal.h
are mostly to do with early initialization, and probably most of the
files including it, even in the core code, don't need it anymore.  Indeed,
only 19 files actually need what remains of libc-internal.h.  23 others
need libc-diag.h instead, and 12 need libc-pointer-arith.h instead.
No file needs more than one of them, and 16 don't need any of them!

So, with this patch, libc-internal.h stops including libc-diag.h as
well as losing the pointer arithmetic macros, and all including files
are adjusted.

        * include/libc-pointer-arith.h: New file.  Define
	cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and
        PTR_ALIGN_DOWN here.
        * include/libc-internal.h: Definitions of above macros
	moved from here.  Don't include libc-diag.h anymore either.
	* posix/wordexp-test.c: Include stdint.h and libc-pointer-arith.h.
        Don't include libc-internal.h.

	* debug/pcprofile.c, elf/dl-tunables.c, elf/soinit.c, io/openat.c
	* io/openat64.c, misc/ptrace.c, nptl/pthread_clock_gettime.c
	* nptl/pthread_clock_settime.c, nptl/pthread_cond_common.c
	* string/strcoll_l.c, sysdeps/nacl/brk.c
	* sysdeps/unix/clock_settime.c
	* sysdeps/unix/sysv/linux/i386/get_clockfreq.c
	* sysdeps/unix/sysv/linux/ia64/get_clockfreq.c
	* sysdeps/unix/sysv/linux/powerpc/get_clockfreq.c
	* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c:
	Don't include libc-internal.h.

	* elf/get-dynamic-info.h, iconv/loop.c
	* iconvdata/iso-2022-cn-ext.c, locale/weight.h, locale/weightwc.h
	* misc/reboot.c, nis/nis_table.c, nptl_db/thread_dbP.h
	* nscd/connections.c, resolv/res_send.c, soft-fp/fmadf4.c
	* soft-fp/fmasf4.c, soft-fp/fmatf4.c, stdio-common/vfscanf.c
	* sysdeps/ieee754/dbl-64/e_lgamma_r.c
	* sysdeps/ieee754/dbl-64/k_rem_pio2.c
	* sysdeps/ieee754/flt-32/e_lgammaf_r.c
	* sysdeps/ieee754/flt-32/k_rem_pio2f.c
	* sysdeps/ieee754/ldbl-128/k_tanl.c
	* sysdeps/ieee754/ldbl-128ibm/k_tanl.c
	* sysdeps/ieee754/ldbl-96/e_lgammal_r.c
	* sysdeps/ieee754/ldbl-96/k_tanl.c, sysdeps/nptl/futex-internal.h:
	Include libc-diag.h instead of libc-internal.h.

        * elf/dl-load.c, elf/dl-reloc.c, locale/programs/locarchive.c
        * nptl/nptl-init.c, string/strcspn.c, string/strspn.c
	* malloc/malloc.c, sysdeps/i386/nptl/tls.h
	* sysdeps/nacl/dl-map-segments.h, sysdeps/x86_64/atomic-machine.h
	* sysdeps/unix/sysv/linux/spawni.c
        * sysdeps/x86_64/nptl/tls.h:
        Include libc-pointer-arith.h instead of libc-internal.h.

	* elf/get-dynamic-info.h, sysdeps/nacl/dl-map-segments.h
	* sysdeps/x86_64/atomic-machine.h:
        Add multiple include guard.
2017-03-01 20:33:46 -05:00
Zack Weinberg
ceaa98897c Add missing header files throughout the testsuite.
* crypt/md5.h: Test _LIBC with #if defined, not #if.
	* dirent/opendir-tst1.c: Include sys/stat.h.
	* dirent/tst-fdopendir.c: Include sys/stat.h.
	* dirent/tst-fdopendir2.c: Include stdlib.h.
	* dirent/tst-scandir.c: Include stdbool.h.
	* elf/tst-auditmod1.c: Include link.h and stddef.h.
	* elf/tst-tls15.c: Include stdlib.h.
	* elf/tst-tls16.c: Include stdlib.h.
	* elf/tst-tls17.c: Include stdlib.h.
	* elf/tst-tls18.c: Include stdlib.h.
	* iconv/tst-iconv6.c: Include endian.h.
	* iconvdata/bug-iconv11.c: Include limits.h.
	* io/test-utime.c: Include stdint.h.
	* io/tst-faccessat.c: Include sys/stat.h.
	* io/tst-fchmodat.c: Include sys/stat.h.
	* io/tst-fchownat.c: Include sys/stat.h.
	* io/tst-fstatat.c: Include sys/stat.h.
	* io/tst-futimesat.c: Include sys/stat.h.
	* io/tst-linkat.c: Include sys/stat.h.
	* io/tst-mkdirat.c: Include sys/stat.h and stdbool.h.
	* io/tst-mkfifoat.c: Include sys/stat.h and stdbool.h.
	* io/tst-mknodat.c: Include sys/stat.h and stdbool.h.
	* io/tst-openat.c: Include stdbool.h.
	* io/tst-readlinkat.c: Include sys/stat.h.
	* io/tst-renameat.c: Include sys/stat.h.
	* io/tst-symlinkat.c: Include sys/stat.h.
	* io/tst-unlinkat.c: Include stdbool.h.
	* libio/bug-memstream1.c: Include stdlib.h.
	* libio/bug-wmemstream1.c: Include stdlib.h.
	* libio/tst-fwrite-error.c: Include stdlib.h.
	* libio/tst-memstream1.c: Include stdlib.h.
	* libio/tst-memstream2.c: Include stdlib.h.
	* libio/tst-memstream3.c: Include stdlib.h.
	* malloc/tst-interpose-aux.c: Include stdint.h.
	* misc/tst-preadvwritev-common.c: Include sys/stat.h.
	* nptl/tst-basic7.c: Include limits.h.
	* nptl/tst-cancel25.c: Include pthread.h, not pthreadP.h.
	* nptl/tst-cancel4.c: Include stddef.h, limits.h, and sys/stat.h.
	* nptl/tst-cancel4_1.c: Include stddef.h.
	* nptl/tst-cancel4_2.c: Include stddef.h.
	* nptl/tst-cond16.c: Include limits.h.
	Use sysconf(_SC_PAGESIZE) instead of __getpagesize.
	* nptl/tst-cond18.c: Include limits.h.
	Use sysconf(_SC_PAGESIZE) instead of __getpagesize.
	* nptl/tst-cond4.c: Include stdint.h.
	* nptl/tst-cond6.c: Include stdint.h.
	* nptl/tst-stack2.c: Include limits.h.
	* nptl/tst-stackguard1.c: Include stddef.h.
	* nptl/tst-tls4.c: Include stdint.h. Don't include tls.h.
	* nptl/tst-tls4moda.c: Include stddef.h.
	Don't include stdio.h, unistd.h, or tls.h.
	* nptl/tst-tls4modb.c: Include stddef.h.
	Don't include stdio.h, unistd.h, or tls.h.
	* nptl/tst-tls5.h: Include stddef.h. Don't include stdlib.h or tls.h.
	* posix/tst-getaddrinfo2.c: Include stdio.h.
	* posix/tst-getaddrinfo5.c: Include stdio.h.
	* posix/tst-pathconf.c: Include sys/stat.h.
	* posix/tst-posix_fadvise-common.c: Include stdint.h.
	* posix/tst-preadwrite-common.c: Include sys/stat.h.
	* posix/tst-regex.c: Include stdint.h.
	Don't include spawn.h or spawn_int.h.
	* posix/tst-regexloc.c: Don't include spawn.h or spawn_int.h.
	* posix/tst-vfork3.c: Include sys/stat.h.
	* resolv/tst-bug18665-tcp.c: Include stdlib.h.
	* resolv/tst-res_hconf_reorder.c: Include stdlib.h.
	* resolv/tst-resolv-search.c: Include stdlib.h.
	* stdio-common/tst-fmemopen2.c: Include stdint.h.
	* stdio-common/tst-vfprintf-width-prec.c: Include stdlib.h.
	* stdlib/test-canon.c: Include sys/stat.h.
	* stdlib/tst-tls-atexit.c: Include stdbool.h.
	* string/test-memchr.c: Include stdint.h.
	* string/tst-cmp.c: Include stdint.h.
	* sysdeps/pthread/tst-timer.c: Include stdint.h.
	* sysdeps/unix/sysv/linux/tst-sync_file_range.c: Include stdint.h.
	* sysdeps/wordsize-64/tst-writev.c: Include limits.h and stdint.h.
	* sysdeps/x86_64/fpu/math-tests-arch.h: Include cpu-features.h.
	Don't include init-arch.h.
	* sysdeps/x86_64/multiarch/test-multiarch.h: Include cpu-features.h.
	Don't include init-arch.h.
	* sysdeps/x86_64/tst-auditmod10b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod3b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod4b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod5b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod6b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod6c.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod7b.c: Include link.h and stddef.h.
	* time/clocktest.c: Include stdint.h.
	* time/tst-posixtz.c: Include stdint.h.
	* timezone/tst-timezone.c: Include stdint.h.
2017-02-16 17:33:18 -05:00
Joseph Myers
983a9637f7 Increase some test timeouts.
This patch increases timeouts on some tests I've observed timing out.

elf/tst-tls13 and iconvdata/tst-loading both dynamically load many
objects and so are slow when testing over NFS.  They had timeouts set
from before the default changed from 2 to 20 seconds; this patch
removes those old settings, so effectively increasing the timeout to
20 seconds (from 3 and 10 seconds respectively).

malloc/tst-malloc-thread-fail.c and malloc/tst-mallocfork2.c are slow
on slow systems and so I set a fairly arbitrary 100 second timeout,
which seems to suffice on the system where I saw them timing out.

nss/tst-cancel-getpwuid_r.c and nss/tst-nss-getpwent.c are slow on
systems with a large passwd file; I set timeouts that empirically
worked for me.  (It seems tst-cancel-getpwuid_r.c is hitting the
100000 getpwuid_r call limit in my testing, with each call taking a
bit over 0.007 seconds, so 700 seconds for the test.)

	* elf/tst-tls13.c (TIMEOUT): Remove.
	* iconvdata/tst-loading.c (TIMEOUT): Likewise.
	* malloc/tst-malloc-thread-fail.c (TIMEOUT): Increase to 100.
	* malloc/tst-mallocfork2.c (TIMEOUT): Define to 100.
	* nss/tst-cancel-getpwuid_r.c (TIMEOUT): Define to 900.
	* nss/tst-nss-getpwent.c (TIMEOUT): Define to 300.
2017-01-05 17:39:38 +00:00
Joseph Myers
bfff8b1bec Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
Florian Weimer
0415d32187 gconv: Adjust GBK to support the Euro sign
Commit aa4d00ca39 only updated the
data used by locales.
2016-11-29 18:35:21 +01:00
Carlos O'Donell
93fe09cb5f Bug 20729: Fix building with -Os.
This commit adds a new DIAG_IGNORE_Os_NEEDS_COMMENT which is only
enabled when compiling with -Os. This allows developers working on
-Os enabled builds to mark false-positive warnings without impacting the
warnings emitted at -O2.

Then using the new DIAG_IGNORE_Os_NEEDS_COMMENT we fix 6 warnings
generated with GCC 5 to get -Os builds working again.
2016-10-29 23:50:56 -04:00