All the crypt related functions, cryptographic algorithms, and
make requirements are removed, with only the exception of md5
implementation which is moved to locale folder since it is
required by localedef for integrity protection (libc's
locale-reading code does not check these, but localedef does
generate them).
Besides thec code itself, both internal documentation and the
manual is also adjusted. This allows to remove both --enable-crypt
and --enable-nss-crypt configure options.
Checked with a build for all affected ABIs.
Co-authored-by: Zack Weinberg <zack@owlfolio.org>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The locale generation are issues in parallel to try speed locale
generation. The maximum number of jobs are limited to the online
CPU (in hope to not overcommit on environments with lower cores
than tests).
On a Ryzen 9, the test execution improves from ~6.7s to ~1.4s.
Tested-by: Mark Wielaard <mark@klomp.org>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
During testing of localedef running in a minimal container
there were several error cases which were hard to diagnose
since they appeared as strerror (errno) values printed by the
higher level functions. This change adds three new verbose
messages for potential failure paths. The new messages give
the user the opportunity to use -v and display additional
information about why localedef might be failing. I found
these messages useful myself while writing a localedef
container test for --no-hard-links.
Since the changes cleanup the code that handle codeset
normalization we add tst-localedef-path-norm which contains
many sub-tests to verify the correct expected normalization of
codeset strings both when installing to default paths (the
only time normalization is enabled) and installing to absolute
paths. During the refactoring I created at least one
buffer-overflow which valgrind caught, but these tests did not
catch because the exec in the container had a very clean heap
with zero-initialized memory. However, between valgrind and
the tests the results are clean.
The new tst-localedef-path-norm passes without regression on
x86_64.
Change-Id: I28b9f680711ff00252a2cb15625b774cc58ecb9d
The arguments passed by Makefile are missing match with the arguments
taken by locale/tst-locale-locpath.sh. Without this patch
cross-test-ssh.sh will be called twice in a single command while
doing the make check test wish ssh test wrapper.
* locale/Makefile: Fix tst-locale-locpath arguments passing.
From localedef --help:
Output control:
...
--no-warnings=<warnings> Comma-separated list of warnings to disable;
supported warnings are: ascii, intcurrsym
...
--warnings=<warnings> Comma-separated list of warnings to enable;
supported warnings are: ascii, intcurrsym
Locales using SHIFT_JIS and SHIFT_JISX0213 character maps are not ASCII
compatible. In order to build locales using these character maps, and
have localedef exit with a status of 0, we add new option to localedef
to disable or enable specific warnings. The options are --no-warnings
and --warnings, to disable and enable specific warnings respectively.
The options take a comma-separated list of warning names. The warning
names are taken directly from the generated warning. When a warning
that can be disabled is issued it will print something like this: foo is
not defined [--no-warnings=foo]
For the initial implementation we add two controllable warnings; first
'ascii' which is used by the localedata installation makefile target to
install SHIFT_JIS and SHIFT_JISX0213-using locales without error; second
'intcurrsym' which allows a program to use a non-standard international
currency symbol without triggering a warning. The 'intcurrsym' is
useful in the future if country codes are added that are not in our
current ISO 4217 list, and the user wants to avoid the warning. Having
at least two warnings to control gives an example for how the changes
can be extended to more warnings if required in the future.
These changes allow ja_JP.SHIFT_JIS and ja_JP.SHIFT_JISX0213 to be
compiled without warnings using --no-warnings=ascii. The
localedata/Makefile $(INSTALL-SUPPORTED-LOCALES) target is adjusted to
automatically add `--no-warnings=ascii` for such charmaps, and likewise
localedata/gen-locale.sh is adjusted with similar logic.
v2: Bring verbose, be_quiet, and all warning control booleans into
record-status.c, and compile this object file to be used by locale,
iconv, and localedef. Any users include record-status.h.
v3: Fix an instance of boolean coercion in set_warning().
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
xlocale.h is already a single-type micro-header, defining struct
__locale_struct and the typedefs __locale_t and locale_t. This patch
brings it into the bits/types/ scheme: there are now
bits/types/__locale_t.h which defines only __locale_struct and
__locale_t, and bits/types/locale_t.h which defines locale_t as well
as the other two. None of *our* headers need __locale_t.h, but it
appears to me that libstdc++ could make use of it.
There are a lot of external uses of xlocale.h, but all the uses I
checked had an autoconf test or equivalent for its existence. It has
never been available from other C libraries, and it has always
contained a comment reading "This file is not standardized, don't rely
on it, it can go away without warning" so I think dropping it is
pretty safe.
I also took the opportunity to clean up comments in various public
header files that still talk about the *_l interfaces as though they
were completely nonstandard. There are a few of them, notably the
strtoX_l and wcstoX_l families, that haven't been standardized, but
the bulk are in POSIX.1-2008.
* locale/xlocale.h: Rename to...
* locale/bits/types/__locale_t.h: ...here. Adjust commentary.
Only define struct __locale_struct and __locale_t, not locale_t.
* locale/bits/types/locale_t.h: New file; define locale_t here.
* locale/Makefile (headers): Update to match.
* include/xlocale.h: Delete wrapper.
* include/bits/types/__locale_t.h: New wrapper.
* include/bits/types/locale_t.h: New wrapper.
* ctype/ctype.h, include/printf.h, include/time.h
* locale/langinfo.h, locale/locale.h, stdlib/monetary.h
* stdlib/stdlib.h, string/string.h, string/strings.h, time/time.h
* wcsmbs/wchar.h, wctype/wctype.h: Use bits/types/locale_t.h.
Correct outdated comments regarding the standardization status of
the functions that take locale_t arguments.
* stdlib/strtod_l.c, stdlib/strtof_l.c, stdlib/strtol_l.c
* stdlib/strtold_l.c, stdlib/strtoul_l.c, stdlib/strtoull_l.c
* sysdeps/ieee754/ldbl-128ibm/strtold_l.c
* sysdeps/ieee754/ldbl-64-128/strtold_l.c
* wcsmbs/wcstod.c, wcsmbs/wcstod_l.c, wcsmbs/wcstof.c
* wcsmbs/wcstof_l.c, wcsmbs/wcstold.c, wcsmbs/wcstold_l.c:
Don't include xlocale.h. If necessary, include locale.h instead.
* stdlib/strtold_l.c: Unconditionally include wchar.h.
cppflags-iterator.mk no longer has anything to do with CPPFLAGS; all
it does is set libof-$(foo) for a list of files. extra-modules.mk
does the same thing, but with a different input variable, and doesn't
let the caller control the module. Therefore, this patch gives
cppflags-iterator.mk a better name, removes extra-modules.mk, and
updates all uses of both.
* extra-modules.mk: Delete file.
* cppflags-iterator.mk: Rename to ...
* libof-iterator.mk: ...this. Adjust comments.
* Makerules, extra-lib.mk, benchtests/Makefile, elf/Makefile
* elf/rtld-Rules, iconv/Makefile, locale/Makefile, malloc/Makefile
* nscd/Makefile, sunrpc/Makefile, sysdeps/s390/Makefile:
Use libof-iterator.mk instead of cppflags-iterator.mk or
extra-modules.mk.
* benchtests/strcoll-inputs/filelist#en_US.UTF-8: Remove
extra-modules.mk and cppflags-iterator.mk, add libof-iterator.mk.
In 1999 the project split "localedir" into "localedir" (path to compiled
locale archives) and "msgcatdir" (path to message catalogs). This
predates the 2002 change in the GNU Coding Standard to document the use
of "localedir" for the path to the message catalogs. It appears that
newlib, gcc, and several other projects also used "msgcatdir" at one
point or another in the past, and so it is in line with historical
precedent that glibc would also use "msgcatdir." However, given that the
GNU Coding Standard uses "localedir", we will switch to that for
consistency as a GNU project. Previous uses of --localdir didn't work
anyway (see bug 14259).
I am committing this patch in the understanding that nobody would object
to fixing #14259 as part of aligning our variable usage to the GNU
Coding Standard.
Given that previous "localedir" uses were converted to "complocaledir"
by [1], we can now convert "msgcatdir" to "localedir" and complete the
transition. With an addition to config.make.in we also fix bug 14259 and
allow users to specify the locale dependent data directory with
"--localedir" at configure time. There is still no way to control at
configure time the location of the *compiled* locale directory.
Tested on x86_64 with no regressions.
Tested using "--localedir" to specify alternate locale dependent data
directory and verified with "make install DESTDIR=/tmp/glibc".
[1] 90fe682d30
In preparation to fix the --localedir configure argument we must
move the existing conflicting definition of localedir to a more
appropriate name. Given that all current internal uses of localedir
relate to the compiled locales we rename to complocaledir.
The current scheme to identify which module a translation unit is
built in depends on defining multiple macros IS_IN_* and also defining
NOT_IN_libc if we're building a non-libc module. In addition, there
is an IN_LIB macro that does effectively the same thing, but for
different modules (notably the systemtap probes). This macro scheme
unifies both ideas to use just one macro IN_MODULE and assign it a
value depending on the module it is being built into. If the module
is not defined, it defaults to MODULE_libc.
Patches that follow will replace uses of IS_IN_* variables with the
IS_IN() macro. libc-symbols.h has been converted already to give an
example of how such a transition will look.
Verified that there are no relevant binary changes. One source change
that will crop up repeatedly is that of nscd_stat, since it uses the
build timestamp as a constant in its logic.
* Makeconfig (in-module): Get value of libof set for the
translation unit.
(CPPFLAGS): Use $(in-module).
* Makerules: Don't suffix routine names for nonlib.
* include/libc-modules.h: New file.
* include/libc-symbols.h: Include libc-modules.h
(IS_IN): New macro to replace IS_IN_* macros.
* elf/Makefile: Set libof-* for each routine.
* elf/rtld-Rules: Likewise.
* extra-modules.mk: Likewise.
* iconv/Makefile: Likewise.
* iconvdata/Makefile: Likewise.
* locale/Makefile: Likewise.
* malloc/Makefile: Likewise.
* nss/Makefile: Likewise.
* sysdeps/gnu/Makefile: Likewise.
* sysdeps/ieee754/ldbl-opt/Makefile: Likewise.
* sysdeps/unix/sysv/linux/Makefile: Likewise.
* sysdeps/s390/s390-64/Makefile: Likewise.
* nscd/Makefile: Set libof-* for each routine. Set CFLAGS and
CPPFLAGS for nscd instead of nonlib.
In <https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html> I
noted it was necessary to add includes of Makeconfig early in various
subdirectory makefiles for the tests-special variable settings added
by that patch to be conditional on configuration information. No-one
commented on the general question there of whether Makeconfig should
always be included immediately after the definition of subdir.
This patch implements that early inclusion of Makeconfig in each
directory (which is a lot easier than consistent placement of includes
of Rules). Includes are added if needed, or moved up if already
present. Subdirectory "all:" targets are removed, since Makeconfig
provides one.
There is potential for further cleanups I haven't done. Rules and
Makerules have code such as
ifneq "$(findstring env,$(origin headers))" ""
headers :=
endif
to override to empty any value of various variables that came from the
environment. I think there is a case for Makeconfig setting all the
subdirectory variables (other than subdir) to empty to ensure no
outside value is going to take effect if a subdirectory fails to
define a variable. (A list of such variables, possibly out of date
and incomplete, is in manual/maint.texi.) Rules and Makerules would
give errors if Makeconfig hadn't already been included, instead of
including it themselves. The special code to override values coming
from the environment would then be obsolete and could be removed.
Tested x86_64, including that installed binaries are identical before
and after the patch.
* argp/Makefile: Include Makeconfig immediately after defining
subdir.
* assert/Makefile: Likewise.
* benchtests/Makefile: Likewise.
* catgets/Makefile: Likewise.
* conform/Makefile: Likewise.
* crypt/Makefile: Likewise.
* csu/Makefile: Likewise.
(all): Remove target.
* ctype/Makefile: Include Makeconfig immediately after defining
subdir.
* debug/Makefile: Likewise.
* dirent/Makefile: Likewise.
* dlfcn/Makefile: Likewise.
* gmon/Makefile: Likewise.
* gnulib/Makefile: Likewise.
* grp/Makefile: Likewise.
* gshadow/Makefile: Likewise.
* hesiod/Makefile: Likewise.
* hurd/Makefile: Likewise.
(all): Remove target.
* iconvdata/Makefile: Include Makeconfig immediately after
defining subdir.
* inet/Makefile: Likewise.
* intl/Makefile: Likewise.
* io/Makefile: Likewise.
* libio/Makefile: Likewise.
(all): Remove target.
* locale/Makefile: Include Makeconfig immediately after defining
subdir.
* login/Makefile: Likewise.
* mach/Makefile: Likewise.
(all): Remove target.
* malloc/Makefile: Include Makeconfig immediately after defining
subdir.
(all): Remove target.
* manual/Makefile: Include Makeconfig immediately after defining
subdir.
* math/Makefile: Likewise.
* misc/Makefile: Likewise.
* nis/Makefile: Likewise.
* nss/Makefile: Likewise.
* po/Makefile: Likewise.
(all): Remove target.
* posix/Makefile: Include Makeconfig immediately after defining
subdir.
* pwd/Makefile: Likewise.
* resolv/Makefile: Likewise.
* resource/Makefile: Likewise.
* rt/Makefile: Likewise.
* setjmp/Makefile: Likewise.
* shadow/Makefile: Likewise.
* signal/Makefile: Likewise.
* socket/Makefile: Likewise.
* soft-fp/Makefile: Likewise.
* stdio-common/Makefile: Likewise.
* stdlib/Makefile: Likewise.
* streams/Makefile: Likewise.
* string/Makefile: Likewise.
* sunrpc/Makefile: Likewise.
(all): Remove target.
* sysvipc/Makefile: Include Makeconfig immediately after defining
subdir.
* termios/Makefile: Likewise.
* time/Makefile: Likewise.
* timezone/Makefile: Likewise.
(all): Remove target.
* wcsmbs/Makefile: Include Makeconfig immediately after defining
subdir.
* wctype/Makefile: Likewise.
libidn/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
localedata/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
(all): Remove target.
nptl/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
nptl_db/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
* crypt/Makefile: Move test targets after toplevel Rules
inclusion. Grab any necessary sysdep routines when linking.
* crypt/md5.c (md5_process_block): Remove define, we will always
name it __md5_process_block.
(md5_finish_ctx): Update md5_process_block call.
(md5_stream): Likewise.
(md5_process_bytes): Likewise.
(md5_process_block): Rename to __md5_process_block and move to ...
* crypt/md5-block.c: ... here.
* crypt/sha256.c (sha256_process_block): Move to ...
* crypt/sha256-block.c: ... here.
* crypt/sha512.c (sha512_process_block): Move to ...
* crypt/sha512-block.c: ... here.
* locale/Makefile (CFLAGS-md5.c): Define to add crypt/ to include
path.
* sysdeps/sparc/sparc-ifunc.c (sparc_libc_ifunc): Define.
* sysdeps/sparc/sparc64/multiarch/Makefile
(libcrypt-sysdep_routines): Add crypto assembler sysdeps when in
crypt subdir.
(localedef-aux): Add md5 crypto assembler when in locale subdir.
* sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile: Mirror sparc64
multiarch changes.
* sysdeps/sparc/sparc64/multiarch/md5-block.c: New file.
* sysdeps/sparc/sparc64/multiarch/md5-crop.S: New file.
* sysdeps/sparc/sparc64/multiarch/sha256-block.c: New file.
* sysdeps/sparc/sparc64/multiarch/sha256-crop.S: New file.
* sysdeps/sparc/sparc64/multiarch/sha512-block.c: New file.
* sysdeps/sparc/sparc64/multiarch/sha512-crop.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/md5-block.c: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/md5-crop.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha256-block.c: New
file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha256-crop.S: New file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha512-block.c: New
file.
* sysdeps/sparc/sparc32/sparcv9/multiarch/sha512-crop.S: New file.
instead of += to append, to be sure $(lib) gets expanded at defn time.
(libof-$(cpp-src)): New variable, define this instead.
* extra-lib.mk (cpp-srcs-left): Reduce duplication in include setup.
(lib): Don't use override.
(CPPFLAGS-$(lib)): New variable, put -D's here.
* Makeconfig (CPPFLAGS): Use basename fn for CPPFLAGS-basename.
Also add $(CPPFLAGS-LIB) before the file-specific flags, for each
LIB found by $(libof-*) for basename, target, or source.
* Makerules (CPPFLAGS-nonlib): New variable.
* nscd/Makefile (lib): Set to nonlib when using cppflags-iterator.mk.
* locale/Makefile (lib): Likewise.
* sunrpc/Makefile (lib): Likewise.
* sysdeps/unix/sysv/linux/fpathconf.c (LINUX_LINK_MAX): Move macro ...
* sysdeps/unix/sysv/linux/linux_fsinfo.h (LINUX_LINK_MAX): ... here.
* sysdeps/unix/sysv/linux/pathconf.h: New file.
(statfs_link_max): New function, guts from fpathconf.c.
* sysdeps/unix/sysv/linux/fpathconf.c: Rewritten using that.
* sysdeps/unix/sysv/linux/pathconf.c (__pathconf): Likewise.
* sysdeps/unix/sysv/linux/alpha/pathconf.c (__pathconf): Rewritten
to use the linux/pathconf.c code by #include rather than duplication.
* sysdeps/unix/sysv/linux/alpha/fpathconf.c (__pathconf): Likewise.
2002-10-16 Jakub Jelinek <jakub@redhat.com>
* sysdeps/unix/sysv/linux/x86_64/sysdep.h (SYSCALL_ERROR_HANDLER):
Use __libc_errno only for libc itself.
* locale/xlocale.c: Revert changes putting _nl_global_locale here.
This file again just defines _nl_C_locobj.
(_nl_C_locobj): Use a categories.def iterator in the initializer.
* locale/global-locale.c: New file. Define _nl_global_locale here,
using all weak references in the initializer.
* locale/Makefile (aux): Add global-locale.
* locale/localeinfo.h (_nl_global_locale, _NL_CURRENT_LOCALE): Make
these unconditional, along with the tsd decl.
[!SHARED && HAVE___THREAD && HAVE_WEAK_SYMBOLS] (NL_CURRENT_INDIRECT):
Define it under these conditions.
[NL_CURRENT_INDIRECT]: Test this instead of [! SHARED].
Don't declare _nl_current. Declare _nl_current_LC_FOO as
`extern __thread struct locale_data *const *'.
[NL_CURRENT_INDIRECT]
(_NL_CURRENT_DATA, _NL_CURRENT, _NL_CURRENT_WSTR): Add indirection.
[NL_CURRENT_INDIRECT] (_NL_CURRENT_DEFINE): Rewritten. Define
the thread variable _nl_current_LC_FOO and also a special absolute
symbol _nl_current_LC_FOO_used.
* locale/uselocale.c (__uselocale) [NL_CURRENT_INDIRECT]:
Set each _nl_current_LC_FOO symbol to point into the new locale,
using weak references to test if _nl_current_LC_FOO_used was linked in.
* locale/setlocale.c [! SHARED]: Replace this conditional ...
[NL_CURRENT_INDIRECT]: ... with this one.
(_nl_current, _nl_C): Variables removed.
[NL_CURRENT_INDIRECT] (_nl_current_used): New variable, table of
weak references to _nl_current_LC_FOO_used.
[NL_CURRENT_INDIRECT] (CATEGORY_USED): Define using that table.
(free_category): New function, broken out of ...
(free_mem): ... here. Call that.
(free_mem) [NL_CURRENT_INDIRECT]: Use a categories.def iterator
instead of a loop.
__USING_NAMESPACE_C99 depending on _GLIBCPP_USE_NAMESPACES.
