This was found by Coverity (CID 1484201). [BZ#24844]
* posix/regex_internal.c (create_cd_newstate): Fix use of bad
pointer and/or memory leak when storage is exhausted.
The fix for BZ#21270 (commit 158d5fa0e1) added a mask to avoid offset larger
than 1^44 to be used along __NR_mmap2. However mips64n32 users __NR_mmap,
as mips64n64, but still defines off_t as old non-LFS type (other ILP32, such
x32, defines off_t being equal to off64_t). This leads to use the same
mask meant only for __NR_mmap2 call for __NR_mmap, thus limiting the maximum
offset it can use with mmap64.
This patch fixes by setting the high mask only for __NR_mmap2 usage. The
posix/tst-mmap-offset.c already tests it and also fails for mips64n32. The
patch also change the test to check for an arch-specific header that defines
the maximum supported offset.
Checked on x86_64-linux-gnu, i686-linux-gnu, and I also tests tst-mmap-offset
on qemu simulated mips64 with kernel 3.2.0 kernel for both mips-linux-gnu and
mips64-n32-linux-gnu.
[BZ #24699]
* posix/tst-mmap-offset.c: Mention BZ #24699.
(do_test_bz21270): Rename to do_test_large_offset and use
mmap64_maximum_offset to check for maximum expected offset value.
* sysdeps/generic/mmap_info.h: New file.
* sysdeps/unix/sysv/linux/mips/mmap_info.h: Likewise.
* sysdeps/unix/sysv/linux/mmap64.c (MMAP_OFF_HIGH_MASK): Define iff
__NR_mmap2 is used.
Keep these functions compatible with Gnulib while adding
__time64_t support. The basic idea is to move private API
declarations from include/time.h to time/mktime-internal.h, since
the former file cannot easily be shared with Gnulib whereas the
latter can.
Also, do some other minor cleanup while in the neighborhood.
* include/time.h: Include stdbool.h, time/mktime-internal.h.
(__mktime_internal): Move this prototype to time/mktime-internal.h,
since Gnulib needs it.
(__localtime64_r, __gmtime64_r) [__TIMESIZE == 64]:
Move these macros to time/mktime-internal.h, since Gnulib needs them.
(__mktime64, __timegm64) [__TIMESIZE != 64]: New prototypes.
(in_time_t_range): New static function.
* posix/bits/types.h (__time64_t) [__TIMESIZE == 64 && !defined __LIBC]:
Do not define as a macro in this case, so that portable code is
less tempted to use __time64_t.
* time/mktime-internal.h: Rewrite so that it does both glibc
and Gnulib work. Include time.h if not _LIBC.
(mktime_offset_t) [!_LIBC]: Define for gnulib.
(__time64_t, __gmtime64_r, __localtime64_r, __mktime64, __timegm64)
[!_LIBC || __TIMESIZE == 64]: New macros, mostly moved here
from include/time.h.
(__gmtime_r, __localtime_r, __mktime_internal) [!_LIBC]:
New macros, taken from GNulib.
(__mktime_internal): New prototype, moved here from include/time.h.
* time/mktime.c (mktime_min, mktime_max, convert_time)
(ranged_convert, __mktime_internal, __mktime64):
* time/timegm.c (__timegm64):
Use __time64_t, not time_t.
* time/mktime.c: Stop worrying about whether time_t is floating-point.
(__mktime64) [! (_LIBC && __TIMESIZE != 64)]:
Rename from mktime.
(mktime) [_LIBC && __TIMESIZE != 64]: New function.
* time/timegm.c [!_LIBC]: Include libc-config.h, not config.h,
for libc_hidden_def.
Include errno.h.
(__timegm64) [! (_LIBC && __TIMESIZE != 64)]:
Rename from timegm.
(timegm) [_LIBC && __TIMESIZE != 64]: New function.
First cut at publicizing __time64_t
The stub implementations are turned into compat symbols.
Linux actually has two reserved system call numbers (for getpmsg
and putpmsg), but these system calls have never been implemented,
and there are no plans to implement them, so this patch replaces
the wrappers with the generic stubs.
According to <https://bugzilla.redhat.com/show_bug.cgi?id=436349>,
the presence of the XSI STREAMS declarations is a minor portability
hazard because they are not actually implemented.
This commit does not change the TIRPC support code in
sunrpc/rpc_svcout.c. It uses additional XTI functionality and
therefore never worked with glibc.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The test for obsolete typedefs in installed headers was implemented
using grep, and could therefore get false positives on e.g. “ulong”
in a comment. It was also scanning all of the headers included by
our headers, and therefore testing headers we don’t control, e.g.
Linux kernel headers.
This patch splits the obsolete-typedef test from
scripts/check-installed-headers.sh to a separate program,
scripts/check-obsolete-constructs.py. Being implemented in Python,
it is feasible to make it tokenize C accurately enough to avoid false
positives on the contents of comments and strings. It also only
examines $(headers) in each subdirectory--all the headers we install,
but not any external dependencies of those headers. Headers whose
installed name starts with finclude/ are ignored, on the assumption
that they contain Fortran.
It is also feasible to make the new test understand the difference
between _defining_ the obsolete typedefs and _using_ the obsolete
typedefs, which means posix/{bits,sys}/types.h no longer need to be
exempted. This uncovered an actual bug in bits/types.h: __quad_t and
__u_quad_t were being used to define __S64_TYPE, __U64_TYPE,
__SQUAD_TYPE and __UQUAD_TYPE. These are changed to __int64_t and
__uint64_t respectively. This is a safe change, despite the comments
in bits/types.h claiming a difference between __quad_t and __int64_t,
because those comments are incorrect. In all current ABIs, both
__quad_t and __int64_t are ‘long’ when ‘long’ is a 64-bit type, and
‘long long’ when ‘long’ is a 32-bit type, and similarly for __u_quad_t
and __uint64_t. (Changing the types to be what the comments say they
are would be an ABI break, as it affects C++ name mangling.) This
patch includes a minimal change to make the comments not completely
wrong.
sys/types.h was defining the legacy BSD u_intN_t typedefs using a
construct that was not necessarily consistent with how the C99 uintN_t
typedefs are defined, and is also too complicated for the new script to
understand (it lexes C relatively accurately, but it does not attempt
to expand preprocessor macros, nor does it do any actual parsing).
This patch cuts all of that out and uses bits/types.h's __uintN_t typedefs
to define u_intN_t instead. This is verified to not change the ABI on
any supported architecture, via the c++-types test, which means u_intN_t
and uintN_t were, in fact, consistent on all supported architectures.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* scripts/check-obsolete-constructs.py: New test script.
* scripts/check-installed-headers.sh: Remove tests for
obsolete typedefs, superseded by check-obsolete-constructs.py.
* Rules: Run scripts/check-obsolete-constructs.py over $(headers)
as a special test. Update commentary.
* posix/bits/types.h (__SQUAD_TYPE, __S64_TYPE): Define as __int64_t.
(__UQUAD_TYPE, __U64_TYPE): Define as __uint64_t.
Update commentary.
* posix/sys/types.h (__u_intN_t): Remove.
(u_int8_t): Typedef using __uint8_t.
(u_int16_t): Typedef using __uint16_t.
(u_int32_t): Typedef using __uint32_t.
(u_int64_t): Typedef using __uint64_t.
This patch adds fall-through comments in some cases where -Wextra
produces implicit-fallthrough warnings.
The patch is non-exhaustive. Apart from architecture-specific code
for non-x86_64 architectures, it does not change sunrpc/xdr.c (legacy
code, probably should have such changes, but left to be dealt with
separately), or places that already had comments about the
fall-through but not matching the form expected by
-Wimplicit-fallthrough=3 (the default level with -Wextra; my
inclination is to adjust those comments to match rather than
downgrading to -Wimplicit-fallthrough=1 to allow any comment), or one
place where I thought the implicit fallthrough was not correct and so
should be handled separately as a bug fix. I think the key thing to
consider in review of this patch is whether the fall-through is indeed
intended and correct in each place where such a comment is added.
Tested for x86_64.
* elf/dl-exception.c (_dl_exception_create_format): Add
fall-through comments.
* elf/ldconfig.c (parse_conf_include): Likewise.
* elf/rtld.c (print_statistics): Likewise.
* locale/programs/charmap.c (parse_charmap): Likewise.
* misc/mntent_r.c (__getmntent_r): Likewise.
* posix/wordexp.c (parse_arith): Likewise.
(parse_backtick): Likewise.
* resolv/ns_ttl.c (ns_parse_ttl): Likewise.
* sysdeps/x86/cpu-features.c (init_cpu_features): Likewise.
* sysdeps/x86_64/dl-machine.h (elf_machine_rela): Likewise.
Non-sysdeps headers cannot be overriden by sysdeps headers across the
entire build, so it is necessary to turn such extension headers into
sysdeps headers themselves. The approach here follows the existing
<bits/shm.h> header (although it uses sysdeps/gnu instead of
sysdeps/generic).
Fixes commit 1d0fc21382 ("Linux: Add
gettid system call wrapper [BZ #6399]") and commit
8f89ab216f ("posix: Fix missing wrapper
header for <bits/unistd_ext.h>").
This commit adds gettid to <unistd.h> on Linux, and not to the
kernel-independent GNU API.
gettid is now supportable on Linux because too many things assume a
1:1 mapping between libpthread threads and kernel threads.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This fixes the same bug in fnmatch that was fixed by commit 7e2f0d2d77 for
regexp matching. As a side effect it also removes the use of an unbound
VLA.
From time to time I get fails in tst-spawn like:
tst-spawn.c:111: numeric comparison failure
left: 0 (0x0); from: xlseek (fd2, 0, SEEK_CUR)
right: 28 (0x1c); from: strlen (fd2string)
error: 1 test failures
tst-spawn.c:252: numeric comparison failure
left: 1 (0x1); from: WEXITSTATUS (status)
right: 0 (0x0); from: 0
error: 1 test failures
It turned out, that a child process is testing it's open file descriptors
with e.g. a sequence of testing the current position, setting the position
to zero and reading a specific amount of bytes.
Unfortunately starting with commit 2a69f853c0
the test is spawning a second child process which is sharing some of the
file descriptors. If the test sequence as mentioned above is running in parallel
it leads to test failures.
As the second call of posix_spawn shall test a NULL pid argument,
this patch is just moving the waitpid of the first child
before the posix_spawn of the second child.
ChangeLog:
* posix/tst-spawn do_test(): Move waitpid before posix_spawn.
Problem found by AddressSanitizer, reported by Hongxu Chen in:
https://debbugs.gnu.org/34140
* posix/regexec.c (proceed_next_node):
Do not read past end of input buffer.
Austin Group issue #411 [1] proposes that posix_spawn file action
posix_spawn_file_actions_adddup2 resets the close-on-exec when
source and destination refer to same file descriptor.
It solves the issue on multi-thread applications which uses
close-on-exec as default, and want to hand-chose specifically
file descriptor to purposefully inherited into a child process.
Current approach to achieve this scenario is to use two adddup2 file
actions and a temporary file description which do not conflict with
any other, coupled with a close file action to avoid leaking the
temporary file descriptor. This approach, besides being complex,
may fail with EMFILE/ENFILE file descriptor exaustion.
This can be more easily accomplished with an in-place removal of
FD_CLOEXEC. Although the resulting adddup2 semantic is slight
different than dup2 (equal file descriptors should be handled as
no-op), the proposed possible solution are either more complex
(fcntl action which a limited set of operations) or results in
unrequired operations (dup3 which also returns EINVAL for same
file descriptor).
Checked on aarch64-linux-gnu.
[BZ #23640]
* posix/tst-spawn.c (do_prepare, handle_restart, do_test): Add
posix_spawn_file_actions_adddup2 test to check O_CLOCEXEC reset.
* sysdeps/unix/sysv/linux/spawni.c (__spawni_child): Add
close-on-exec reset for adddup2 file action.
* sysdeps/posix/spawni.c (__spawni_child): Likewise.
[1] http://austingroupbugs.net/view.php?id=411
From the glibc point of view, this removes duplicate macro
definitions and is obviously safe.
From the Gnulib point of view, this pacifies xlc 12.01 on AIX 7.1.
* posix/regex_internal.h:
(__attribute__, __attribute_warn_unused_result__):
Remove; already defined elsewhere.
This simplifies the code, by removing stuff intended for porting
to Gnulib but no longer needed there.
* posix/regcomp.c [!_LIBC]: No need to put #ifdef _LIBC around
uses of libc_hidden_def, weak_alias.
* posix/regcomp.c, posix/regexec.c: Use __restrict rather than
_Restrict_ except for public-facing headers.
* posix/regex_internal.h (attribute_hidden) [!_LIBC]:
Remove; already defined elsewhere.
* posix/regex.c, posix/regex_internal.h:
Use __GNUC_PREREQ instead of rolling our own.
* posix/regex_internal.h (__GNUC_PREREQ): Remove duplicate defn.
[BZ #18040]
Problem reported by Saito Takaaki <tails.saito@gmail.com> in
https://debbugs.gnu.org/32592
Call stack get_subexp->get_subexp_sub->clean_state_log_if_needed may
call extend_buffers which reallocates the re_string_t internal buffer.
Local variable 'buf' was not updated in such case, resulting in
use-after-free.
* posix/regexec.c (get_subexp): Update 'buf' after call to
get_subexp_sub.
Along with posix_spawn_file_actions_addchdir,
posix_spawn_file_actions_addfchdir is the subject of a change proposal
for POSIX: <http://austingroupbugs.net/view.php?id=1208>
These files were both auto-generated and shipped in the source tree.
We can assume that sed is available and always generate the files
during the build.
Building posix/bug-regex22.c fails with GCC mainline because of
-Wformat-overflow= warnings for NULL arguments to %s formats.
This is *not* testing how glibc handles such format arguments; in the
context of the messages in question it makes no sense to pass NULL to
such a %s format (the code passes s, inside "if (s == NULL)"). So
this patch changes the code not to pass such a format argument at all
(which means the string passed is constant, so no need to use printf
at all - however, there are two separate tests here with different
length arguments passed to re_compile_pattern, so it *does* make sense
to make the strings used different so that in the event of failure
it's clear which one of the tests failed).
Tested with build-many-glibcs.py with GCC mainline for
aarch64-linux-gnu.
* posix/bug-regex22.c (main): Use puts with distinct error
messages for unexpected success of re_compile_pattern, not printf
with NULL argument to %s.
glibc support for 64-bit time_t on 32-bit architectures
will involve:
- Using 64-bit times inside glibc, with conversions
to and from 32-bit times taking place as necessary
for interfaces using such times.
- Adding 64-bit-time support in the glibc public API.
This support should be dynamic, i.e. glibc should
provide both 32-bit and 64-bit implementations and
let user code choose at compile time whether to use
the 32-bit or 64-bit interfaces.
This requires a glibc-internal name for a type for times
that are always 64-bit.
Based on __TIMESIZE, a new macro is defined, __TIME64_T_TYPE,
which is always the right __*_T_TYPE to hold a 64-bit-time.
__TIME64_T_TYPE equals __TIME_T_TYPE if __TIMESIZE equals 64
and equals __SQUAD_T_TYPE otherwise.
__time64_t can then replace uses of internal_time_t.
This patch was tested by running 'make check' on branch
master then applying this patch and its predecessor and
running 'make check' again, and checking that both 'make
check' yield identical results. This was done on
x86_64-linux-gnu and i686-linux-gnu.
* bits/time64.h: New file.
* include/time.h: Replace internal_time_t with __time64_t.
* posix/bits/types (__time64_t): Add.
* stdlib/Makefile: Add bits/time64.h to includes.
* time/tzfile.c: Replace internal_time_t with __time64_t.
[BZ#23744]
This refactoring was prompted by a problem when the regex code is
used as part of Gnulib and when the builder’s compiler does not grok
__builtin_expect. Problem reported for Gawk by Nelson H.F. Beebe in:
https://lists.gnu.org/r/bug-gnulib/2018-09/msg00137.html
Although this refactoring does not fix the problem directly,
we might as well have Gawk use the now-preferred glibc style for when
__builtin_expect is unavailable.
* posix/regex_internal.h (BE): Remove.
All uses replaced by __glibc_unlikely or __glibc_likely.
Adjust the non-glibc code to agree with what Gawk needs for
rational range interpretation (RRI) for regular expression ranges.
In unibyte locales, Gawk wants ranges to use the underlying byte
rather than the character code point. This change does not affect
glibc proper.
* posix/regcomp.c (parse_byte) [!LIBC && RE_ENABLE_I18N]:
In unibyte locales, use the byte value rather than
running it through btowc.
Problem and fix reported by Assaf Gordon in:
https://lists.gnu.org/r/bug-gnulib/2018-07/txtqLKNwBdefE.txt
* posix/regcomp.c (free_charset) [!_LIBC]: Free range_starts and
range_ends members too, as they are defined in 'struct
re_charset_t' even if not _LIBC. This affects only Gnulib.
In commit 9479b6d5e0 we updated all of
the collation data to harmonize with the new version of ISO 14651
which is derived from Unicode 9.0.0. This collation update brought
with it some changes to locales which were not desirable by some
users, in particular it altered the meaning of the
locale-dependent-range regular expression, namely [a-z] and [A-Z], and
for en_US it caused uppercase letters to be matched by [a-z] for the
first time. The matching of uppercase letters by [a-z] is something
which is already known to users of other locales which have this
property, but this change could cause significant problems to en_US
and other similar locales that had never had this change before.
Whether this behaviour is desirable or not is contentious and GNU Awk
has this to say on the topic:
https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html
While the POSIX standard also has this further to say: "RE Bracket
Expression":
http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap09.html
"The current standard leaves unspecified the behavior of a range
expression outside the POSIX locale. ... As noted above, efforts were
made to resolve the differences, but no solution has been found that
would be specific enough to allow for portable software while not
invalidating existing implementations."
In glibc we implement the requirement of ISO POSIX-2:1993 and use
collation element order (CEO) to construct the range expression, the
API internally is __collseq_table_lookup(). The fact that we use CEO
and also have 4-level weights on each collation rule means that we can
in practice reorder the collation rules in iso14651_t1_common (the new
data) to provide consistent range expression resolution *and* the
weights should maintain the expected total order. Therefore this
patch does three things:
* Reorder the collation rules for the LATIN script in
iso14651_t1_common to deinterlace uppercase and lowercase letters in
the collation element orders.
* Adds new test data en_US.UTF-8.in for sort-test.sh which exercises
strcoll* and strxfrm* and ensures the ISO 14651 collation remains.
* Add back tests to tst-fnmatch.input and tst-regexloc.c which
exercise that [a-z] does not match A or Z.
The reordering of the ISO 14651 data is done in an entirely mechanical
fashion using the following program attached to the bug:
https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c28
It is up for discussion if the iso14651_t1_common data should be
refined further to have 3 very tight collation element ranges that
include only a-z, A-Z, and 0-9, which would implement the solution
sought after in:
https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c12
and implemented here:
https://www.sourceware.org/ml/libc-alpha/2018-07/msg00854.html
No regressions on x86_64.
Verified that removal of the iso14651_t1_common change causes tst-fnmatch
to regress with:
422: fnmatch ("[a-z]", "A", 0) = 0 (FAIL, expected FNM_NOMATCH) ***
...
425: fnmatch ("[A-Z]", "z", 0) = 0 (FAIL, expected FNM_NOMATCH) ***
This bug is very similar to bug 23036: The existing code assumed that
the length count included the length byte itself.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Various glibc testcases use tmpnam in ways subject to race conditions
(generate a temporary file name, then later open that file without
O_EXCL).
This patch fixes those tests to use mkstemp - generally a minimal
local fix to use mkstemp instead of tmpnam, rather than a larger fix
to use other testsuite infrastructure for temporary files. The
unchanged use of tmpnam in posix/wordexp-test.c would fail safe in the
event of a race (it's generating a name for use with mkdir rather than
for a file to be opened for writing).
Tested for x86_64.
* grp/tst_fgetgrent.c: Include <unistd.h>.
(main): Use mkstemp instead of tmpnam.
* io/test-utime.c (main): Likewise.
* posix/annexc.c (macrofile): Change to modifiable array.
(get_null_defines): Use mkstemp instead of tmpnam. Do not remove
macrofile here.
* posix/bug-getopt1.c: Include <stdlib.h>.
(do_test): Use mkstemp instead of tmpnam.
* posix/bug-getopt2.c: Include <stdlib.h>.
(do_test): Use mkstemp instead of tmpnam.
* posix/bug-getopt3.c: Include <stdlib.h>.
(do_test): Use mkstemp instead of tmpnam.
* posix/bug-getopt4.c: Include <stdlib.h>.
(do_test): Use mkstemp instead of tmpnam.
* posix/bug-getopt5.c: Include <stdlib.h>.
(do_test): Use mkstemp instead of tmpnam.
* stdio-common/bug7.c: Include <stdlib.h> and <unistd.h>.
(main): Use mkstemp instead of tmpnam.
* stdio-common/tst-fdopen.c: Include <stdlib.h>.
(main): Use mkstemp instead of tmpnam.
* stdio-common/tst-ungetc.c: Include <stdlib.h>.
(main): use mkstemp instead of tmpnam.
* stdlib/isomac.c (macrofile): Change to modifiable array.
(get_null_defines): Use mkstemp instead of tmpnam. Do not remove
macrofile here.
Each weight is prefixed by its length, and the length does not include
itself in the count. This can be seen clearly from the find_idx
function in string/strxfrm_l.c, for example. The old code behaved as if
the length itself counted, thus comparing an additional byte after the
weight, leading to spurious comparison failures and incorrect further
partitioning of character equivalence classes.
On some platforms the inclusion of regex-internal.h in bug-regex33
testcase show a MAX redefinition if test-skeleton.c is include later.
This patch fixes by removing regex-internal.h inclusion and using
SBC_MAX value directly.
Checked on aarch64-linux-gnu.
* posix/bug-regex33.c: Fix build after regex sync.
This patch syncs the regex implementation with gnulib (commit 0ee5212).
Only two changes in GLIBC regex testing are required:
1. posix/bug-regex28.c: as previously discussed [1] the change of
expected results on the pattern should be safe.
2. posix/PCRE.tests: the ERE (a)|\1 is malformed (in the sense that
the \1 doesn't mean anything) and although current GLIBC accepts
it has undefined behavior. This patch removes the specific test.
This sync contains some patches from thread 'Regex: Make libc regex
more usable outside GLIBC.' [2] which have been pushed upstream in
gnulib. This patches also fixes some regex issues (BZ #23233,
BZ #21163, BZ #18986, BZ #13762) and I did not add testcases for
both #23233 and #13762 because I couldn't think a simple way to
trigger the expected failure path to trigger them.
Checked on x86_64-linux-gnu and i686-linux-gnu.
[BZ #23233]
[BZ #21163]
[BZ #18986]
[BZ #13762]
* posix/Makefile (tests): Add bug-regex37 and bug-regex38.
* posix/PCRE.tests: Remove invalid test.
* posix/bug-regex28.c: Fix expected values for used syntax.
* posix/bug-regex37.c: New file.
* posix/bug-regex38.c: Likewise.
* posix/regcomp.c: Sync with gnulib.
* posix/regex.c: Likewise.
* posix/regex.h: Likewise.
* posix/regex_internal.c: Likewise.
* posix/regex_internal.h: Likewise.
* posix/regexec.c: Likewise.
[1] https://sourceware.org/ml/libc-alpha/2017-12/msg00807.html
[2] https://sourceware.org/ml/libc-alpha/2017-12/msg00237.html
This is a major rewrite of the description of 'crypt', 'getentropy',
and 'getrandom'.
A few highlights of the content changes:
- Throughout the manual, public headers, and user-visible messages,
I replaced the term "password" with "passphrase", the term
"password database" with "user database", and the term
"encrypt(ion)" with "(one-way) hashing" whenever it was applied to
passphrases. I didn't bother making this change in internal code
or tests. The use of the term "password" in ruserpass.c survives,
because that refers to a keyword in netrc files, but it is adjusted
to make this clearer.
There is a note in crypt.texi explaining that they were
traditionally called passwords but single words are not good enough
anymore, and a note in users.texi explaining that actual passphrase
hashes are found in a "shadow" database nowadays.
- There is a new short introduction to the "Cryptographic Functions"
section, explaining how we do not intend to be a general-purpose
cryptography library, and cautioning that there _are_, or have
been, legal restrictions on the use of cryptography in many
countries, without getting into any kind of detail that we can't
promise to keep up to date.
- I added more detail about what a "one-way function" is, and why
they are used to obscure passphrases for storage. I removed the
paragraph saying that systems not connected to a network need no
user authentication, because that's a pretty rare situation
nowadays. (It still says "sometimes it is necessary" to
authenticate the user, though.)
- I added documentation for all of the hash functions that glibc
actually supports, but not for the additional hash functions
supported by libxcrypt. If we're going to keep this manual section
around after the transition is more advanced, it would probably
make sense to add them then.
- There is much more detailed discussion of how to generate a salt,
and the failure behavior for crypt is documented. (Returning an
invalid hash on failure is what libxcrypt does; Solar Designer's
notes say that this was done "for compatibility with old programs
that assume crypt can never fail".)
- As far as I can tell, the header 'crypt.h' is entirely a GNU
invention, and never existed on any other Unix lineage. The
function 'crypt', however, was in Issue 1 of the SVID and is now
in the XSI component of POSIX. I tried to make all of the
@standards annotations consistent with this, but I'm not sure I got
them perfectly right.
- The genpass.c example has been improved to use getentropy instead
of the current time to generate the salt, and to use a SHA-256 hash
instead of MD5. It uses more random bytes than is strictly
necessary because I didn't want to complicate the code with proper
base64 encoding.
- The testpass.c example has three hardwired hashes now, to
demonstrate that different one-way functions produce different
hashes for the same input. It also demonstrates how DES hashing
only pays attention to the first eight characters of the input.
- There is new text explaining in more detail how a CSPRNG differs
from a regular random number generator, and how
getentropy/getrandom are not exactly a CSPRNG. I tried not to make
specific falsifiable claims here. I also tried to make the
blocking/cancellation/error behavior of both getentropy and
getrandom clearer.
The functions encrypt, setkey, encrypt_r, setkey_r, cbc_crypt,
ecb_crypt, and des_setparity should not be used in new programs,
because they use the DES block cipher, which is unacceptably weak by
modern standards. Demote all of them to compatibility symbols, and
remove their prototypes from installed headers. cbc_crypt, ecb_crypt,
and des_setparity were already compat symbols when glibc was
configured with --disable-obsolete-rpc.
POSIX requires encrypt and setkey to be available when _XOPEN_CRYPT
is defined, so this change also removes the definition of X_OPEN_CRYPT
from <unistd.h>.
The entire "DES Encryption" section is dropped from the manual, as is
the mention of AUTH_DES and FIPS 140-2 in the introduction to
crypt.texi. The documentation of 'memfrob' cross-referenced the DES
Encryption section, which is replaced by a hyperlink to libgcrypt, and
while I was in there I spruced up the actual documentation of
'memfrob' and 'strfry' a little. It's still fairly jokey, because
those functions _are_ jokes, but they do also have real use cases, so
people trying to use them for real should have all the information
they need.
DES-based authentication for Sun RPC is also insecure and should be
deprecated or even removed, but maybe that can be left as TI-RPC's
problem.
As noted in bug 13888, and as I noted previously in
<https://sourceware.org/ml/libc-alpha/2000-10/msg00111.html>, various
tests used hardcoded paths in /tmp, so posing issues for simultaneous
test runs from different build directories.
This patch fixes such uses of hardcoded file names to put them in the
build directory instead (in the case of stdio-common/bug5 the file
names are changed as well, to avoid a conflict with the name bug5.out
also used for the automatic test output redirection). It also fixes
test-installation.pl likewise (that was using filenames with $$ in
them rather than strictly hardcoded names, but that's still not good
practice for temporary file naming).
Note that my list of files changed is not identical to that in bug
13888. I added tst-spawn3.c and test-installation.pl, and removed
some tests that seem to me (now) to create temporary files securely
(simply using /tmp is not itself a problem if the temporary files are
handled properly with mkstemp; I haven't checked whether those tests
used to do things insecurely). conformtest is not changed because the
makefiles always pass a --tmpdir option so the /tmp default is
irrelevant, and for the same reason there is no actual problem with
nptl/tst-umask1.c because again the makefiles always override the
default.
nptl/sockperf.c is ignored because there is no code to run it;
probably that file should actually be removed.
Some tests use the mktemp function, but I think they all use it in a
way that *is* secure (for generating names for directories / sockets /
fifos / symlinks, where the operation using the name will not follow
symlinks and so there is no potential for a symlink attack on the
account running the testsuite).
Some tests use the tmpnam function to generate temporary file names.
This is in principle insecure, but not addressed by this patch (I
consider it a separate issue from the fully hardcoded paths).
Tested for x86_64.
[BZ #13888]
* posix/Makefile (CFLAGS-tst-spawn3.c): New variable.
* posix/tst-spawn3.c (do_test): Put tst-spwan3.pid in OBJPFX, not
/tmp.
* scripts/test-installation.pl: Put temporary files in build
directory, not /tmp.
* stdio-common/Makefile (CFLAGS-bug3.c): New variable.
(CFLAGS-bug4.c): Likewise.
(CFLAGS-bug5.c): Likewise.
(CFLAGS-test-fseek.c): Likewise.
(CFLAGS-test-popen.c): Likewise.
(CFLAGS-test_rdwr.c): Likewise.
* stdio-common/bug3.c (main): Put temporary file in OBJPFX, not
/tmp.
* stdio-common/bug4.c (main): Likewise.
* stdio-common/bug5.c (main): Likewise.
* stdio-common/test-fseek.c (TESTFILE): Likewise.
* stdio-common/test-popen.c (do_test): Likewise.
* stdio-common/test_rdwr.c (main): Likewise.
Neither the <dlfcn.h> entry points, nor lazy symbol resolution, nor
initial shared library load-up, are cancellation points, so ld.so
should exclusively use I/O primitives that are not cancellable. We
currently achieve this by having the cancellation hooks compile as
no-ops when IS_IN(rtld); this patch changes to using exclusively
_nocancel primitives in the source code instead, which makes the
intent clearer and significantly reduces the amount of code compiled
under IS_IN(rtld) as well as IS_IN(libc) -- in particular,
elf/Makefile no longer thinks we require a copy of unwind.c in
rtld-libc.a. (The older mechanism is preserved as a backstop.)
The bulk of the change is splitting up the files that define the
_nocancel I/O functions, so they don't also define the variants that
*are* cancellation points; after which, the existing logic for picking
out the bits of libc that need to be recompiled as part of ld.so Just
Works. I did this for all of the _nocancel functions, not just the
ones used by ld.so, for consistency.
fcntl was a little tricky because it's only a cancellation point for
certain opcodes (F_SETLKW(64), which can block), and the existing
__fcntl_nocancel wasn't applying the FCNTL_ADJUST_CMD hook, which
strikes me as asking for trouble, especially as the only nontrivial
definition of FCNTL_ADJUST_CMD (for powerpc64) changes F_*LK* opcodes.
To fix this, fcntl_common moves to fcntl_nocancel.c along with
__fcntl_nocancel, and changes its name to the extern (but hidden)
symbol __fcntl_nocancel_adjusted, so that regular fcntl can continue
calling it. __fcntl_nocancel now applies FCNTL_ADJUST_CMD; so that
both both fcntl.c and fcntl_nocancel.c can see it, the only nontrivial
definition moves from sysdeps/u/s/l/powerpc/powerpc64/fcntl.c to
.../powerpc64/sysdep.h and becomes entirely a macro, instead of a macro
that calls an inline function.
The nptl version of libpthread also changes a little, because its
"compat-routines" formerly included files that defined all the
_nocancel functions it uses; instead of continuing to duplicate them,
I exported the relevant ones from libc.so as GLIBC_PRIVATE. Since the
Linux fcntl.c calls a function defined by fcntl_nocancel.c, it can no
longer be used from libpthread.so; instead, introduce a custom
forwarder, pt-fcntl.c, and export __libc_fcntl from libc.so as
GLIBC_PRIVATE. The nios2-linux ABI doesn't include a copy of vfork()
in libpthread, and it was handling that by manipulating
libpthread-routines in .../linux/nios2/Makefile; it is cleaner to do
what other such ports do, and have a pt-vfork.S that defines no symbols.
Right now, it appears that Hurd does not implement _nocancel I/O, so
sysdeps/generic/not-cancel.h will forward everything back to the
regular functions. This changed the names of some of the functions
that sysdeps/mach/hurd/dl-sysdep.c needs to interpose.
* elf/dl-load.c, elf/dl-misc.c, elf/dl-profile.c, elf/rtld.c
* sysdeps/unix/sysv/linux/dl-sysdep.c
Include not-cancel.h. Use __close_nocancel instead of __close,
__open64_nocancel instead of __open, __read_nocancel instead of
__libc_read, and __write_nocancel instead of __libc_write.
* csu/check_fds.c (check_one_fd)
* sysdeps/posix/fdopendir.c (__fdopendir)
* sysdeps/posix/opendir.c (__alloc_dir): Use __fcntl_nocancel
instead of __fcntl and/or __libc_fcntl.
* sysdeps/unix/sysv/linux/pthread_setname.c (pthread_setname_np)
* sysdeps/unix/sysv/linux/pthread_getname.c (pthread_getname_np)
* sysdeps/unix/sysv/linux/i386/smp.h (is_smp_system):
Use __open64_nocancel instead of __open_nocancel.
* sysdeps/unix/sysv/linux/not-cancel.h: Move all of the
hidden_proto declarations to the end and issue them if either
IS_IN(libc) or IS_IN(rtld).
* sysdeps/unix/sysv/linux/Makefile [subdir=io] (sysdep_routines):
Add close_nocancel, fcntl_nocancel, nanosleep_nocancel,
open_nocancel, open64_nocancel, openat_nocancel, pause_nocancel,
read_nocancel, waitpid_nocancel, write_nocancel.
* io/Versions [GLIBC_PRIVATE]: Add __libc_fcntl,
__fcntl_nocancel, __open64_nocancel, __write_nocancel.
* posix/Versions: Add __nanosleep_nocancel, __pause_nocancel.
* nptl/pt-fcntl.c: New file.
* nptl/Makefile (pthread-compat-wrappers): Remove fcntl.
(libpthread-routines): Add pt-fcntl.
* include/fcntl.h (__fcntl_nocancel_adjusted): New function.
(__libc_fcntl): Remove attribute_hidden.
* sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Call
__fcntl_nocancel_adjusted, not fcntl_common.
(__fcntl_nocancel): Move to new file fcntl_nocancel.c.
(fcntl_common): Rename to __fcntl_nocancel_adjusted; also move
to fcntl_nocancel.c.
* sysdeps/unix/sysv/linux/fcntl_nocancel.c: New file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/fcntl.c: Remove file.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h:
Define FCNTL_ADJUST_CMD here, as a self-contained macro.
* sysdeps/unix/sysv/linux/close.c: Move __close_nocancel to...
* sysdeps/unix/sysv/linux/close_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/nanosleep.c: Move __nanosleep_nocancel to...
* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/open.c: Move __open_nocancel to...
* sysdeps/unix/sysv/linux/open_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/open64.c: Move __open64_nocancel to...
* sysdeps/unix/sysv/linux/open64_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/openat.c: Move __openat_nocancel to...
* sysdeps/unix/sysv/linux/openat_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/openat64.c: Move __openat64_nocancel to...
* sysdeps/unix/sysv/linux/openat64_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/pause.c: Move __pause_nocancel to...
* sysdeps/unix/sysv/linux/pause_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/read.c: Move __read_nocancel to...
* sysdeps/unix/sysv/linux/read_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/waitpid.c: Move __waitpid_nocancel to...
* sysdeps/unix/sysv/linux/waitpid_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/write.c: Move __write_nocancel to...
* sysdeps/unix/sysv/linux/write_nocancel.c: ...this new file.
* sysdeps/unix/sysv/linux/nios2/Makefile: Don't override
libpthread-routines.
* sysdeps/unix/sysv/linux/nios2/pt-vfork.S: New file which
defines nothing.
* sysdeps/mach/hurd/dl-sysdep.c: Define __read instead of
__libc_read, and __write instead of __libc_write. Define
__open64 in addition to __open.
Current posix_spawnp implementation wrongly tries to execute invalid
binaries (for instance script without shebang) as a shell script in
non compat mode. It was a regression introduced by
9ff72da471 when __spawni started to use
__execvpe instead of __execve (glibc __execvpe try to execute ENOEXEC
as shell script regardless).
This patch fixes it by using an internal symbol (__execvpex) with the
faulty semantic (since compat mode is handled by spawni.c itself).
It was reported by Daniel Drake on libc-help [1].
Checked on x86_64-linux-gnu and i686-linux-gnu.
[BZ #23264]
* include/unistd.h (__execvpex): New prototype.
* posix/Makefile (tests): Add tst-spawn4.
(tests-internal): Add tst-spawn4-compat.
* posix/execvpe.c (__execvpe_common, __execvpex): New functions.
* posix/tst-spawn4-compat.c: New file.
* posix/tst-spawn4.c: Likewise.
* sysdeps/unix/sysv/linux/spawni.c (__spawni): Do not interpret invalid
binaries as shell scripts.
* sysdeps/posix/spawni.c (__spawni): Likewise.
[1] https://sourceware.org/ml/libc-help/2018-06/msg00012.html
* bits/sched.h: Include <bits/types/struct_sched_param.h> and move struct
sched_param definition to it.
* sysdeps/unix/sysv/linux/bits/sched.h: Likewise.
* bits/types/struct_sched_param.h: New file.
* sysdeps/htl/bits/types/struct___pthread_attr.h: Include
<bits/types/struct_sched_param.h> instead of <sched.h>.
* posix/Makefile (headers): Add bits/types/struct_sched_param.h.
See:
http://pubs.opengroup.org/onlinepubs/7908799/xbd/re.html
> A range expression represents the set of collating elements that fall
> between two elements in the current collation sequence,
> inclusively. It is expressed as the starting point and the ending
> point separated by a hyphen (-).
>
> Range expressions must not be used in portable applications because
> their behaviour is dependent on the collating sequence. Ranges will be
> treated according to the current collating sequence, and include such
> characters that fall within the range based on that collating
> sequence, regardless of character values. This, however, means that
> the interpretation will differ depending on collating sequence. If,
> for instance, one collating sequence defines ä as a variant of a,
> while another defines it as a letter following z, then the expression
> [ä-z] is valid in the first language and invalid in the second.
Therefore, using [a-z] does not make much sense except in the C/POSIX locale.
The new iso14651_t1_common lists upper case and lower case Latin characters
in a different order than the old one which causes surprising results
for example in the de_DE locale: [a-z] now includes A because A comes
after a in iso14651_t1_common but does not include Z because that comes
after z in iso14651_t1_common.
* posix/tst-fnmatch.input: Fix results for range expressions
for non C locales.
* posix/tst-regexloc.c: Do not use a range expression for
de_DE.ISO-8859-1 locale.
This test case tests how many collating elements are defined in
da_DK.ISO-8859-1 locale. The da_DK locale source defines 4:
collating-element <A-A> from "<U0041><U0041>"
collating-element <A-a> from "<U0041><U0061>"
collating-element <a-A> from "<U0061><U0041>"
collating-element <a-a> from "<U0061><U0061>"
The new iso14651_t1_common file defines more collating elements, two
of them are in the ISO-8859-1 range:
collating-element <U004C_00B7> from "<U004C><U00B7>" % decomposition of LATIN CAPITAL LETTER L WITH MIDDLE DOT
collating-element <U006C_00B7> from "<U006C><U00B7>" % decomposition of LATIN SMALL LETTER L WITH MIDDLE DOT
So the total count is now 6 instead of 4.
* posix/bug-regex5.c: Fix test case because with the new
iso14651_t1_common file, the da_DK locale now has 6 collating elements
in the ISO-8859-1 range instead of 4 with the old iso14651_t1_common
file.
The tst-glob_lstat_compat test needs to run tests on the previous
version of glob. On alpha, there are three versions of glob, GLIBC_2.0,
GLIBC_2.1 and GLIBC_2.27, while on other architectures there are only
the GLIBC_2.0 and GLIBC_2.27 version. Therefore on alpha the previous
version is GLIBC_2.1 and not GLIBC_2.0.
Changelog:
[BZ #22818]
* posix/tst-glob_lstat_compat.c [__alpha__] (glob): Access
the GLIBC_2.1 version.
This completes the deprecation and removal of this inclusion, which
was begun in the 2.25 release.
* posix/sys/types.h: Don't include sys/sysmacros.h.
* misc/sys/sysmacros.h: Remove the conditional deprecation
warnings for the macros defined by this header.
As noted in bug 17979 (and as I noted earlier in
<https://sourceware.org/ml/libc-alpha/2012-02/msg00647.html>), uchar.h
has gratuitously complicated code to determine the types for char16_t
and char32_t, and to reject including that header for pre-C11
compilers not defining __CHAR16_TYPE__ and __CHAR32_TYPE__. Since
those types are always required to match uint_least16_t and
uint_least32_t, which glibc knows how to define without reference to
such predefined macros, it's safe just to define those types the same
as the *least* types are defined in stdint.h, so allowing the header
to work with (for example) GCC 4.3.
This patch implements that. bits/types.h is made to define
__int_leastN_t and __uint_leastN_t so the logic for those types can
stay in a single place, and stdint.h is made to use those __*_t to
define the public *_t types. uchar.h is then made to use
__uint_least16_t and __uint_least32_t to define char16_t and char32_t,
so simplifying the logic there. A new test is added that verifies the
types chosen for char16_t and char32_t do indeed match the types the
compiler uses for u"" and U"" string literals.
Tested for x86_64. (I have not tested with any of the older compilers
for which this would actually make a difference to whether you can
include uchar.h.)
[BZ #17979]
* posix/bits/types.h (__int_least8_t): New typedef.
(__uint_least8_t): Likewise.
(__int_least16_t): Likewise.
(__uint_least16_t): Likewise.
(__int_least32_t): Likewise.
(__uint_least32_t): Likewise.
(__int_least64_t): Likewise.
(__uint_least64_t): Likewise.
* sysdeps/generic/stdint.h (int_least8_t): Define using
__int_least8_t.
(int_least16_t): Define using __int_least16_t.
(int_least32_t): Define using __int_least32_t.
(int_least64_t): Define using __int_least64_t.
(uint_least8_t): Define using __uint_least8_t.
(uint_least16_t): Define using __uint_least16_t.
(uint_least32_t): Define using __uint_least32_t.
(uint_least64_t): Define using __uint_least64_t.
* wcsmbs/uchar.h: Include <bits/types.h>.
(char16_t): Define using __uint_least16_t conditional only on
[!__USE_ISOCXX11].
(char32_t): Define using __uint_least32_t conditional only on
[!__USE_ISOCXX11].
* wcsmbs/test-char-types.c: New file.
* wcsmbs/Makefile (tests): Add test-char-types.
Building with -Os produces linknamespace and localplt failures for,
among other functions, gnu_dev_major, gnu_dev_minor and
gnu_dev_makedev.
The issue is that those functions are not inlined when building with
-Os. While one could force them to be inlined in that case, it seems
more natural to fix this issue similarly to other namespace issues.
Thus, this patch makes gnu_dev_* into weak aliases for hidden symbols
__gnu_dev_*; __gnu_dev_* are then defined as inlines in the internal
include/sys/sysmacros.h, and uses of gnu_dev_* (often via the macros
major, minor and makedev) for which there are namespace issues are
changed to use __gnu_dev_*; where there are no namespace issues, use
of libc_hidden_proto serves to avoid unnecessary local PLT entry use.
Tested for x86_64, (a) without -Os, to verify the testsuite continues
to pass without problems and that the functions called under their new
names continue to be inlined as expected in that case; (b) with -Os,
to verify that the linknamespace and localplt failures in question go
away (but because of other such failures present, neither of the
relevant bugs can yet be closed).
[BZ #15105]
[BZ #19463]
* include/sys/sysmacros.h [!_ISOMAC]
(__SYSMACROS_NEED_IMPLEMENTATION): Define macro.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC]
(_SYS_SYSMACROS_H_WRAPPER): Likewise.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (gnu_dev_major): Use
libc_hidden_proto.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (gnu_dev_minor): Likewise.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (gnu_dev_makedev):
Likewise.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (__SYSMACROS_DECL_TEMPL):
Undefine and redefine to add use __gnu_dev_ prefix.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (__SYSMACROS_IMPL_TEMPL):
Likewise.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (__gnu_dev_major): Declare
and define as hidden inline function.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (__gnu_dev_minor):
Likewise.
[!_SYS_SYSMACROS_H_WRAPPER && !_ISOMAC] (__gnu_dev_makedev):
Likewise.
* misc/makedev.c (OUT_OF_LINE_IMPL_TEMPL): Use __gnu_dev_ prefix.
(gnu_dev_major): Use weak_alias and libc_hidden_weak.
(gnu_dev_minor): Likewise.
(gnu_dev_makedev): Likewise.
* csu/check_fds.c (check_one_fd): Use __gnu_dev_makedev instead of
makedev.
* posix/wordexp.c (exec_comm_child): Likewise.
* sysdeps/mach/hurd/xmknodat.c (__xmknodat): Use __gnu_dev_minor
instead of minor and __gnu_dev_major instead of major.
* sysdeps/unix/sysv/linux/device-nrs.h (DEV_TTY_P): Use
__gnu_dev_major instead of major.
* sysdeps/unix/sysv/linux/pathconf.c (distinguish_extX): Use
__gnu_dev_major instead of gnu_dev_major and __gnu_dev_minor
instead of gnu_dev_minor.
* sysdeps/unix/sysv/linux/ptsname.c (MASTER_P): Likewise.
(SLAVE_P): Likewise.
(__ptsname_internal): Use __gnu_dev_minor instead of minor.
* sysdeps/unix/sysv/linux/ttyname.h (is_pty): Use __gnu_dev_major
instead of major.
Bug 13575 reports that SSIZE_MAX is wrongly defined as LONG_MAX on
32-bit systems where ssize_t is defined as int (which is most 32-bit
systems supported by glibc).
This patch fixes the definition, using a conditional on
__WORDSIZE32_SIZE_ULONG to determine the appropriate type in the
32-bit case. Formally ssize_t need not be the signed type
corresponding to size_t, but as it is for all current glibc
configurations, there is no need for a new macro different from the
one used for defining SIZE_MAX. A testcase is added for both the type
and the value of SSIZE_MAX.
There is a relevant peculiarity in
sysdeps/unix/sysv/linux/s390/bits/typesizes.h:
/* Compatibility with g++ 2.95.x. */
/* size_t is unsigned long int on s390 -m31. */
This has the effect that for GCC 2 for s390, ssize_t does not match
__WORDSIZE32_SIZE_ULONG. I don't think such a conditional on the GCC
version makes sense - to have a well-defined ABI, the choices of
standard types should not depend on the GCC version. It's also the
case that upstream GCC 2.95 did not support s390, and glibc headers
don't in general try to support past development GCC versions - only
actual releases and current mainline development. But whether or not
that GCC 2 case should be removed (with or without a NEWS entry for
such a change), this patch does not result in any changes for s390;
the value is always still LONG_MAX in the s390 case because
__WORDSIZE32_SIZE_ULONG is always defined for 32-bit s390. I don't
think any such oddity in code only active for unofficial or unreleased
old compiler versions should block closing the present bug as fixed
once this patch is in.
Tested for x86_64 and x86, and with build-many-glibcs.py.
[BZ #13575]
* posix/bits/posix1_lim.h: Include <bits/wordsize.h>.
[!SSIZE_MAX && !(__WORDSIZE == 64 || __WORDSIZE32_SIZE_ULONG)]
(SSIZE_MAX): Define to INT_MAX.
* posix/test-ssize-max.c: New file.
* posix/Makefile (tests): Add test-ssize-max.
Bug 14553 reports that sys/types.h defines loff_t unconditionally,
despite it not being part of any supported standard. This is
permitted by the POSIX *_t reservation, but as a
quality-of-implementation issue it's still best not to define it
except for __USE_MISC. This patch conditions the definition
accordingly, updating a macro in sysdeps/unix/sysv/linux/sys/quota.h
to use __loff_t so it still works even if __USE_MISC is not defined.
codesearch.debian.net suggests there are quite a lot of loff_t uses
outside glibc, but it might well make sense to change all (few) uses
of loff_t or __loff_t inside glibc to use off64_t or __off64_t
instead, leaving only the definitions, treating this name as
obsolescent.
Tested for x86_64.
[BZ #14553]
* posix/sys/types.h (loff_t): Only define for [__USE_MISC].
* sysdeps/unix/sysv/linux/sys/quota.h (dqoff): Use __loff_t
instead of loff_t.
This patch increases timeouts on three tests I observed timing out on
slow systems.
* malloc/tst-malloc-tcache-leak.c (TIMEOUT): Define to 50.
* posix/tst-glob-tilde.c (TIMEOUT): Define to 200.
* resolv/tst-resolv-res_ninit.c (TIMEOUT): Define to 50.
Sync with gnulib 0e14f025d2.
Checked on x86_64-linux-gnu.
* lib/glob.c (glob): Use a 'char *', not a 'void *', in pointer
arithmetic.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Problem reported by Tim Rühsen [1]. Sync with gnulib 0e14f025d2.
[1] https://lists.gnu.org/archive/html/bug-gnulib/2017-10/msg00054.html
Checked on x86_64-linux-gnu.
* lib/glob.c (glob): Do not pass NULL to mempcpy.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The test posix/tst-glob_symlinks.c fails to build with GCC mainline:
tst-glob_symlinks.c: In function 'do_test':
tst-glob_symlinks.c:124:30: error: 'snprintf' output may be truncated before the last format character [-Werror=format-truncation=]
snprintf (buf, sizeof buf, "%s?", dangling_link);
^~~~~
tst-glob_symlinks.c:124:3: note: 'snprintf' output between 2 and 4097 bytes into a destination of size 4096
snprintf (buf, sizeof buf, "%s?", dangling_link);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tst-glob_symlinks.c:128:30: error: 'snprintf' output may be truncated before the last format character [-Werror=format-truncation=]
snprintf (buf, sizeof buf, "%s*", dangling_link);
^~~~~
tst-glob_symlinks.c:128:3: note: 'snprintf' output between 2 and 4097 bytes into a destination of size 4096
snprintf (buf, sizeof buf, "%s*", dangling_link);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This patch fixes the test to avoid such truncation warnings by
increasing the buffer in question by one byte, to ensure it can hold
any possible result of %s? or %s* formats where %s comes from a buffer
of size PATH_MAX.
Tested compilation with build-many-glibcs.py for aarch64-linux-gnu.
* posix/tst-glob_symlinks.c (do_test): Increase size of buf.
Three tests fail with a read-only source directory because they try to
write into the source directory. None of these write into it in a way
that should actually be problematic for concurrent builds sharing the
same writable source directory, but avoiding any writing into the
source directory (from testing, or from building glibc if the source
timestamps are properly ordered) is still a good idea, as being able
to build with read-only sources helps make sure there isn't anything
that could cause problems for concurrent builds.
This patch changes the tests in question to use either /tmp or the
build directory to write their temporary files (or to test O_TMPFILE,
as applicable).
Tested for x86_64.
* io/Makefile (tst-open-tmpfile-ARGS): New variable.
* posix/tst-mmap-offset.c (fname): Use /tmp.
* stdlib/tst-setcontext3.sh (tempfile): Use ${objpfx}.
These tests need a working Internet connection with DNS.
We have additional coverage of getaddrinfo through the resolv tests, so
the loss of default test coverage seems acceptable.
Hide internal __sched_setparam function to allow direct access within
libc.so and libc.a without using GOT nor PLT.
__GI___sched_setparam is defined when sysdeps/unix/syscalls.list is
used to generate sched_setparam. Otherwise libc_hidden_def is needed
explicitly.
[BZ #18822]
* include/sched.h (__sched_setparam): Add libc_hidden_proto.
* posix/sched_setp.c (__sched_setparam): Add libc_hidden_def.
Hide internal __posix_spawn_file_actions_realloc and /__spawni functions
to allow direct access within libc.so and libc.a without using GOT nor
PLT.
[BZ #18822]
* posix/spawn_int.h (__posix_spawn_file_actions_realloc): Add
attribute_hidden.
(__spawni): Likewise.
This patch follows commit 5554304f0 (posix: Allow glob to match dangling
symlinks [BZ #866]) by adding a compat symbol that follow previous
semantic of not following dangling symlinks and thus avoiding call
gl_lstat with GLOB_ALTDIRFUNC.
It avoids failure with old binaries that not set the alternate function
pointer for lstat (GNUmake for instance). The following scenario, for
instance, fails with current GNUmake because glibc will access unitialized
memory when calling gl_lstat:
$ cat src/t/t.c
int main ()
{
return 0;
}
$ cat Makefile
SRC = $(wildcard src/*/t.c)
OBJ = $(patsubst src/%.c, obj/%.o, $(SRC))
prog: $(OBJ)
$(CC) $(CFLAGS) $(LDFLAGS) $(LIBS) $(OBJ) -o prog
obj/%.o: src/%.c
$(CC) $(CFLAGS) -c $< -o $@
$ make
This works as expected with the patch applied. Since it is for generic
ABI, default compat symbols are added with override for Linux due LFS.
Now we have two compat symbols for glob on Linux:
1. sysdeps/unix/sysv/linux/oldglob.c which implements glob64 with
the old dirent layout. For this implementation I also set it to
not follow dangling symlinks (which is the safest path).
2. sysdeps/unix/sysv/linux/glob{64}-lstat-compat.c which implements
the compat symbol for dangling symlinks. As for generic glob,
the implementation uses XSTAT_IS_XSTAT64 to define whether
both __glob_lstat_compat and __glob64_lstat_compat should be
different implementations. For archictures that define
XSTAT_IS_XSTAT64, __glob_lstat_compat is aliased to
__glob64_lstat_compat.
3. sysdeps/unix/sysv/linux/alpha/oldglob.c with a different glob_t
layout. As for 1. this patch changes it to not follow dangling
symlinks.
The patch also bumps _GNU_GLOB_INTERFACE_VERSION to 2 to advertise the
new semantic. On GNUmake, for instance, it will force to it use its
internal glob implementation instead and avoiding triggering the same
failure on builds against newer GLIBCs.
Checked on x86_64-linux-gnu and i686-linux-gnu. I also checked
with a build against the major ABIs required to check for the abilist.
The changes should also work on gnulib (I run gnulib-tool.py check glob
and it shown no regressions).
[BZ #22183]
* include/gnu-versions.h (_GNU_GLOB_INTERFACE_VERSION): Increase
version to 2.
* posix/Makefile (routines): Add glob-lstat-compat and
glob64-lstat-compat.
* posix/Versions (GLIBC_2.27, glob, glob64): Add symbol version.
* posix/glob-lstat-compat.c: New file.
* posix/glob64-lstat-compat.c: Likewise.
* posix/tst-glob_lstat_compat.c: Likewise.
* sysdeps/unix/sysv/linux/glob-lstat-compat.c: Likewise.
* sysdeps/unix/sysv/linux/alpha/glob-lstat-compat.c: Likewise.
* sysdeps/unix/sysv/linux/glob64-lstat-compat.c: Likewise.
* sysdeps/unix/sysv/linux/alpha/glob.c: Remove file.
* posix/glob.c (glob_lstat): New function.
(glob): Rename to __glob and add versioned symbol to 2.27.
(glob_in_dir): Use glob_lstat.
* posix/glob64.c (glob64): Add GLOB_ATTRIBUTE.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/glob.c (glob): Add versioned symbol for
2.27.
* sysdeps/unix/sysv/linux/glob64.c (glob64): Likewise.
* sysdeps/unix/sysv/linux/oldglob.c (GLOB_NO_LSTAT): Define.
* sysdeps/unix/sysv/linux/alpha/oldglob.c (__old_glob): Do not use
gl_lstat on glob call.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Add GLIBC_2.27 glob
and glob64 symbols.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/linux/powerpc/powerpc32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
This patch replaces the internal integer constant from
__glob_pattern_type return with a proper enum.
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
* posix/glob_internal.h (glob_pattern_type_t): New enumeration.
(__glob_pattern_type): Use __glob_pat_types.
* posix/glob_pattern_p.c (__glob_pattern_p): Likewise.
* posix/glob.c (glob): Likewise.
(glob_in_dir): Likewise.
Current glob implementation allows unlimited user name for home
directory construction on GLOB_TILDE case. To accomplish it glob
either construct a name on stack if size are small enough (based
on current alloca_used) or in heap otherwise.
This patch simplifies storage allocation by using the same scratch
buffer for both get_rlogin_r and getpwnam_r.
This also syncs with gnulib commit 064df0b (glob: fix bugs with long
login names).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
* posix/glob.c (GET_LOGIN_NAME_MAX): Remove.
(glob): Use the same scratch buffer for both getlogin_r and
getpwnam_r. Don’t require preallocation of the login name. This
simplifies storage allocation, and corrects the handling of
long login names.
This patch fixes longstanding misuse of errno after getpwnam_r,
which returns an error number rather than setting errno. This is
sync with gnulib commit 5db9301.
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #1062]
* posix/glob.c (glob): Port recent patches to platforms
lacking getpwnam_r.
(glob): Fix longstanding misuse of errno after getpwnam_r, which
returns an error number rather than setting errno.
This patch makes glob match dangling symlinks. Compared to other glob
implementation (*BSD, bash, musl, and other shells as well), GLIBC seems
the be the only one that does not match dangling symlinks. As for
comment #5 in BZ #866, POSIX does not have any strict specification for
dangling symlinks match and it is reasonable that trying to glob everything
in a path should return all types of files (such as for a 'rm *'). Also,
comment #7 shows even more example where GLIBC current behavior is
unexepected.
I avoided adding another GNU specific flag to set this behavior and
instead make it the default. Although this change the semanthic from
previous implementation, I think adding another compat symbol to be
really unecessary as from aforementioned reasons (current behavior not
defined in any standard, general idea of different implementation is
to list dangling symbols).
This also sync glob with gnulib commit fd1daf4 (glob: match dangling
symlinks).
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #866]
[BZ #1062]
* posix/Makefile (tests): Remove bug-glob1 and tst-glob_symlinks.
* posix/bug-glob1.c: Remove file.
* posix/tst-glob_symlinks.c: New file.
* posix/glob.c (__lstat64): New macro.
(is_dir): New function.
(glob, glob_in_dir): Match symlinks even if they are dangling.
(link_stat, link_exists_p): Remove. All uses removed.
According to this kernel commit 2adc376c55194, d_ino 0 is a regular inode
number on Linux (which also matches POSIX, as it does not treat the value
as special). This patch makes glob accept is a valid inode number.
This is also a sync with gnulib commit c8e57c1.
Checked on x86_64-linux-gnu and on a build using build-many-glibcs.py
for all major architectures.
[BZ #1062]
[BZ #19971]
* posix/glob.c (struct readdir_result): Remove skip_entry member.
(readdir_result_skip_entry, D_INO_TO_RESULT): Remove.
All uses removed.
