Commit c402355dfa ("libio: Disable
vtable validation in case of interposition [BZ #23313]") only covered
the interposable glibc 2.1 handles, in libio/stdfiles.c. The
parallel code in libio/oldstdfiles.c needs similar detection logic.
Fixes (again) commit db3476aff1
("libio: Implement vtable verification [BZ #20191]").
Change-Id: Ief6f9f17e91d1f7263421c56a7dc018f4f595c21
When the commit
commit 03992356e6
Author: Zack Weinberg <zackw@panix.com>
Date: Sat Feb 10 11:58:35 2018 -0500
Use C99-compliant scanf under _GNU_SOURCE with modern compilers.
added the DEPRECATED_SCANF macro to select when redirections of *scanf
functions to their ISO C99 compliant versions should happen, it
accidentally missed doing it for vfwscanf, vwscanf, and vswscanf.
Tested for powerpc64le and with build-many-glibcs (i686-linux-gnu and
nios2-linux-gnu are failing with current master, and with this patch,
but I didn't see a regression).
Change-Id: I706b344a3fb50be017cdab9251d9da18a3ba8c60
libio can only deal with gconv conversions which consist of a single
step. Not using __gconv_info simplifies the data structures somewhat.
This eliminates a new GCC 10 warning about subscribing an inner
zero-length array.
Tested on x86_64-linux-gnu with mainline GCC. Built with
build-many-glibcs.py, also with mainline GCC. Due to GCC PR 92039,
there are failures left on 32-bit architectures with float128 support.
Change-Id: I8b4c489b619a53154712ff32e1b6f13bb92d4203
The changes introduce a memory leak for gconv steps arrays whose
first element is an internal conversion, which has a fixed
reference count which is not decremented. As a result, after the
change in commit 50ce3eae5b, the steps
array is never freed, resulting in an unbounded memory leak.
This reverts commit 50ce3eae5b
("gconv: Check reference count in __gconv_release_cache
[BZ #24677]") and commit 7e740ab2e7
("libio: Fix gconv-related memory leak [BZ #24583]"). It
reintroduces bug 24583. (Bug 24677 was just a regression caused by
the second commit.)
On powerpc64le, long double can currently take two formats: the same as
double (-mlong-double-64) or IBM Extended Precision (default with
-mlong-double-128 or explicitly with -mabi=ibmlongdouble). The internal
implementation of printf-like functions is aware of these possibilities
and properly parses floating-point values from the variable arguments,
before making calls to __printf_fp and __printf_fphex. These functions
are also aware of the format possibilities and know how to convert both
formats to string.
When library support for TS 18661-3 was added to glibc, __printf_fp and
__printf_fphex were extended with support for an additional type
(__float128/_Float128) with a different format (binary128). Now that
powerpc64le is getting support for its third long double format, and
taking into account that this format is the same as the format of
__float128/_Float128, this patch extends __vfprintf_internal to properly
call __printf_fp and __printf_fphex with this new format.
Tested for powerpc64le (with additional patches to actually enable the
use of these preparations) and for x86_64.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Commit a601b74d31 aka glibc-2.23~693
("In preparation for fixing BZ#16734, fix failure in misc/tst-error1-mem
when _G_HAVE_MMAP is turned off.") introduced a regression:
_IO_unbuffer_all now invokes _IO_wsetb to free wide buffers of all
files, including legacy standard files which are small statically
allocated objects that do not have wide buffers and the _mode member,
causing memory corruption.
Another memory corruption in _IO_unbuffer_all happens when -1
is assigned to the _mode member of legacy standard files that
do not have it.
[BZ #24228]
* libio/genops.c (_IO_unbuffer_all)
[SHLIB_COMPAT (libc, GLIBC_2_0, GLIBC_2_1)]: Do not attempt to free wide
buffers and access _IO_FILE_complete members of legacy libio streams.
* libio/tst-bz24228.c: New file.
* libio/tst-bz24228.map: Likewise.
* libio/Makefile [build-shared] (tests): Add tst-bz24228.
[build-shared] (generated): Add tst-bz24228.mtrace and
tst-bz24228.check.
[run-built-tests && build-shared] (tests-special): Add
$(objpfx)tst-bz24228-mem.out.
(LDFLAGS-tst-bz24228, tst-bz24228-ENV): New variables.
($(objpfx)tst-bz24228-mem.out): New rule.
struct gconv_fcts for the C locale is statically allocated,
and __gconv_close_transform deallocates the steps object.
Therefore this commit introduces __wcsmbs_close_conv to avoid
freeing the statically allocated steps objects.
The codecvt vtable is not a real vtable because it also contains the
conversion state data. Furthermore, wide stream support was added to
GCC 3.0, after a C++ ABI bump, so there is no compatibility
requirement with libstdc++.
This change removes several unmangled function pointers which could
be used with a corrupted FILE object to redirect execution. (libio
vtable verification did not cover the codecvt vtable.)
Reviewed-by: Yann Droneaud <ydroneaud@opteya.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
When computing the length of the converted part of the stdio buffer, use
the number of consumed wide characters, not the (negative) distance to the
end of the wide buffer.
These variables are only used to determine if a stdio stream is
a pre-allocated stream, but it is possible to do so by comparing
a FILE * to all pre-allocated stream objects. As a result, it is
not necessary to keep those pointers in separate variables.
Behavior with symbol interposition is unchanged because _IO_stdin_,
_IO_stdout_, _IO_stderr_ are exported, and refer to objects outside of
libc if symbol interposition or copy relocations are involved. (The
removed variables _IO_stdin, _IO_stdout, _IO_stderr were not exported,
of course.)
One group of warnings seen building glibc with -Wextra is -Wempty-body
warnings about an 'if' body (or in one case an 'else' body) that is
just a semicolon, "warning: suggest braces around empty body in an
'if' statement [-Wempty-body]" - I think the point of the warning
being to make it more visible whether an 'if' body is actually present
or not.
This patch fixes such warnings in glibc. There's one place, with a
semicolon at the end of a comment, where this is clearly making the
presence of an 'else' body more visible. The other cases involve
macro definitions expanding to nothing. While there's no issue there
with visibility at the call sites, I think it's still cleaner to have
a macro that expands to something nonempty appropriate for the context
- so do {} while (0) if it's only intended to be usable as a
statement, or ((void) 0) where the macro definition is an alternative
to a call to a function returning void, so this patch makes those
changes.
Tested for x86_64.
* catgets/gencat.c (normalize_line): Use braces around empty
'else' body.
* include/stap-probe.h [!USE_STAP_PROBE && !__ASSEMBLER__]
(STAP_PROBE0): Use do {} while (0) for do-nothing definition.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE1): Likewise.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE2): Likewise.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE3): Likewise.
[!USE_STAP_PROBE && !__ASSEMBLER__] (STAP_PROBE4): Likewise.
* libio/libio.h (_IO_funlockfile): Use ((void) 0) for do-nothing
definition.
On big endian systems the test fails with:
tst-bz24153.c:88: numeric comparison failure
left: 1660944385 (0x63000001); from: ch
right: 99 (0x63); from: L'c'
tst-bz24153.c:90: numeric comparison failure
left: 1677721601 (0x64000001); from: ch
right: 100 (0x64); from: L'd'
error: 2 test failures
One 'char' ("%c") is stored to the 'wchar_t *':
ch = 0x00000001 | 0x63000000
This patch is using "%lc" as format specifier to read a wchar_t.
ChangeLog:
* libio/tst-bz24153.c (wide): Use wide char format specifier.
GLIBC explicitly allows one to assign a new FILE pointer to stdout and
other standard streams. printf and wprintf were honouring assignment to
stdout and using the new value, but puts, putchar, and wide char variants
did not.
The stdout part is fixed here. The stdin part will be fixed in a followup.
The only difference between noncompliant and C99-compliant scanf is
that the former accepts the archaic GNU extension '%as' (also %aS and
%a[...]) meaning to allocate space for the input string with malloc.
This extension conflicts with C99's use of %a as a format _type_
meaning to read a floating-point number; POSIX.1-2008 standardized
equivalent functionality using the modifier letter 'm' instead (%ms,
%mS, %m[...]).
The extension was already disabled in most conformance modes:
specifically, any mode that doesn't involve _GNU_SOURCE and _does_
involve either strict conformance to C99 or loose conformance to both
C99 and POSIX.1-2001 would get the C99-compliant scanf. With
compilers new enough to use -std=gnu11 instead of -std=gnu89, or
equivalent, that includes the default mode.
With this patch, we now provide C99-compliant scanf in all
configurations except when _GNU_SOURCE is defined *and*
__STDC_VERSION__ or __cplusplus (whichever is relevant) indicates
C89/C++98. This leaves the old scanf available under e.g. -std=c89
-D_GNU_SOURCE, but removes it from e.g. -std=gnu11 -D_GNU_SOURCE (it
was already not present under -std=gnu11 without -D_GNU_SOURCE) and
from -std=gnu89 without -D_GNU_SOURCE.
There needs to be an internal override so we can compile the
noncompliant scanf itself. This is the same problem we had when we
removed 'gets' from _GNU_SOURCE and it's dealt with the same way:
there's a new __GLIBC_USE symbol, DEPRECATED_SCANF, which defaults to
off under the appropriate conditions for external code, but can be
overridden by individual files within stdio.
We also run into problems with PLT bypass for internal uses of sscanf,
because libc_hidden_proto uses __REDIRECT and so does the logic in
stdio.h for choosing which implementation of scanf to use; __REDIRECT
isn't transitive, so include/stdio.h needs to bridge the gap with a
macro. As far as I can tell, sscanf is the only function in this
family that's internally called by unrelated code.
Finally, there are several tests in stdio-common that use the
extension. bug21.c is a regression test for a crash; it still
exercises the relevant code when changed to use %ms instead of %as.
scanf14.c through scanf17.c are more complicated since they are
actually testing the subtleties of the extension - under what
circumstances is 'a' treated as a modifier letter, etc. I changed all
of them to use %ms instead of %as as well, but duplicated scanf14.c
and scanf16.c as scanf14a.c and scanf16a.c. These still use %as and
are compiled with -std=gnu89 to access the old extension. A bunch of
diagnostic overrides and manual workarounds for the old stdio.h
behavior become unnecessary. Yay!
* include/features.h (__GLIBC_USE_DEPRECATED_SCANF): New __GLIBC_USE
parameter. Only use deprecated scanf when __USE_GNU is defined
and __STDC_VERSION__ is less than 199901L or __cplusplus is less
than 201103L, whichever is relevant for the language being compiled.
* libio/stdio.h, libio/bits/stdio-ldbl.h: Decide whether to redirect
scanf, fscanf, sscanf, vscanf, vfscanf, and vsscanf to their
__isoc99_ variants based only on __GLIBC_USE (DEPRECATED_SCANF).
* wcsmbs/wchar.h: wcsmbs/bits/wchar-ldbl.h: Likewise for
wscanf, fwscanf, swscanf, vwscanf, vfwscanf, and vswscanf.
* libio/iovsscanf.c
* libio/fwscanf.c
* libio/iovswscanf.c
* libio/swscanf.c
* libio/vscanf.c
* libio/vwscanf.c
* libio/wscanf.c
* stdio-common/fscanf.c
* stdio-common/scanf.c
* stdio-common/vfscanf.c
* stdio-common/vfwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-compat.c
* sysdeps/ieee754/ldbl-opt/nldbl-fscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-fwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-iovfscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-scanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-sscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-swscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vfscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vfwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vsscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vswscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-vwscanf.c
* sysdeps/ieee754/ldbl-opt/nldbl-wscanf.c:
Override __GLIBC_USE_DEPRECATED_SCANF to 1.
* stdio-common/sscanf.c: Likewise. Remove ldbl_hidden_def for __sscanf.
* stdio-common/isoc99_sscanf.c: Add libc_hidden_def for __isoc99_sscanf.
* include/stdio.h: Provide libc_hidden_proto for __isoc99_sscanf,
not sscanf.
[!__GLIBC_USE (DEPRECATED_SCANF)]: Define sscanf as __isoc99_scanf
with a preprocessor macro.
* stdio-common/bug21.c, stdio-common/scanf14.c:
Use %ms instead of %as, %mS instead of %aS, %m[] instead of %a[];
remove DIAG_IGNORE_NEEDS_COMMENT for -Wformat.
* stdio-common/scanf16.c: Likewise. Add __attribute__ ((format (scanf)))
to xscanf, xfscanf, xsscanf.
* stdio-common/scanf14a.c: New copy of scanf14.c which still uses
%as, %aS, %a[]. Remove DIAG_IGNORE_NEEDS_COMMENT for -Wformat.
* stdio-common/scanf16a.c: New copy of scanf16.c which still uses
%as, %aS, %a[]. Add __attribute__ ((format (scanf))) to xscanf,
xfscanf, xsscanf.
* stdio-common/scanf15.c, stdio-common/scanf17.c: No need to
override feature selection macros or provide definitions of u_char etc.
* stdio-common/Makefile (tests): Add scanf14a and scanf16a.
(CFLAGS-scanf15.c, CFLAGS-scanf17.c): Remove.
(CFLAGS-scanf14a.c, CFLAGS-scanf16a.c): New. Compile these files
with -std=gnu89.
According to ISO C99, passing the same buffer as source and destination
to sprintf, snprintf, vsprintf, or vsnprintf has undefined behavior.
Until the commit
commit 4e2f43f842
Author: Zack Weinberg <zackw@panix.com>
Date: Wed Mar 7 14:32:03 2018 -0500
Use PRINTF_FORTIFY instead of _IO_FLAGS2_FORTIFY (bug 11319)
a call to sprintf or vsprintf with overlapping buffers, for instance
vsprintf (buf, "%sTEXT", buf), would append `TEXT' into buf, while a
call to snprintf or vsnprintf would override the contents of buf.
After the aforementioned commit, the behavior of sprintf and vsprintf
changed (so that they also override the contents of buf).
This patch reverts this behavioral change, because it will likely break
applications that rely on the previous behavior, even though it is
undefined by ISO C. As noted by Szabolcs Nagy, this is used in SPEC2017
507.cactuBSSN_r/src/PUGH/PughUtils.c:
sprintf(mess," Size:");
for (i=0;i<dim+1;i++)
{
sprintf(mess,"%s %d",mess,pughGH->GFExtras[dim]->nsize[i]);
}
More important to notice is the fact that the overwriting of the
destination buffer is not the only behavior affected by the refactoring.
Before the refactoring, sprintf and vsprintf would use _IO_str_jumps,
whereas __sprintf_chk and __vsprintf_chk would use _IO_str_chk_jumps.
After the refactoring, all use _IO_str_chk_jumps, which would make
sprintf and vsprintf report buffer overflows and terminate the program.
This patch also reverts this behavior, by installing the appropriate
jump table for each *sprintf functions.
Apart from reverting the changes, this patch adds a test case that has
the old behavior hardcoded, so that regressions are noticed if something
else unintentionally changes the behavior.
Tested for powerpc64le.
Since the commit
commit 698fb75b9f
Author: Zack Weinberg <zackw@panix.com>
Date: Wed Mar 7 14:32:01 2018 -0500
Add __v*printf_internal with flags arguments
_IO_vfprintf is gone. This did not trigger any test case failures on
powerpc and powerpc64le, because there were no tests that covered it.
However, new test cases for nldbl versions of argp.h functions exposed
the problem.
Tested for powerpc64 and powerpc64le.
On powerpc64le, long double can currently take two formats: the same as
double (-mlong-double-64) or IBM Extended Precision (default with
-mlong-double-128 or explicitly with -mabi=ibmlongdouble). The internal
implementation of scanf-like functions is aware of these possibilites
and, based on the format in use, properly calls __strtold_internal or
__strtod_internal, saving the return to a variable of type double or
long double.
When library support for TS 18661-3 was added to glibc, a new function,
__strtof128_internal, was added to enable reading of floating-point
values with IEEE binary128 format into the _Float128 type. Now that
powerpc64le is getting support for its third long double format, and
taking into account that this format is the same as the format of
_Float128, this patch extends __vfscanf_internal and __vfwscanf_internal
to call __strtof128_internal or __wcstof128_internal when appropriate.
The result gets saved into a variable of _Float128 type.
Tested for powerpc64le.
The _chk variants of all of the printf functions become much simpler.
This is the last thing that we needed _IO_acquire_lock_clear_flags2
for, so it can go as well. I took the opportunity to make the headers
included and the names of all local variables consistent across all the
affected files.
Since we ultimately want to get rid of __no_long_double as well, it
must be possible to get all of the nontrivial effects of the _chk
functions by calling the _internal functions with appropriate flags.
For most of the __(v)xprintf_chk functions, this is covered by
PRINTF_FORTIFY plus some up-front argument checks that can be
duplicated. However, __(v)sprintf_chk installs a custom jump table so
that it can crash instead of overflowing the output buffer. This
functionality is moved to __vsprintf_internal, which now has a
'maxlen' argument like __vsnprintf_internal; to get the unsafe
behavior of ordinary (v)sprintf, pass -1 for that argument.
obstack_printf_chk and obstack_vprintf_chk are no longer in the same
file.
As a side-effect of the unification of both fortified and non-fortified
vdprintf initialization, this patch fixes bug 11319 for __dprintf_chk
and __vdprintf_chk, which was previously fixed only for dprintf and
vdprintf by the commit
commit 7ca890b88e
Author: Ulrich Drepper <drepper@redhat.com>
Date: Wed Feb 24 16:07:57 2010 -0800
Fix reporting of I/O errors in *dprintf functions.
This patch adds a test case to avoid regressions.
Tested for powerpc and powerpc64le.
There are a lot more printf variants than there are scanf variants,
and the code for setting up and tearing down their custom FILE
variants around the call to __vf(w)printf is more complicated and
variable. Therefore, I have added _internal versions of all the
v*printf variants, rather than introducing helper routines so that
they can all directly call __vf(w)printf_internal, as was done with
scanf.
As with the scanf changes, in this patch the _internal functions still
look at the environmental mode bits and all callers pass 0 for the
flags parameter.
Several of the affected public functions had _IO_ name aliases that
were not exported (but, in one case, appeared in libio.h anyway);
I was originally planning to leave them as aliases to avoid having
to touch internal callers, but it turns out ldbl_*_alias only work
for exported symbols, so they've all been removed instead. It also
turns out there were hardly any internal callers. _IO_vsprintf and
_IO_vfprintf *are* exported, so those two stick around.
Summary for the changes to each of the affected symbols:
_IO_vfprintf, _IO_vsprintf:
All internal calls removed, thus the internal declarations, as well
as uses of libc_hidden_proto and libc_hidden_def, were also removed.
The external symbol is now exposed via uses of ldbl_strong_alias
to __vfprintf_internal and __vsprintf_internal, respectively.
_IO_vasprintf, _IO_vdprintf, _IO_vsnprintf,
_IO_vfwprintf, _IO_vswprintf,
_IO_obstack_vprintf, _IO_obstack_printf:
All internal calls removed, thus declaration in internal headers
were also removed. They were never exported, so there are no
aliases tying them to the internal functions. I.e.: entirely gone.
__vsnprintf:
Internal calls were always preceded by macros such as
#define __vsnprintf _IO_vsnprintf, and
#define __vsnprintf vsnprintf
The macros were removed and their uses replaced with calls to the
new internal function __vsnprintf_internal. Since there were no
internal calls, the internal declaration was also removed. The
external symbol is preserved with ldbl_weak_alias to ___vsnprintf.
__vfwprintf:
All internal calls converted into calls to __vfwprintf_internal,
thus the internal declaration was removed. The function is now a
wrapper that calls __vfwprintf_internal. The external symbol is
preserved.
__vswprintf:
Similarly, but no external symbol.
__vasprintf, __vdprintf, __vfprintf, __vsprintf:
New internal wrappers. Not exported.
vasprintf, vdprintf, vfprintf, vsprintf, vsnprintf,
vfwprintf, vswprintf,
obstack_vprintf, obstack_printf:
These functions used to be aliases to the respective _IO_* function,
they are now aliases to their respective __* functions.
Tested for powerpc and powerpc64le.
Change the callers of __vfscanf_internal and __vfwscanf_internal that
want C99-compliant behavior to communicate this via the new flags
argument, rather than setting bits on the FILE object. This also
means these functions do not need to do their own locking.
Tested for powerpc and powerpc64le.
There are two flags currently defined: SCANF_LDBL_IS_DBL is the mode
used by __nldbl_ scanf variants, and SCANF_ISOC99_A is the mode used
by __isoc99_ scanf variants. In this patch, the new functions honor
these flag bits if they're set, but they still also look at the
corresponding bits of environmental state, and callers all pass zero.
The new functions do *not* have the "errp" argument possessed by
_IO_vfscanf and _IO_vfwscanf. All internal callers passed NULL for
that argument. External callers could theoretically exist, so I
preserved wrappers, but they are flagged as compat symbols and they
don't preserve the three-way distinction among types of errors that
was formerly exposed. These functions probably should have been in
the list of deprecated _IO_ symbols in 2.27 NEWS -- they're not just
aliases for vfscanf and vfwscanf.
(It was necessary to introduce ldbl_compat_symbol for _IO_vfscanf.
Please check that part of the patch very carefully, I am still not
confident I understand all of the details of ldbl-opt.)
This patch also introduces helper inlines in libio/strfile.h that
encapsulate the process of initializing an _IO_strfile object for
reading. This allows us to call __vfscanf_internal directly from
sscanf, and __vfwscanf_internal directly from swscanf, without
duplicating the initialization code. (Previously, they called their
v-counterparts, but that won't work if we want to control *both* C99
mode and ldbl-is-dbl mode using the flags argument to__vfscanf_internal.)
It's still a little awkward, especially for wide strfiles, but it's
much better than what we had.
Tested for powerpc and powerpc64le.
This patch uses posix_spawn on popen instead of fork and execl. On Linux
this has the advantage of much lower memory consumption (usually 32 Kb
minimum for the mmap stack area).
Two issues are also fixed with this change:
* BZ#17490: although POSIX pthread_atfork description only list 'fork'
as the function that should execute the atfork handlers, popen
description states that:
'[...] shall be *as if* a child process were created within the popen()
call using the fork() function [...]'
Other libc/system seems to follow the idea atfork handlers should not be
executed for popen:
libc/system | run atfork handles | notes
------------|----------------------|---------------------------------------
Freebsd | no | uses vfork
Solaris 11 | no |
MacOSX 11 | no | implemented through posix_spawn syscall
------------|----------------------|----------------------------------------
Similar to posix_spawn and system, popen idea is to spawn a different
binary so all the POSIX rationale to run the atfork handlers to avoid
internal process inconsistency is not really required and in some cases
might be unsafe.
* BZ#22834: the described scenario, where the forked process might access
invalid memory due an inconsistent state in multithreaded environment,
should not happen because posix_spawn does not access the affected
data structure (proc_file_chain).
Checked on x86_64-linux-gnu and i686-linux-gnu.
[BZ #22834]
[BZ #17490]
* NEWS: Add new semantic for atfork with popen and system.
* libio/iopopen.c (_IO_new_proc_open): use posix_spawn instead of
fork and execl.
Signal zero does not terminate a process, so it is safe to use negative
values for signal numbers.
Adjust libio/tst-vtables-common.c to use this new functionality,
instead of determining the termination status for a signal indirectly.
Increase timeout from the default 20s to 100s. This test makes close to
20 million syscalls with distribution:
12327675 read
4143204 lseek
929475 close
929471 openat
92817 fstat
1431 write
...
The default timeout assumes each can finish in 1us on average which
is not true on slow machines.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* libio/tst-readline.c (TIMEOUT): Define.
As POSIX states [1] a freopen call should first flush the stream as if by a
call fflush. C99 (n1256) and C11 (n1570) only states the function should
first close any file associated with the specific stream. Although current
implementation only follow C specification, current BSD and other libc
implementation (musl) are in sync with POSIX and fflush the stream.
This patch change freopen{64} to fflush the stream before actually reopening
it (or returning if the stream does not support reopen). It also changes the
Linux implementation to avoid a dynamic allocation on 'fd_to_filename'.
Checked on x86_64-linux-gnu.
[BZ #21037]
* libio/Makefile (tests): Add tst-memstream4 and tst-wmemstream4.
* libio/freopen.c (freopen): Sync stream before reopen and adjust to
new fd_to_filename interface.
* libio/freopen64.c (freopen64): Likewise.
* libio/tst-memstream.h: New file.
* libio/tst-memstream4.c: Likewise.
* libio/tst-wmemstream4.c: Likewise.
* sysdeps/generic/fd_to_filename.h (fd_to_filename): Change signature.
* sysdeps/unix/sysv/linux/fd_to_filename.h (fd_to_filename): Likewise
and remove internal dynamic allocation.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/
This is a variant of fgets which fails with ERANGE if the
buffer is too small, and the buffer length is given as an
argument of type size_t.
This function will be useful for implementing NSS file reading
operations. Compared to a direct implementation using the public API,
it avoids an lseek system call in case the line terminator can be
found in the internal read buffer.
The implementation falls back to renameat if renameat2 is not available
in the kernel (or in the kernel headers) and the flags argument is zero.
Without kernel support, a non-zero argument returns EINVAL, not ENOSYS.
This mirrors what the kernel does for invalid renameat2 flags.
These unmangled function pointers reside on the heap and could
be targeted by exploit writers, effectively bypassing libio vtable
validation. Instead, we ignore these pointers and always call
malloc or free.
In theory, this is a backwards-incompatible change, but using the
global heap instead of the user-supplied callback functions should
have little application impact. (The old libstdc++ implementation
exposed this functionality via a public, undocumented constructor
in its strstreambuf class.)
C99 specifies that the EOF condition on a file is "sticky": once EOF
has been encountered, all subsequent reads should continue to return
EOF until the file is closed or something clears the "end-of-file
indicator" (e.g. fseek, clearerr). This is arguably a change from
C89, where the wording was ambiguous; the BSDs always had sticky EOF,
but the System V lineage would attempt to read from the underlying fd
again. GNU libc has followed System V for as long as we've been
using libio, but nowadays C99 conformance and BSD compatibility are
more important than System V compatibility.
You might wonder if changing the _underflow impls is sufficient to
apply the C99 semantics to all of the many stdio functions that
perform input. It should be enough to cover all paths to _IO_SYSREAD,
and the only other functions that call _IO_SYSREAD are the _seekoff
impls, which is OK because seeking clears EOF, and the _xsgetn impls,
which, as far as I can tell, are unused within glibc.
The test programs in this patch use a pseudoterminal to set up the
necessary conditions. To facilitate this I added a new test-support
function that sets up a pair of pty file descriptors for you; it's
almost the same as BSD openpty, the only differences are that it
allocates the optionally-returned tty pathname with malloc, and that
it crashes if anything goes wrong.
[BZ #1190]
[BZ #19476]
* libio/fileops.c (_IO_new_file_underflow): Return EOF immediately
if the _IO_EOF_SEEN bit is already set; update commentary.
* libio/oldfileops.c (_IO_old_file_underflow): Likewise.
* libio/wfileops.c (_IO_wfile_underflow): Likewise.
* support/support_openpty.c, support/tty.h: New files.
* support/Makefile (libsupport-routines): Add support_openpty.
* libio/tst-fgetc-after-eof.c, wcsmbs/test-fgetwc-after-eof.c:
New test cases.
* libio/Makefile (tests): Add tst-fgetc-after-eof.
* wcsmbs/Makefile (tests): Add tst-fgetwc-after-eof.
This patch eliminates a number of #if 0 and #ifdef TODO blocks, macros
that are never used, macros that provide portability to substrates that
lack basic things like EINVAL and off_t, and other such debris.
I preserved IO_DEBUG and CHECK_FILE, even though as far as I can tell
IO_DEBUG is never defined and therefore CHECK_FILE never does
anything, because it seems like we might actually want to turn it _on_.
Installed stripped libraries and executables are unchanged, except,
again, that the line number of an assertion changes (this time it's
somewhere in fileops.c).
* libio/libio.h (_IO_pos_BAD, _IO_pos_0, _IO_pos_adjust):
Define here, unconditionally.
* libio/iolibio.h (_IO_pos_BAD): Don't define here.
* libio/libioP.h: Remove #if 0 blocks.
(_IO_pos_BAD, _IO_pos_0, _IO_pos_adjust): Don't define here.
(_IO_va_start, COERCE_FILE, MAYBE_SET_EINVAL): Don't define.
(CHECK_FILE): Don't use MAYBE_SET_EINVAL or COERCE_FILE. Fix style.
* libio/clearerr.c, libio/fputc.c, libio/getchar.c:
Assume weak_alias is always defined.
* libio/fileops.c, libio/genops.c, libio/oldfileops.c
* libio/oldpclose.c, libio/pclose.c, libio/wfileops.c:
Remove #if 0 and #ifdef TODO blocks.
Assume text_set_element is always defined.
* libio/iofdopen.c, libio/iogetdelim.c, libio/oldiofdopen.c
Use __set_errno (EINVAL) instead of MAYBE_SET_EINVAL.
* libio/tst-mmap-eofsync.c: Make #if 1 block unconditional.
This entirely mechanical (except for some indentation fixups) patch
replaces all uses of _IO_file_flags with _flags and removes the #define.
Installed stripped libraries and executables are unchanged by this patch.
* libio/libio.h (_IO_file_flags): Remove macro.
All uses changed to _flags.
This patch eliminates the "compatibility defines"
_IO_UNIFIED_JUMPTABLES (always defined to 1, used in a number of #ifs
which are therefore always false), _STDIO_USES_IOSTREAM (unused),
__HAVE_COLUMN (unused), _IO_BE (replaced with __glibc_unlikely), and
yet another redundant definition of EOF.
Installed stripped libraries are unchanged by this patch.
* libio/libio.h (_IO_UNIFIED_JUMPTABLES, _STDIO_USES_IOSTREAM)
(__HAVE_COLUMN, _IO_BE): Don't define.
(_IO_peekc_unlocked, _IO_getwc_unlocked, _IO_putwc_unlocked)
(_IO_fwide_maybe_incompatible): Use __glibc_unlikely.
* libio/libioP.h (EOF): Don't define.
* libio/iofdopen.c, libio/iofopen.c, libio/iopopen.c
* libio/iovdprintf.c, libio/oldiofdopen.c, libio/oldiofopen.c
* libio/oldiopopen.c, debug/vdprintf_chk.c: Remove #if block
testing _IO_UNIFIED_JUMPTABLES.
This patch mechanically removes all remaining uses, and the
definitions, of the following libio name aliases:
name replaced with
---- -------------
_IO_FILE FILE
_IO_fpos_t __fpos_t
_IO_fpos64_t __fpos64_t
_IO_size_t size_t
_IO_ssize_t ssize_t or __ssize_t
_IO_off_t off_t
_IO_off64_t off64_t
_IO_pid_t pid_t
_IO_uid_t uid_t
_IO_wint_t wint_t
_IO_va_list va_list or __gnuc_va_list
_IO_BUFSIZ BUFSIZ
_IO_cookie_io_functions_t cookie_io_functions_t
__io_read_fn cookie_read_function_t
__io_write_fn cookie_write_function_t
__io_seek_fn cookie_seek_function_t
__io_close_fn cookie_close_function_t
I used __fpos_t and __fpos64_t instead of fpos_t and fpos64_t because
the definitions of fpos_t and fpos64_t depend on the largefile mode.
I used __ssize_t and __gnuc_va_list in a handful of headers where
namespace cleanliness might be relevant even though they're
internal-use-only. In all other cases, I used the public-namespace
name.
There are a tiny handful of places where I left a use of 'struct _IO_FILE'
alone, because it was being used together with 'struct _IO_FILE_plus'
or 'struct _IO_FILE_complete' in the same arithmetic expression.
Because this patch was almost entirely done with search and replace, I
may have introduced indentation botches. I did proofread the diff,
but I may have missed something.
The ChangeLog below calls out all of the places where this was not a
pure search-and-replace change.
Installed stripped libraries and executables are unchanged by this patch,
except that some assertions in vfscanf.c change line numbers.
* libio/libio.h (_IO_FILE): Delete; all uses changed to FILE.
(_IO_fpos_t): Delete; all uses changed to __fpos_t.
(_IO_fpos64_t): Delete; all uses changed to __fpos64_t.
(_IO_size_t): Delete; all uses changed to size_t.
(_IO_ssize_t): Delete; all uses changed to ssize_t or __ssize_t.
(_IO_off_t): Delete; all uses changed to off_t.
(_IO_off64_t): Delete; all uses changed to off64_t.
(_IO_pid_t): Delete; all uses changed to pid_t.
(_IO_uid_t): Delete; all uses changed to uid_t.
(_IO_wint_t): Delete; all uses changed to wint_t.
(_IO_va_list): Delete; all uses changed to va_list or __gnuc_va_list.
(_IO_BUFSIZ): Delete; all uses changed to BUFSIZ.
(_IO_cookie_io_functions_t): Delete; all uses changed to
cookie_io_functions_t.
(__io_read_fn): Delete; all uses changed to cookie_read_function_t.
(__io_write_fn): Delete; all uses changed to cookie_write_function_t.
(__io_seek_fn): Delete; all uses changed to cookie_seek_function_t.
(__io_close_fn): Delete: all uses changed to cookie_close_function_t.
* libio/iofopncook.c: Remove unnecessary forward declarations.
* libio/iolibio.h: Correct outdated commentary.
* malloc/malloc.c (__malloc_stats): Remove unnecessary casts.
* stdio-common/fxprintf.c (__fxprintf_nocancel):
Remove unnecessary casts.
* stdio-common/getline.c: Use _IO_getdelim directly.
Don't redefine ssize_t.
* stdio-common/printf_fp.c, stdio_common/printf_fphex.c
* stdio-common/printf_size.c: Don't redefine size_t or FILE.
Remove outdated comments.
* stdio-common/vfscanf.c: Don't redefine va_list.
As requested by Adhemerval, this patch removes some preprocessor
conditionals from the libio headers that were only relevant when
building libio outside glibc.
Installed stripped libraries and executables are unchanged by this
patch.
* libio/iolibio.h, libio/libioP.h: Remove extern "C".
* libio/libio.h: Remove __BEGIN_DECLS and __END_DECLS.
Remove preprocessor conditionals on _LIBC and __USE_GNU,
which are always true, and __cplusplus, which is always false.
Continuing the fixes for linknamespace and localplt test failures with
-Os that arise from functions not being inlined in that case, this
patch fixes such failures for putc_unlocked and fputc_unlocked.
libc_hidden_* are used for both functions, while namespace issues are
addressed by making putc_unlocked a weak alias of hidden
__putc_unlocked, which is called in the one place where namespace
issues arise (and defined as an inline function in include/stdio.h).
Tested for x86_64 (both without -Os to make sure that case continues
to work, and with -Os to make sure all the relevant linknamespace and
localplt test failures are resolved). This completes fixing the -Os
linknamespace failures (at least for x86_64); localplt failures remain
after this patch.
2018-02-19 Joseph Myers <joseph@codesourcery.com>
[BZ #15105]
[BZ #19463]
* libio/fputc_u.c (fputc_unlocked): Use libc_hidden_def.
* libio/putc_u.c (putc_unlocked): Rename to __putc_unlocked and
define as weak alias of __putc_unlocked. Use libc_hidden_weak.
* include/stdio.h [!_ISOMAC] (fputc_unlocked): Use
libc_hidden_proto.
[!_ISOMAC] (putc_unlocked): Likewise.
[!_ISOMAC] (__putc_unlocked): Declare as hidden function, and
define inline if [__USE_EXTERN_INLINES].
* misc/syslog.c (__vsyslog_chk): Call __putc_unlocked instead of
putc_unlocked.
Continuing the fixes for linknamespace and localplt test failures with
-Os that arise from functions not being inlined in that case, this
patch fixes such failures for getc_unlocked.
__getc_unlocked already exists; this patch makes it explicitly hidden,
calls it where needed for namespace reasons, adds an inline function
for it when inline functions are used and adds libc_hidden_proto /
libc_hidden_weak for getc_unlocked.
Tested for x86_64 (both without -Os to make sure that case continues
to work, and with -Os to make sure all the relevant linknamespace and
localplt test failures are resolved). Because of other such failures
that remain after this patch, neither of the bugs can yet be closed.
[BZ #15105]
[BZ #19463]
* libio/getc_u.c (getc_unlocked): Use libc_hidden_weak.
* include/stdio.h [!_ISOMAC] (__getc_unlocked): Use
attribute_hidden, and define inline if [__USE_EXTERN_INLINES].
[!_ISOMAC] (getc_unlocked): Use libc_hidden_proto.
* misc/getttyent.c (__getttyent): Call __getc_unlocked instead of
getc_unlocked.
* time/tzfile.c (__tzfile_read): Likewise.
Continuing the fixes for linknamespace and localplt test failures with
-Os that arise from functions not being inlined in that case, this
patch fixes such failures for ferror_unlocked.
The usual approach is followed of adding __ferror_unlocked (inlined
when ferror_unlocked is), making calls use it when required for
namespace reasons (only one such call), and using libc_hidden_proto /
libc_hidden_weak for the ferror_unlocked weak alias when only localplt
but not namespace issues are involved.
Tested for x86_64 (both without -Os to make sure that case continues
to work, and with -Os to make sure all the relevant linknamespace and
localplt test failures are resolved). Because of other such failures
that remain after this patch, neither of the bugs can yet be closed.
[BZ #15105]
[BZ #19463]
* libio/ferror_u.c (ferror_unlocked): Rename to __ferror_unlocked
and define as weak alias of __ferror_unlocked. Use
libc_hidden_weak.
* include/stdio.h [!_ISOMAC] (ferror_unlocked): Use
libc_hidden_proto.
[!_ISOMAC] (__ferror_unlocked) New declaration, and inline
function if [__USE_EXTERN_INLINES].
* time/getdate.c (__getdate_r): Call __ferror_unlocked instead of
ferror_unlocked.
