The cached offset is reliable to use in ftell when the stream handle
is active. We can consider a stream as being active when there is
unflushed data. However, even in this case, we can use the cached
offset only when the stream is not being written to in a+ mode,
because this case may have unflushed data and a stale offset; the
previous read could have sent it off somewhere other than the end of
the file.
There were a couple of adjustments necessary to get this to work.
Firstly, fdopen now ceases to use _IO_attach_fd because it sets the
offset cache to the current file position. This is not correct
because there could be changes to the file descriptor before the
stream handle is activated, which would not get reflected.
A similar offset caching action is done in _IO_fwide, claiming that
wide streams have 'problems' with the file offsets. There don't seem
to be any obvious problems with not having the offset cache available,
other than that it will have to be queried in a subsequent
read/write/seek. I have removed this as well.
The testsuite passes successfully with these changes on x86_64.
ftell semantics are distinct from fseek(SEEK_CUR) especially when it
is called on a file handler that is not yet active. Due to this
caveat, much care needs to be taken while modifying the handler data
and hence, this first iteration on separating out ftell focusses on
maintaining handler data integrity at all times while it figures out
the current stream offset. The result is that it makes a syscall for
every offset request.
There is scope for optimizing this by caching offsets when we know
that the handler is active. A simple way to find out is when the
buffers have data. It is not so simple to find this out when the
buffer is empty without adding some kind of flag.
In <https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html> I
noted it was necessary to add includes of Makeconfig early in various
subdirectory makefiles for the tests-special variable settings added
by that patch to be conditional on configuration information. No-one
commented on the general question there of whether Makeconfig should
always be included immediately after the definition of subdir.
This patch implements that early inclusion of Makeconfig in each
directory (which is a lot easier than consistent placement of includes
of Rules). Includes are added if needed, or moved up if already
present. Subdirectory "all:" targets are removed, since Makeconfig
provides one.
There is potential for further cleanups I haven't done. Rules and
Makerules have code such as
ifneq "$(findstring env,$(origin headers))" ""
headers :=
endif
to override to empty any value of various variables that came from the
environment. I think there is a case for Makeconfig setting all the
subdirectory variables (other than subdir) to empty to ensure no
outside value is going to take effect if a subdirectory fails to
define a variable. (A list of such variables, possibly out of date
and incomplete, is in manual/maint.texi.) Rules and Makerules would
give errors if Makeconfig hadn't already been included, instead of
including it themselves. The special code to override values coming
from the environment would then be obsolete and could be removed.
Tested x86_64, including that installed binaries are identical before
and after the patch.
* argp/Makefile: Include Makeconfig immediately after defining
subdir.
* assert/Makefile: Likewise.
* benchtests/Makefile: Likewise.
* catgets/Makefile: Likewise.
* conform/Makefile: Likewise.
* crypt/Makefile: Likewise.
* csu/Makefile: Likewise.
(all): Remove target.
* ctype/Makefile: Include Makeconfig immediately after defining
subdir.
* debug/Makefile: Likewise.
* dirent/Makefile: Likewise.
* dlfcn/Makefile: Likewise.
* gmon/Makefile: Likewise.
* gnulib/Makefile: Likewise.
* grp/Makefile: Likewise.
* gshadow/Makefile: Likewise.
* hesiod/Makefile: Likewise.
* hurd/Makefile: Likewise.
(all): Remove target.
* iconvdata/Makefile: Include Makeconfig immediately after
defining subdir.
* inet/Makefile: Likewise.
* intl/Makefile: Likewise.
* io/Makefile: Likewise.
* libio/Makefile: Likewise.
(all): Remove target.
* locale/Makefile: Include Makeconfig immediately after defining
subdir.
* login/Makefile: Likewise.
* mach/Makefile: Likewise.
(all): Remove target.
* malloc/Makefile: Include Makeconfig immediately after defining
subdir.
(all): Remove target.
* manual/Makefile: Include Makeconfig immediately after defining
subdir.
* math/Makefile: Likewise.
* misc/Makefile: Likewise.
* nis/Makefile: Likewise.
* nss/Makefile: Likewise.
* po/Makefile: Likewise.
(all): Remove target.
* posix/Makefile: Include Makeconfig immediately after defining
subdir.
* pwd/Makefile: Likewise.
* resolv/Makefile: Likewise.
* resource/Makefile: Likewise.
* rt/Makefile: Likewise.
* setjmp/Makefile: Likewise.
* shadow/Makefile: Likewise.
* signal/Makefile: Likewise.
* socket/Makefile: Likewise.
* soft-fp/Makefile: Likewise.
* stdio-common/Makefile: Likewise.
* stdlib/Makefile: Likewise.
* streams/Makefile: Likewise.
* string/Makefile: Likewise.
* sunrpc/Makefile: Likewise.
(all): Remove target.
* sysvipc/Makefile: Include Makeconfig immediately after defining
subdir.
* termios/Makefile: Likewise.
* time/Makefile: Likewise.
* timezone/Makefile: Likewise.
(all): Remove target.
* wcsmbs/Makefile: Include Makeconfig immediately after defining
subdir.
* wctype/Makefile: Likewise.
libidn/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
localedata/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
(all): Remove target.
nptl/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
nptl_db/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
Most glibc tests that use mtrace to verify that there were no memory
leaks from the glibc facilities used in a given test depend on the
.out file of the previous test so that the mtrace test runs mtrace and
nothing else.
Two, however, have a single target combining mtrace with something
else. In the case of libio/tst-fopenloc.check, the test both compares
the output with an expected baseline and runs mtrace. In the case of
posix/tst-rxspencer-mem, the test is run (with different command line
from the main run) and then mtrace is run, from the same makefile
target.
This patch splits both of these tests up to use separate makefile
targets for each thing tested; in the tst-rxspencer case, a file
tst-rxspencer-no-utf8.c is created that just includes tst-rxspencer.c,
as is usual for tests where the same code gets tested in different
compile-time or runtime configurations.
Adding $(evaluate-test) to test commands, as in
<https://sourceware.org/ml/libc-alpha/2014-01/msg00194.html>, will no
longer need to insert && between multiple commands, as all tests will
either have just a single command or already use &&.
Tested x86_64.
* libio/Makefile ($(objpfx)tst-fopenloc.check): Split into
separate $(objpfx)tst-fopenloc-cmp.out and
$(objpfx)tst-fopenloc-mem.out targets.
(tests): Update dependencies.
* posix/Makefile (tests variable): Add tst-rxspencer-no-utf8.
(generated): Change tst-rxspencer-mem and tst-rxspencer.mtrace to
tst-rxspencer-no-utf8-mem and tst-rxspencer-no-utf8.mtrace.
(tst-rxspencer-no-utf8-ARGS): New variable.
(tst-rxspencer-no-utf8-ENV): Likewise.
(tests target): Depend on $(objpfx)tst-rxspencer-no-utf8-mem
instead of $(objpfx)tst-rxspencer-mem.
($(objpfx)tst-rxspencer-mem): Change target to
$(objpfx)tst-rxspencer-no-utf8-mem. Depend on
$(objpfx)tst-rxspencer-no-utf8.out instead of running test program.
* posix/tst-rxspencer-no-utf8.c: New file.
This patch cleans up cases of __USE_MISC that are trivially redundant
after the recent substitution of __USE_MISC for __USE_BSD and
__USE_SVID: either in constructs such as "defined __USE_MISC ||
defined __USE_MISC", or else (in the bits/mman.h case) a conditional
on __USE_MISC nested inside another __USE_MISC conditional. (The
cleanups remaining after this patch are still quite large, but it
seems a reasonable piece to separate out.)
Tested x86_64.
* bits/mman.h [__USE_MISC]: Remove redundant conditionals.
* ctype/ctype.h [__USE_MISC]: Likewise.
* dirent/dirent.h [__USE_MISC]: Likewise.
* grp/grp.h [__USE_MISC]: Likewise.
* io/fcntl.h [__USE_MISC]: Likewise.
* io/sys/stat.h [__USE_MISC]: Likewise.
* libio/stdio.h [__USE_MISC]: Likewise.
* posix/unistd.h [__USE_MISC]: Likewise.
* pwd/pwd.h [__USE_MISC]: Likewise.
* stdlib.h [__USE_MISC]: Likewise.
* string/bits/string2.h [__USE_MISC]: Likewise.
* string/string.h [__USE_MISC]: Likewise.
* time/time.h [__USE_MISC]: Likewise.
ftell tries to avoid flushing the buffer when it is in write mode by
converting the wide char data and placing it into the binary buffer.
If the output buffer space is full and there is data to write, the
code reverts to flushing the buffer. This breaks when there is space
in the buffer but it is not enough to convert the next character in
the wide data buffer, due to which __codecvt_do_out returns a
__codecvt_partial status. In this case, ftell keeps running in an
infinite loop.
The fix here is to detect the __codecvt_partial status in addition to
checking if the buffer is full. I have also added a test case that
demonstrates the infinite loop.
Partially revert commits 2b766585f9 and
de2fd463b1, which were intended to fix BZ#11741
but caused another, likely worse bug, namely that fwrite() and fputs() could,
in an error path, read data beyond the end of the specified buffer, and
potentially even write this data to the file.
Fix BZ#11741 properly by checking the return value from _IO_padn() in
stdio-common/vfprintf.c.
Fixes 15381.
Using wide character function is on byte oriented memstream is undefined
behaviour. This behaviour was masked by not initializing wide struct
info. We now initialize it to cause a predictable crash.
In BZ #12724, partial support for POSIX 2008 fclose behavior was added.
Since it isn't entirely conforming to the spec, some applications are
known to be breaking in this intermediate state. So revert the partial
support until we can get things fully implemented.
This reverts commit fcabc0f8b1.
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
[BZ #14543]
Set the internal buffer state correctly whenever the external buffer
state is modified by fseek by either computing the current
_IO_read_ptr/end for the internal buffer based on the new _IO_read_ptr
in the external buffer or converting the content read into the
external buffer, up to the extent of the requested fseek offset.
[BZ #11959]
* libio/stdio.h (fwrite, fwrite_unlocked): Remove __wur.
It is not necessarily an error to ignore fwrite's return
value. One can reliably use ferror to test for errors after
the fact.
fopen should set the FD_CLOEXEC flag if requested evenif the kernel does
not support an aotmic operation.
freopen should reuse the file descriptor for the stream. This is
especially important for calls to change the standard streams (stin,
stdout, stderr).
* libio/wfileops.c (_IO_wfile_underflow): Fix handling of
incomplete characters at end of input buffer.
* libio/Makefile (tests): Add tst-fgetwc.
* libio/tst-fgetwc.c: New file.
* libio/tst-fgetwc.input: New file.
* libio/fmemopen.c: Implement binary mode. In this mode no NUL
byte gets added to writes and seeks from the end use the length of
the buffer and not the currently terminating NUL byte.
file descriptors with close-on-exec set.
(exec_comm_child): Fix the case where the write end of the pipe is
STDOUT_FILENO already. In case it is, clear close-on-exec.
* time/tzset.c (tzset_internal): Break TZ string parsing out into
__tzset_parse_tz and updating of daylight, timezone, tzname into
update_vars.
(__tz_compute): Renamed from tz_compute. Take additional parameters.
(__tz_convert): Updating of tm_isdst, tm_zone, and tm_gmtoff now
happens in __tz_compute.
* time/tzfile.c (__tzfile_read): Also read TZ string.
(find_transition): Fold into __tzfile_compute.
(__tzfile_compute): For times beyond the last transition try to
use the TZ string.
* timezone/tst-timezone.c: Information in daylight and tzname does
change for Asia/Tokyo timezone with more concrete information.
Remove the test.
* include/stdio.h: Add libc_hidden_proto for ftello.
* libio/ftello.c: Add libc_hidden_def.
with __warning__/__error__ attributes.
(__warnattr): Define.
* stdlib/bits/stdlib.h (__realpath_chk_warn, __ptsname_r_chk_warn,
__mbstowcs_chk_warn, __wcstombs_chk_warn): New aliases with
__warnattr.
(realpath, ptsname_r, mbstowcs, wcstombs): Call __*_chk_warn instead
of __*_chk if compile time detectable overflow is found.
* libio/bits/stdio2.h (__fgets_chk_warn, __fread_chk_warn,
__fgets_unlocked_chk_warn, __fread_unlocked_chk_warn): New aliases
with __warnattr.
(fgets, fread, fgets_unlocked, fread_unlocked): Call __*_chk_warn
instead of __*_chk if compile time detectable overflow is found.
(__gets_alias): Rename to...
(__gets_warn): ... this. Add __warnattr.
(gets): Call __gets_warn instead of __gets_alias.
* socket/bits/socket2.h (__recv_chk_warn, __recvfrom_chk_warn): New
aliases with __warnattr.
(recv, recvfrom): Call __*_chk_warn instead of __*_chk if compile
time detectable overflow is found.
* posix/bits/unistd.h (__read_chk_warn, __pread_chk_warn,
__pread64_chk_warn, __readlink_chk_warn, __readlinkat_chk_warn,
__getcwd_chk_warn, __confstr_chk_warn, __getgroups_chk_warn,
__ttyname_r_chk_warn, __getlogin_r_chk_warn, __gethostname_chk_warn,
__getdomainname_chk_warn): New aliases with __warnattr.
(read, pread, pread64, readlink, readlinkat, getcwd, confstr,
getgroups, ttyname_r, getlogin_r, gethostname, getdomainname): Call
__*_chk_warn instead of __*_chk if compile time detectable overflow
is found.
(__getgroups_chk): Rename argument to __listlen from listlen.
(__getwd_alias): Rename to...
(__getwd_warn): ... this. Add __warnattr.
(getwd): Call __getwd_warn instead of __getwd_alias.
* wcsmbs/bits/wchar2.h (__wmemcpy_chk_warn, __wmemmove_chk_warn,
__wmempcpy_chk_warn, __wmemset_chk_warn, __wcsncpy_chk_warn,
__wcpncpy_chk_warn, __fgetws_chk_warn, __fgetws_unlocked_chk_warn,
__mbsrtowcs_chk_warn, __wcsrtombs_chk_warn, __mbsnrtowcs_chk_warn,
__wcsnrtombs_chk_warn): New aliases with __warnattr.
(wmemcpy, wmemmove, wmempcpy, wmemset, mbsrtowcs, wcsrtombs,
mbsnrtowcs, wcsnrtombs): Call __*_chk_warn instead of __*_chk if
compile time detectable overflow is found.
(wcsncpy, wcpncpy): Likewise. For constant __n fix check whether
to use __*_chk or not.
(fgetws, fgetws_unlocked): Divide __bos by sizeof (wchar_t), both
in comparisons which function should be called and in __*_chk*
arguments. Call __*_chk_warn instead of __*_chk if compile time
detectable overflow is found.
(swprintf, vswprintf): Divide __bos by sizeof (wchar_t) in
__*_chk argument.
* debug/tst-chk1.c (do_test): Add a few more tests.
* misc/bits/syslog.h (syslog): When __va_arg_pack is defined,
implement as __extern_always_inline function.
(vsyslog): Define as __extern_always_inline function unconditionally.
* libio/bits/stdio2.h (sprintf, snprintf, printf, fprintf):
When __va_arg_pack is defined, implement as __extern_always_inline
functions.
(vsprintf, vsnprintf, vprintf, vfprintf): Define as
__extern_always_inline functions unconditionally.
* libio/bits/stdio.h (vprintf): Ifdef out the inline when
bits/stdio2.h will be included.
* wcsmbs/bits/wchar2.h (__swprintf_alias): New redirect.
(swprintf, wprintf, fwprintf): When __va_arg_pack is defined,
implement as __extern_always_inline functions.
(vswprintf, vwprintf, vfwprintf): Define as
__extern_always_inline functions unconditionally.
* debug/tst-chk1.c (do_test): Enable remaining tests for C++.
2007-09-03 Jakub Jelinek <jakub@redhat.com>
* misc/sys/cdefs.h (__extern_inline, __extern_always_inline): Only
define in C++ for GCC 4.3+, in C++ always use __gnu_inline__
attribute.
* include/features.h (__USE_EXTERN_INLINES): Define only when
__extern_inline is defined.
* stdlib/stdlib.h: Include bits/stdlib.h when __extern_always_inline
is defined instead of when not __cplusplus.
* misc/sys/syslog.h: Include bits/syslog.h when __extern_always_inline
is defined instead of when not __cplusplus.
* socket/sys/socket.h: Include bits/socket2.h when
__extern_always_inline is defined instead of when not __cplusplus.
* libio/stdio.h: Include bits/stdio2.h when __extern_always_inline
is defined instead of when not __cplusplus.
* posix/unistd.h: Include bits/unistd.h when __extern_always_inline
is defined instead of when not __cplusplus.
* string/string.h: Include bits/string3.h when __extern_always_inline
is defined instead of when not __cplusplus.
* wcsmbs/wchar.h: Include bits/wchar2.h when __extern_always_inline
is defined instead of when not __cplusplus.
(btowc, wctob): Don't guard the inlines with ifndef __cplusplus.
* io/fcntl.h: Don't include bits/fcntl2.h if __extern_always_inline
is not defined.
* misc/bits/syslog-ldbl.h: Guard *_chk stuff with
defined __extern_always_inline instead of !defined __cplusplus.
* libio/bits/stdio-ldbl.h: Likewise.
* wcsmbs/bits/wchar-ldbl.h: Likewise.
* misc/bits/syslog.h (syslog): Don't define for C++.
(vsyslog): Use __extern_always_inline function for C++ instead of
a macro.
* libio/bits/stdio.h (__STDIO_INLINE): Define to __extern_inline
whenever that macro is defined.
(vprintf): Don't provide the inline for C++.
(fread_unlocked, fwrite_unlocked): Don't define the macros for C++.
* libio/bits/stdio2.h (sprintf, snprintf, printf, fprintf): Don't
define the macros for C++.
(vsprintf, vsnprintf, vprintf, vfprintf): Define as
__extern_always_inline functions for C++.
* io/sys/stat.h (stat, lstat, fstat, fstatat, mknod, mknodat,
stat64, lstat64, fstat64, fstatat64): Don't define if not
__USE_EXTERN_INLINES.
* wcsmbs/bits/wchar2.h: Fix #error message.
(swprintf, wprintf, fwprintf): Don't define the macros for C++.
(vswprintf, vwprintf, vfwprintf): Define using
__extern_always_inline functions for C++.
* string/bits/string3.h: Don't #undef macros if __cplusplus.
(memcpy, memmove, mempcpy, memset, bcopy, bzero, strcpy, stpcpy,
strncpy, strcat, strncat): Define as __extern_always_inline
functions instead of macros for C++.
* math/bits/cmathcalls.h: Guard __extern_inline routines with
defined __extern_inline.
* sysdeps/alpha/fpu/bits/mathinline.h (__MATH_INLINE): Define
to __extern_inline whenever that macro is defined.
* sysdeps/ia64/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/i386/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/i386/i486/bits/string.h (__STRING_INLINE): Likewise.
* sysdeps/s390/bits/string.h (__STRING_INLINE): Likewise.
* sysdeps/s390/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/powerpc/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/x86_64/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/sparc/fpu/bits/mathinline.h (__MATH_INLINE): Likewise.
* sysdeps/unix/sysv/linux/sys/sysmacros.h (gnu_dev_major,
gnu_dev_minor, gnu_dev_makedev): Remove __extern_inline from
prototypes. Only provide __extern_inline routines if
__USE_EXTERN_INLINES.
* debug/Makefile: Add rules to build and run tst-{,lfs}chk{4,5,6}
tests.
* debug/tst-chk1.c (do_prepare, do_test): Allow compilation as C++.
For now avoid some *printf tests in C++. Skip all testing
if __USE_FORTIFY_LEVEL is defined, but __extern_always_inline macro
is not.
* debug/tst-chk4.cc: New file.
* debug/tst-chk5.cc: New file.
* debug/tst-chk6.cc: New file.
* debug/tst-lfschk4.cc: New file.
* debug/tst-lfschk5.cc: New file.
* debug/tst-lfschk6.cc: New file.
* include/wchar.h (__vfwprintf_chk, __vswprintf_chk): Avoid
prototypes in C++.
* include/stdio.h (__sprintf_chk, __snprintf_chk, __vsprintf_chk,
__vsnprintf_chk, __printf_chk, __fprintf_chk, __vprintf_chk,
__vfprintf_chk, __fgets_unlocked_chk, __fgets_chk): Likewise.
* iconv/gconv_cache.c (__gconv_lookup_cache): Return __GCONV_NULCONV
if from and to charsets are the same.
* iconv/gconv_db.c (__gconv_find_transform): Likewise.
* intl/dcigettext.c (_nl_find_msg): Return NULL even if __gconv_open
returns __GCONV_NOCONV, but not for __GCONV_NULCONV.
2007-07-17 Jakub Jelinek <jakub@redhat.com>
* wcsmbs/wchar.h: Only define wint_t if __need_wint_t.
Don't define wint_t when __need_mbstate_t unless it
is necessary.
(__mbstate_t): Use __WINT_TYPE__ rather than wint_t
in the typedef if possible.
* wctype/wctype.h (wint_t): Define by including
wchar.h with __need_wint_t instead of including stddef.h
with __need_wint_t and as fallback definining it ourselves.
* iconv/gconv.h (__need_wint_t): Define before including
wchar.h.
* sysdeps/gnu/_G_config.h: Don't include gconv.h if not _LIBC
or _GLIBCPP_USE_WCHAR_T.
(__need_wchar_t): Don't define
if not _LIBC or _GLIBCPP_USE_WCHAR_T.
(__need_wint_t): Don't define before including stddef.h,
define before including wchar.h only if _LIBC or
_GLIBCPP_USE_WCHAR_T.
(_G_iconv_t): Don't define if not _LIBC or _GLIBCPP_USE_WCHAR_T.
* sysdeps/mach/hurd/_G_config.h: Likewise.
* sysdeps/generic/_G_config.h: Likewise.
* libio/libio.h (__wunderflow, __wuflow, __woverflow): Only
prototype if _LIBC or _GLIBCPP_USE_WCHAR_T.
(_IO_getwc_unlocked, _IO_putwc_unlocked): Only define
if _LIBC or _GLIBCPP_USE_WCHAR_T.
