I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
Either the __NR_semtimedop_time64 (for 32-bit) or the __NR_semtimedop
(for 64-bit) syscall is used as default. The 32-bit fallback is used
iff __ASSUME_TIME64_SYSCALLS is not defined, which assumes the kernel
ABI provides either __NR_ipc or __NR_semtimeop (for 32-bit time_t).
Checked on x86_64-linux-gnu and i686-linux-gnu (on 5.4 and on 4.15
kernel).
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
To provide a y2038 safe interface a new symbol __shmctl64 is added
and __shmctl is change to call it instead (it adds some extra buffer
copying for the 32 bit time_t implementation).
Two new structures are added:
1. kernel_shmid64_ds: used internally only on 32-bit architectures
to issue the syscall. A handful of architectures (hppa, i386,
mips, powerpc32, and sparc32) require specific implementations
due to their kernel ABI.
2. shmid_ds64: this is only for __TIMESIZE != 64 to use along with
the 64-bit shmctl. It is different than the kernel struct because
the exported 64-bit time_t might require different alignment
depending on the architecture ABI.
So the resulting implementation does:
1. For 64-bit architectures it assumes shmid_ds already contains
64-bit time_t fields and will result in just the __shmctl symbol
using the __shmctl64 code. The shmid_ds argument is passed as-is
to the syscall.
2. For 32-bit architectures with default 64-bit time_t (newer ABIs
such riscv32 or arc), it will also result in only one exported
symbol but with the required high/low time handling.
3. Finally for 32-bit architecture with both 32-bit and 64-bit time_t
support we follow the already set way to provide one symbol with
64-bit time_t support and implement the 32-bit time_t support
using of the 64-bit one.
The default 32-bit symbol will allocate and copy the shmid_ds
over multiple buffers, but this should be deprecated in favor
of the __shmctl64 anyway.
Checked on i686-linux-gnu and x86_64-linux-gnu. I also did some sniff
tests on powerpc, powerpc64, mips, mips64, armhf, sparcv9, and
sparc64.
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
To provide a y2038 safe interface a new symbol __msgctl64 is added
and __msgctl is change to call it instead (it adds some extra buffer
coping for the 32 bit time_t implementation).
Two new structures are added:
1. kernel_msqid64_ds: used internally only on 32-bit architectures
to issue the syscall. A handful of architectures (hppa, i386, mips,
powerpc32, and sparc32) require specific implementations due to
their kernel ABI.
2. msqid_ds64: this is only for __TIMESIZE != 64 to use along with
the 64-bit msgctl. It is different than the kernel struct because
the exported 64-bit time_t might require different alignment
depending on the architecture ABI.
So the resulting implementation does:
1. For 64-bit architectures it assumes msqid_ds already contains
64-bit time_t fields and will result in just the __msgctl symbol
using the __msgctl64 code. The msgid_ds argument is passed as-is
to the syscall.
2. For 32-bit architectures with default 64-bit time_t (newer ABIs
such riscv32 or arc), it will also result in only one exported
symbol but with the required high/low time handling.
3. Finally for 32-bit architecture with both 32-bit and 64-bit time_t
support we follow the already set way to provide one symbol with
64-bit time_t support and implement the 32-bit time_t support using
the 64-bit time_t.
The default 32-bit symbol will allocate and copy the msqid_ds
over multiple buffers, but this should be deprecated in favor
of the __msgctl64 anyway.
Checked on i686-linux-gnu and x86_64-linux-gnu. I also did some sniff
tests on powerpc, powerpc64, mips, mips64, armhf, sparcv9, and
sparc64.
Tested-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Alistair Francis <alistair.francis@wdc.com>
Different than others 64-bit time_t syscalls, the SysIPC interface
does not provide a new set of syscall for y2038 safeness. Instead it
uses unused fields in semid_ds structure to return the high bits for
the timestamps.
To provide a y2038 safe interface a new symbol __semctl64 is added
and __semctl is change to call it instead (it adds some extra buffer
copying for the 32 bit time_t implementation).
Two new structures are added:
1. kernel_semid64_ds: used internally only on 32-bit architectures
to issue the syscall. A handful of architectures (hppa, i386,
mips, powerpc32, sparc32) require specific implementations due
their kernel ABI.
2. semid_ds64: this is only for __TIMESIZE != 64 to use along with
the 64-bit semctl. It is different than the kernel struct because
the exported 64-bit time_t might require different alignment
depending on the architecture ABI.
So the resulting implementation does:
1. For 64-bit architectures it assumes semid_ds already contains
64-bit time_t fields and will result in just the __semctl symbol
using the __semctl64 code. The semid_ds argument is passed as-is
to the syscall.
2. For 32-bit architectures with default 64-bit time_t (newer ABIs
such riscv32 or arc), it will also result in only one exported
symbol but with the required high/low handling.
It might be possible to optimize it further to avoid the
kernel_semid64_ds to semun transformation if the exported ABI
for the architectures matches the expected kernel ABI, but the
implementation is already complex enough and don't think this
should be a hotspot in any case.
3. Finally for 32-bit architecture with both 32-bit and 64-bit time_t
support we follow the already set way to provide one symbol with
64-bit time_t support and implement the 32-bit time_t support
using the 64-bit one.
The default 32-bit symbol will allocate and copy the semid_ds
over multiple buffers, but this should be deprecated in favor
of the __semctl64 anyway.
Checked on i686-linux-gnu and x86_64-linux-gnu. I also did some sniff
tests on powerpc, powerpc64, mips, mips64, armhf, sparcv9, and
sparc64.
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Vineet Gupta <vgupta@synopsys.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Tested-by: Carlos O'Donell <carlos@redhat.com>
This patch provides new __ntp_gettimex64 explicit 64 bit function for getting
time parameters via NTP interface.
The call to __adjtimex in __ntp_gettime64 function has been replaced with
direct call to __clock_adjtime64 syscall, to simplify the code.
Moreover, a 32 bit version - __ntp_gettimex has been refactored to internally
use __ntp_gettimex64.
The __ntp_gettimex is now supposed to be used on systems still supporting 32
bit time (__TIMESIZE != 64) - hence the necessary conversions between struct
ntptimeval and 64 bit struct __ntptimeval64.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
https://github.com/lmajewski/meta-y2038 and run tests:
https://github.com/lmajewski/y2038-tests/commits/master
Above tests were performed with Y2038 redirection applied as well as without to
test the proper usage of both __ntp_gettimex64 and __ntp_gettimex.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch provides new __ntp_gettime64 explicit 64 bit function for getting
time parameters via NTP interface.
Internally, the __clock_adjtime64 syscall is used instead of __adjtimex. This
patch is necessary for having architectures with __WORDSIZE == 32 Y2038 safe.
Moreover, a 32 bit version - __ntp_gettime has been refactored to internally
use __ntp_gettime64.
The __ntp_gettime is now supposed to be used on systems still supporting 32
bit time (__TIMESIZE != 64) - hence the necessary conversions between struct
ntptimeval and 64 bit struct __ntptimeval64.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
https://github.com/lmajewski/meta-y2038 and run tests:
https://github.com/lmajewski/y2038-tests/commits/master
Above tests were performed with Y2038 redirection applied as well as without to
test the proper usage of both __ntp_gettime64 and __ntp_gettime.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Those functions allow easy conversion between Y2038 safe, glibc internal
struct __ntptimeval64 and struct ntptimeval.
The reserved fields (i.e. __glibc_reserved{1234}) during conversion are
zeroed as well, to provide behavior similar to one in ntp_gettimex function
(where those are cleared before the struct ntptimeval is returned).
Those functions are put in Linux specific sys/timex.h file, as putting
them into glibc's local include/time.h would cause build break on HURD as
it doesn't support struct timex related syscalls.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This type is a glibc's "internal" type to get time parameters data from
Linux kernel (NTP daemon interface). It stores time in struct __timeval64
rather than struct timeval, which makes it Y2038-proof.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch provides new ___adjtimex64 explicit 64 bit function for adjusting
Linux kernel clock.
Internally, the __clock_adjtime64 syscall is used. This patch is necessary
for having architectures with __WORDSIZE == 32 Y2038 safe.
Moreover, a 32 bit version - ___adjtimex has been refactored to internally
use ___adjtimex64.
The ___adjtimex is now supposed to be used on systems still supporting 32
bit time (__TIMESIZE != 64) - hence the necessary conversions between struct
timex and 64 bit struct __timex64.
Last but not least, in ___adjtimex64 function the __clock_adjtime syscall has
been replaced with __clock_adjtime64 to support 64 bit time on architectures
with __WORDSIZE == 32 and __TIMESIZE != 64.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
https://github.com/lmajewski/meta-y2038 and run tests:
https://github.com/lmajewski/y2038-tests/commits/master
Above tests were performed with Y2038 redirection applied as well as without to
test the proper usage of both ___adjtimex64 and ___adjtimex.
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This patch replaces auto generated wrapper (as described in
sysdeps/unix/sysv/linux/syscalls.list) for clock_adjtime with one which adds
extra support for reading 64 bit time values on machines with __TIMESIZE != 64.
To achieve this goal new __clock_adjtime64 explicit 64 bit function for
adjusting Linux clock has been added.
Moreover, a 32 bit version - __clock_adjtime has been refactored to internally
use __clock_adjtime64.
The __clock_adjtime is now supposed to be used on systems still supporting 32
bit time (__TIMESIZE != 64) - hence the necessary conversions between 64 bit
struct __timespec64 and struct timespec.
The new __clock_adjtime64 syscall available from Linux 5.1+ has been used, when
applicable.
Up till v5.4 in the Linux kernel there was a bug preventing this call from
obtaining correct struct's timex time.tv_sec time after time_t overflow
(i.e. not being Y2038 safe).
Build tests:
- ./src/scripts/build-many-glibcs.py glibcs
Run-time tests:
- Run specific tests on ARM/x86 32bit systems (qemu):
https://github.com/lmajewski/meta-y2038 and run tests:
https://github.com/lmajewski/y2038-tests/commits/master
Linux kernel, headers and minimal kernel version for glibc build test matrix:
- Linux v5.1 (with clock_adjtime64) and glibc build with v5.1 as
minimal kernel version (--enable-kernel="5.1.0")
The __ASSUME_TIME64_SYSCALLS flag defined.
- Linux v5.1 and default minimal kernel version
The __ASSUME_TIME64_SYSCALLS not defined, but kernel supports clock_adjtime64
syscall.
- Linux v4.19 (no clock_adjtime64 support) with default minimal kernel version
for contemporary glibc (3.2.0)
This kernel doesn't support clock_adjtime64 syscall, so the fallback to
clock_adjtime is tested.
Above tests were performed with Y2038 redirection applied as well as without
(so the __TIMESIZE != 64 execution path is checked as well).
No regressions were observed.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Only alpha and ia64 do not support __NR_umount2 (defined as
__NR_umount), but recent kernel fixes (74cd2184833f for ia64, and
12b57c5c70f39 for alpha) add the required alias.
Checked with a build against all affected ABIs.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
Those functions allow easy conversion between Y2038 safe, glibc internal
struct __timex64 and struct timex.
Those functions are put in Linux specific sys/timex.h file, as putting
them into glibc's local include/time.h would cause build break on HURD as
it doesn't support struct timex related syscalls.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The introduced glibc's 'internal' struct __timex64 is a copy of Linux kernel's
struct __kernel_timex (v5.6) introduced for properly handling data for
clock_adjtime64 syscall.
As the struct's __kernel_timex size is the same as for archs with
__WORDSIZE == 64, proper padding and data types conversion (i.e. long to long
long) had to be added for architectures with __WORDSIZE == 32 &&
__TIMESIZE != 64.
Moreover, it stores time in struct __timeval64 rather than struct
timeval, which makes it Y2038-proof.
Build tests:
./src/scripts/build-many-glibcs.py glibcs
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Use <arch-syscall.h> instead of <asm/unistd.h> to obtain the system
call numbers. A few direct includes of <asm/unistd.h> need to be
removed (if the system call numbers are already provided indirectly
by <sysdep.h>) or replaced with <sys/syscall.h>.
Current Linux headers for alpha define the required system call names,
so most of the _NR_* hacks are no longer needed. For the 32-bit arm
architecture, eliminate the INTERNAL_SYSCALL_ARM macro, now that we
have regular system call names for cacheflush and set_tls. There are
more such cleanup opportunities for other architectures, but these
cleanups are required to avoid macro redefinition errors during the
build.
For ia64, it is desirable to use <asm/break.h> directly to obtain
the break number for system calls (which is not a system call number
itself). This requires replacing __BREAK_SYSCALL with
__IA64_BREAK_SYSCALL because the former is defined as an alias in
<asm/unistd.h>, but not in <asm/break.h>.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Hide internal __sysinfo function to allow direct access within libc.so and
libc.a without using GOT nor PLT.
[BZ #18822]
* sysdeps/unix/sysv/linux/include/sys/sysinfo.h (__sysinfo): Add
attribute_hidden.
On Linux-based configurations, bits/syscall.h is a generated file.
To avoid build-ordering problems, the Linux sys/syscall.h only includes
bits/syscall.h if _LIBC is not defined. After the _ISOMAC-testsuite
changes, this means any test case that includes sys/syscall.h tries to
pull in bits/syscall.h. This would be fine, because it'll definitely
have been generated by the time we start compiling tests, except that
the generated <builddir>/misc/bits/syscall.h is not visible in the
include path, because nothing needed it till now. So we either get
the bits/syscall.h from the host system, or the build fails.
The fix is simple: add a shim header for bits/syscall.h. I put it in
sysdeps/unix/sysv/linux/include instead of the top-level include/
because bits/syscall.h doesn't exist at all on other configurations as
far as I can tell.
This is known to affect nptl/tst-cond2[45]. Thanks to John David
Anglin for noticing the problem.
[BZ #21514]
* sysdeps/unix/sysv/linux/include/bits/syscall.h: New shim header
pointing to the generated file in <builddir>/misc/bits/syscall.h.
This adds a test to ensure that the problems fixed in the last several
patches do not recur. Each directory checks the headers that it
installs for two properties: first, each header must be compilable in
isolation, as both C and C++, under a representative combination of
language and library conformance levels; second, there is a blacklist
of identifiers that may not appear in any installed header, currently
consisting of the legacy BSD typedefs. (There is an exemption for the
headers that define those typedefs, and for the RPC headers. It may be
necessary to make this more sophisticated if we add more stuff to the
blacklist in the future.)
In order for this test to work correctly, every wrapper header
that actually defines something must guard those definitions with
#ifndef _ISOMAC. This is the existing mechanism used by the conform/
tests to tell wrapper headers not to define anything that the public
header wouldn't, and not to use anything from libc-symbols.h. conform/
only cares for headers that we need to check for standards conformance,
whereas this test applies to *every* header. (Headers in include/ that
are either installed directly, or are internal-use-only and do *not*
correspond to any installed header, are not affected.)
* scripts/check-installed-headers.sh: New script.
* Rules: In each directory that defines header files to be installed,
run check-installed-headers.sh on them as a special test.
* Makefile: Likewise for the headers installed at top level.
* include/aliases.h, include/alloca.h, include/argz.h
* include/arpa/nameser.h, include/arpa/nameser_compat.h
* include/elf.h, include/envz.h, include/err.h
* include/execinfo.h, include/fpu_control.h, include/getopt.h
* include/gshadow.h, include/ifaddrs.h, include/libintl.h
* include/link.h, include/malloc.h, include/mcheck.h
* include/mntent.h, include/netinet/ether.h
* include/nss.h, include/obstack.h, include/printf.h
* include/pty.h, include/resolv.h, include/rpc/auth.h
* include/rpc/auth_des.h, include/rpc/auth_unix.h
* include/rpc/clnt.h, include/rpc/des_crypt.h
* include/rpc/key_prot.h, include/rpc/netdb.h
* include/rpc/pmap_clnt.h, include/rpc/pmap_prot.h
* include/rpc/pmap_rmt.h, include/rpc/rpc.h
* include/rpc/rpc_msg.h, include/rpc/svc.h
* include/rpc/svc_auth.h, include/rpc/xdr.h
* include/rpcsvc/nis_callback.h, include/rpcsvc/nislib.h
* include/rpcsvc/yp.h, include/rpcsvc/ypclnt.h
* include/rpcsvc/ypupd.h, include/shadow.h
* include/stdio_ext.h, include/sys/epoll.h
* include/sys/file.h, include/sys/gmon.h, include/sys/ioctl.h
* include/sys/prctl.h, include/sys/profil.h
* include/sys/statfs.h, include/sys/sysctl.h
* include/sys/sysinfo.h, include/ttyent.h, include/utmp.h
* sysdeps/arm/nacl/include/bits/setjmp.h
* sysdeps/mips/include/sys/asm.h
* sysdeps/unix/sysv/linux/include/sys/sysinfo.h
* sysdeps/unix/sysv/linux/include/sys/timex.h
* sysdeps/x86/fpu/include/bits/fenv.h:
Add #ifndef _ISOMAC guard around internal declarations.
Add multiple-inclusion guard if not already present.
Profiling git's test suite, Linus noted [1] that a disproportionately
large amount of time was spent reading /proc/meminfo. This is done by
the glibc functions get_phys_pages and get_avphys_pages, but they only
need the MemTotal and MemFree fields, respectively. That same
information can be obtained with a single syscall, sysinfo, instead of
six: open, fstat, mmap, read, close, munmap. While sysinfo also
provides more than necessary, it does a lot less work than what the
kernel needs to do to provide the entire /proc/meminfo. Both strace -T
and in-app microbenchmarks shows that the sysinfo() approach is
roughly an order of magnitude faster.
sysinfo() is much older than what glibc currently requires, so I don't
think there's any reason to keep the old parsing code. Moreover, this
makes get_[av]phys_pages work even in the absence of /proc.
Linus noted that something as simple as 'bash -c "echo"' would trigger
the reading of /proc/meminfo, but gdb says that many more applications
than just bash are affected:
Starting program: /bin/bash "-c" "echo"
Breakpoint 1, __get_phys_pages () at ../sysdeps/unix/sysv/linux/getsysstats.c:283
283 ../sysdeps/unix/sysv/linux/getsysstats.c: No such file or directory.
(gdb) bt
So it seems that any application that uses qsort on a moderately sized
array will incur this cost (once), which is obviously proportionately
more expensive for lots of short-lived processes (such as the git test
suite).
[1] http://thread.gmane.org/gmane.linux.kernel/2019285
Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk>
* sysdeps/unix/sysv/linux/getsysstats.c (__get_phys_pages):
Use sysinfo system call instead of parsing /proc/meminfo.
* sysdeps/unix/sysv/linux/getsysstats.c (__get_avphys_pages):
Likewise.
Bug 14132 is removal of the old INTDEF/INTUSE system of *_internal
aliases as obsoleted by the hidden_proto / hidden_def system. Various
cases were cleaned up in 2012, but some remain. This patch removes
the use of this mechanism for __adjtimex.
Tested for x86_64 that stripped installed shared libraries are
unchanged by the patch.
[BZ #14132]
* sysdeps/unix/sysv/linux/include/sys/timex.h: New file.
* sysdeps/unix/sysv/linux/adjtime.c [!ADJTIMEX] (ADJTIMEX): Do not
use INTUSE.
[!ADJTIMEX] (INTUSE(__adjtimex)): Remove declaration.
* sysdeps/unix/sysv/linux/alpha/adjtime.c (__adjtimex_internal):
Remove alias.
(__adjtimex): Define using libc_hidden_ver.
* sysdeps/unix/sysv/linux/ntp_gettime.c (INTUSE(__adjtimex)):
Remove declaration.
(ntp_gettime): Call __adjtimex directly.
* sysdeps/unix/sysv/linux/ntp_gettimex.c (INTUSE(__adjtimex)):
Remove declaration.
(ntp_gettimex): Call __adjtimex directly.
* sysdeps/unix/sysv/linux/syscalls.list (adjtimex): Remove
__adjtimex_internal alias.
