I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
The accept4, recvmmsg and sendmmsg functions had macros
__ASSUME_*_SYSCALL_WITH_SOCKETCALL. Before we could assume kernels
with the relevant functionality, these macros represented the
conditions under which, on a socketcall architecture, glibc could just
call the syscall unconditionally and not have to deal with socketcall
at all for those functions, because if the syscall didn't work for
them the socketcall call wouldn't either.
Now we can assume kernels with the relevant functionality, the only
question is whether we can assume the syscall is present; if not, we
are on a socketcall architecture and just use socketcall instead.
Thus, this patch removes the macros that are no longer necessary, and
simplifies the code for accept4, recvmmsg and sendmmsg to use the same
logic as the other C implementations of socket functions that may use
a syscall or socketcall depending on kernel support.
Tested for x86_64 and x86.
* sysdeps/unix/sysv/linux/accept4.c (accept4): Use syscall if
[__ASSUME_ACCEPT4_SYSCALL], otherwise socketcall.
* sysdeps/unix/sysv/linux/recvmmsg.c (recvmmsg): Use syscall if
[__ASSUME_RECVMMSG_SYSCALL], otherwise socketcall.
* sysdeps/unix/sysv/linux/sendmmsg.c (__sendmmsg): Use syscall if
[__ASSUME_SENDMMSG_SYSCALL], otherwise socketcall.
* sysdeps/unix/sysv/linux/kernel-features.h
(__ASSUME_ACCEPT4_SYSCALL): Move to general list of macros for
socket syscalls.
(__ASSUME_RECVMMSG_SYSCALL): Likewise.
(__ASSUME_SENDMMSG_SYSCALL): Likewise.
* sysdeps/unix/sysv/linux/i386/kernel-features.h
(__ASSUME_RECVMMSG_SYSCALL_WITH_SOCKETCALL): Remove.
(__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL): Likewise.
* sysdeps/unix/sysv/linux/microblaze/kernel-features.h
(__ASSUME_RECVMMSG_SYSCALL_WITH_SOCKETCALL): Remove.
* sysdeps/unix/sysv/linux/powerpc/kernel-features.h
(__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL): Likewise.
* sysdeps/unix/sysv/linux/sh/kernel-features.h
(__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL): Likewise.
* sysdeps/unix/sysv/linux/sparc/kernel-features.h
(__ASSUME_ACCEPT4_SYSCALL_WITH_SOCKETCALL): Likewise.
(__ASSUME_RECVMMSG_SYSCALL_WITH_SOCKETCALL): Likewise.
(__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL): Likewise.
Now we can assume a kernel with sendmmsg support, this patch
simplifies the implementation to be similar to that for accept4:
either using socketcall or the syscall according to whether the
syscall is known to be available, without further fallback
implementations. The __ASSUME_SENDMMSG macro is kept (now defined
unconditionally), since it's used in resolv/res_send.c.
Tested for x86_64 and x86.
* sysdeps/unix/sysv/linux/kernel-features.h
(__ASSUME_SENDMMSG_SYSCALL): Define unconditionally.
(__ASSUME_SENDMMSG): Likewise.
(__ASSUME_SENDMMSG_SOCKETCALL): Remove macro.
* sysdeps/unix/sysv/linux/sendmmsg.c (__sendmmsg): Define using
sendmmsg syscall if that can be assumed to be present, socketcall
otherwise, with no fallback for runtime failure.
After some discussion in libc-alpha about this POSIX compliance fix, I see
that GLIBC should indeed revert back to previous definition of msghdr and
cmsghdr and implementation of sendmsg, recvmsg, sendmmsg, recvmmsg due some
reasons:
* The possible issue where the syscalls wrapper add the compatibility
layer is quite limited in scope and range. And kernel current
also add some limits to the values on the internal msghdr and
cmsghdr fields:
- msghdr::msg_iovlen larger than UIO_MAXIOV (1024) returns
EMSGSIZE.
- msghdr::msg_controllen larger than INT_MAX returns ENOBUFS.
* There is a small performance hit for recvmsg/sendmsg/recmmsg which
is neglectable, but it is a big hit for sendmmsg since now instead
of calling the syscall for the packed structure, GLIBC is calling
multiple sendmsg. This defeat the very existence of the syscall.
* It currently breaks libsanitizer build on GCC [1] (I fixed on compiler-rt).
However the fix is incomplete because it does add any runtime check
since libsanitizer currently does not have any facility to intercept
symbols with multiple version [2].
This, along with incorret dlsym/dlvsym return for versioned symbol due
another bug [3], makes hard to interpose versioned symbols.
Also, current approach of fixing GCC PR#71445 leads to half-baked
solutions without versioned symbol interposing.
This patch basically reverts commits 2f0dc39029, 222c2d7f43,
af7f7c7ec8. I decided to not revert abf29edd4a (Adjust
kernel-features.h defaults for recvmsg and sendmsg) mainly because it
does not really address the POSIX compliance original issue and also
adds some cleanups.
Tested on x86, i386, s390, s390x, aarch64, and powerpc64le.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71445
[2] https://github.com/google/sanitizers/issues/628
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=14932
* conform/data/sys/socket.h-data (msghdr.msg_iovlen): Add xfail-.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
* nptl/Makefile (libpthread-routines): Remove ptw-oldrecvmsg and
ptw-oldsendmsg.
(CFLAGS-oldrecvmsg.c): Remove rule.
(CFLAGS-oldsendmsg.c): Likewise.
(CFLAGS-recvmsg.c): Add rule.
(CFLAGS-sendmsg.c): Likewise.
* sysdeps/unix/sysv/linux/Makefile (sysdep_routines): Remove
oldrecvmsg, oldsendmsg, oldrecvmmsg, oldsendmmsg.
(CFLAGS-recvmsg.c): Remove rule.
(CFLAGS-sendmsg.c): Likewise.
(CFLAGS-oldrecvmsg.c): Likewise.
(CFLAGS-oldsendmsg.c): Likewise.
(CFLAGS-recvmmsg.c): Likewise.
* sysdeps/unix/sysv/linux/bits/socket.h (msghdr.msg_iovlen): Revert
to kernel defined interfaces.
(msghdr.msg_controllen): Likewise.
(cmsghdr.cmsg_len): Likewise.
(msghdr.__glibc_reserved1): Remove member.
(msghdr.__glibc_reserved2): Likewise.
(cmsghdr.__glibc_reserved1): Likewise.
* sysdeps/unix/sysv/linux/oldrecvmmsg.c: Remove file.
* sysdeps/unix/sysv/linux/oldrecvmsg.c: Likewise.
* sysdeps/unix/sysv/linux/oldsendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/oldsendmsg.c: Likewise.
* sysdeps/unix/sysv/linux/recvmmsg.c: Revert back to previous
version.
* sysdeps/unix/sysv/linux/recvmsg.c: Likewise.
* sysdeps/unix/sysv/linux/sendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/sendmsg.c: Likewise.
* sysdeps/unix/sysv/linux/aarch64/Versions [libc] (GLIBC_2.24):
Remove recvmsg and sendmsg.
* sysdeps/unix/sysv/linux/alpha/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/hppa/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/i386/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/m68k/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/microblaze/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/Versions
[libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/nios2/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/Versions
[libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/sh/Versions [libc] (GLIBC_2.24): Likewise.
* sysdeps/unix/sysv/linux/sparc/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/tile/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/x86_64/Versions [libc] (GLIBC_2.24):
Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/Versions: Remove file
* sysdeps/unix/sysv/linux/x86_64/64/Versions: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/Versions: Likewise.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Remove new 2.24
version for {recv,send,recm,sendm}msg.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
POSIX specifies that both msghdr::msg_iovlen and msghdr::msg_controllen
to be of size int and socklen_t respectively, however Linux implements
it as a size_t. So for 64-bits architecture where sizeof of size_t is
larger than socklen_t, both sendmmsg and recvmmsg need to adjust the
mmsghdr::msg_hdr internal fields before issuing the syscall itself.
This patch fixes it by operating on the padding if it the case.
For recvmmsg, the most straightfoward case, only zero padding the fields
is suffice. However, for sendmmsg, where adjusting the buffer is out
of the contract (since it may point to a read-only data), the function
is rewritten to use sendmsg instead (which from previous patch
allocates a temporary msghdr to operate on).
Also for 64-bit ports that requires it, a new recvmmsg and sendmmsg
compat version is created (which uses size_t for both cmsghdr::cmsg_len
and internal
Tested on x86_64, i686, aarch64, armhf, and powerpc64le.
* sysdeps/unix/sysv/linux/Makefile
[$(subdir) = socket] (sysdep_routines): Add oldrecvmmsg and
oldsendmmsg.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Add recvmmsg and
sendmmsg.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/oldrecvmmsg.c: New file.
* sysdeps/unix/sysv/linux/oldsendmmsg.c: Likewise.
* sysdeps/unix/sysv/linux/recvmmsg.c (__recvmmsg): Adjust msghdr
iovlen and controllen fields to adjust to POSIX specification.
* sysdeps/unix/sysv/linux/sendmmsg.c (__sendmmsg): Likewise.
This patch removes the socket.S implementation for all ports and replace
it by a C implementation using socketcall. For ports that implement
the syscall directly, there is no change.
The patch idea is to simplify the socket function implementation that
uses the socketcall to be based on C implemetation instead of a pseudo
assembly implementation with arch specific parts. The patch then remove
the assembly implementatation for the ports which uses socketcall
(i386, microblaze, mips, powerpc, sparc, m68k, s390 and sh).
I have cross-build GLIBC for afore-mentioned ports and tested on both
i386 and ppc32 without regressions.
Similar to the issues for accept4 and recvmmsg, __ASSUME_SENDMMSG is
also confused about whether it relates to function availability or
socketcall operation availability, and the conditions for the
definition are always wrong (sendmmsg appeared in Linux kernel 3.0,
not 2.6.39); this is now bug 16611.
This patch splits the macro into separate macros like those for
accept4 and recvmmsg, defining them for appropriate kernel versions.
Tested x86_64, including that disassembly of the installed shared
libraries is unchanged by this patch.
[BZ #16611]
* sysdeps/unix/sysv/linux/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030000 && __ASSUME_SOCKETCALL]
(__ASSUME_SENDMMSG_SOCKETCALL): Define.
[__LINUX_KERNEL_VERSION >= 0x030000 && (__i386__ || __x86_64__ ||
__powerpc__ || __sh__ || __sparc__)] (__ASSUME_SENDMMSG_SYSCALL):
Likewise.
[__i386__ || __powerpc__ || __sh__ || __sparc__]
(__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL): Likewise.
[__ASSUME_SENDMMSG_SOCKETCALL || __ASSUME_SENDMMSG_SYSCALL]
(__ASSUME_SENDMMSG): Define instead of using previous
[__LINUX_KERNEL_VERSION >= 0x020627] condition.
* sysdeps/unix/sysv/linux/aarch64/kernel-features.h
(__ASSUME_SENDMMSG_SYSCALL): Define.
* sysdeps/unix/sysv/linux/alpha/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030200] (__ASSUME_SENDMMSG_SYSCALL):
Likewise.
* sysdeps/unix/sysv/linux/arm/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030000] (__ASSUME_SENDMMSG_SYSCALL):
Likewise.
* sysdeps/unix/sysv/linux/ia64/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030000] (__ASSUME_SENDMMSG_SYSCALL):
Likewise.
* sysdeps/unix/sysv/linux/internal_sendmmsg.S [__ASSUME_SOCKETCALL
&& !__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL &&
!__ASSUME_SENDMMSG_SYSCALL] (__NR_sendmmsg): Undefine.
[__ASSUME_SENDMMSG]: Change conditionals to
[__ASSUME_SENDMMSG_SOCKETCALL].
* sysdeps/unix/sysv/linux/microblaze/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030300] (__ASSUME_SENDMMSG_SYSCALL):
Define.
* sysdeps/unix/sysv/linux/mips/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030100] (__ASSUME_SENDMMSG_SYSCALL):
Likewise.
* sysdeps/unix/sysv/linux/sendmmsg.c [__ASSUME_SOCKETCALL &&
!__ASSUME_SENDMMSG_SYSCALL_WITH_SOCKETCALL &&
!__ASSUME_SENDMMSG_SYSCALL] (__NR_sendmmsg): Undefine.
[!__ASSUME_SENDMMSG]: Change conditional to
[!__ASSUME_SENDMMSG_SOCKETCALL].
* sysdeps/unix/sysv/linux/tile/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030000] (__ASSUME_SENDMMSG_SYSCALL):
Define.
* sysdeps/unix/sysv/linux/hppa/kernel-features.h
[__LINUX_KERNEL_VERSION >= 0x030100] (__ASSUME_SENDMMSG_SYSCALL):
Define.
