Commit Graph

12004 Commits

Author SHA1 Message Date
H.J. Lu
77a8ae0948 i386: Use _CET_NOTRACK in memset-sse2-rep.S
* sysdeps/i386/i686/multiarch/memset-sse2-rep.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 08:04:12 -07:00
H.J. Lu
90d15dc577 i386: Use _CET_NOTRACK in strcat-sse2.S
* sysdeps/i386/i686/multiarch/strcat-sse2.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 08:03:15 -07:00
H.J. Lu
f1574581c7 i386: Use _CET_NOTRACK in strcpy-sse2.S
* sysdeps/i386/i686/multiarch/strcpy-sse2.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 08:02:26 -07:00
H.J. Lu
7fb613361c i386: Use _CET_NOTRACK in memcpy-ssse3.S
* sysdeps/i386/i686/multiarch/memcpy-ssse3.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 08:01:06 -07:00
H.J. Lu
0a899af097 i386: Use _CET_NOTRACK in memcpy-ssse3-rep.S
* sysdeps/i386/i686/multiarch/memcpy-ssse3-rep.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
	(BRANCH_TO_JMPTBL_ENTRY_TAIL): Likewise.
2018-07-18 08:00:09 -07:00
H.J. Lu
177824e232 i386: Use _CET_NOTRACK in memcmp-sse4.S
* sysdeps/i386/i686/multiarch/memcmp-sse4.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 07:59:04 -07:00
H.J. Lu
00e7b76a8f i386: Use _CET_NOTRACK in memset-sse2.S
* sysdeps/i386/i686/multiarch/memset-sse2.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 07:57:59 -07:00
H.J. Lu
7e119cd582 i386: Use _CET_NOTRACK in i686/memcmp.S
* sysdeps/i386/i686/memcmp.S (memcmp): Add _CET_NOTRACK before
	indirect jump to jump table.
2018-07-18 07:56:58 -07:00
H.J. Lu
03aaf49b68 x86-64: Use _CET_NOTRACK in memcpy-ssse3.S
* sysdeps/x86_64/multiarch/memcpy-ssse3.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
	(MEMCPY): Likewise.
2018-07-18 06:39:46 -07:00
H.J. Lu
811e9e52b2 x86-64: Use _CET_NOTRACK in memcpy-ssse3-back.S
* sysdeps/x86_64/multiarch/memcpy-ssse3-back.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
	(MEMCPY): Likewise.
2018-07-18 06:38:23 -07:00
H.J. Lu
8817df4265 x86-64: Use _CET_NOTRACK in strcmp-sse42.S
* sysdeps/x86_64/multiarch/strcmp-sse42.S (STRCMP_SSE42): Add
	_CET_NOTRACK before indirect jump to jump table.
2018-07-18 06:37:09 -07:00
H.J. Lu
921595d151 x86-64: Use _CET_NOTRACK in strcpy-sse2-unaligned.S
* sysdeps/x86_64/multiarch/strcpy-sse2-unaligned.S
	(BRANCH_TO_JMPTBL_ENTRY): Add _CET_NOTRACK before indirect jump
	to jump table.
2018-07-18 06:33:06 -07:00
H.J. Lu
4ef60d9597 x86_64: Use _CET_NOTRACK in strcmp.S
* sysdeps/x86_64/strcmp.S (STRCMP): Add _CET_NOTRACK before
	indirect jump to jump table.
2018-07-18 06:31:53 -07:00
H.J. Lu
be9ccd27c0 i386: Add _CET_ENDBR to indirect jump targets in add_n.S/sub_n.S
i386 add_n.S and sub_n.S use a trick to implment jump tables with LEA.
We can't use conditional branches nor normal jump tables since jump
table entries use EFLAGS set by jump table index.  This patch adds
_CET_ENDBR to indirect jump targets and adjust destination for
_CET_ENDBR.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	* sysdeps/i386/add_n.S: Include <sysdep.h>, instead of
	"sysdep.h".
	(__mpn_add_n): Save and restore %ebx if IBT is enabed.  Add
	_CET_ENDBR to indirect jump targets and adjust jump destination
	for _CET_ENDBR.
	* sysdeps/i386/i686/add_n.S: Include <sysdep.h>, instead of
	"sysdep.h".
	(__mpn_add_n): Save and restore %ebx if IBT is enabed.  Add
	_CET_ENDBR to indirect jump targets and adjust jump destination
	for _CET_ENDBR.
	* sysdeps/i386/sub_n.S: Include <sysdep.h>, instead of
	"sysdep.h".
	(__mpn_sub_n): Save and restore %ebx if IBT is enabed.  Add
	_CET_ENDBR to indirect jump targets and adjust jump destination
	for _CET_ENDBR.
2018-07-17 16:11:44 -07:00
H.J. Lu
5efc6777ad x86-64: Add _CET_ENDBR to STRCMP_SSE42
Add _CET_ENDBR to STRCMP_SSE42, which is called indirectly, to support
IBT.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	* sysdeps/x86_64/multiarch/strcmp-sse42.S (STRCMP_SSE42): Add
	_CET_ENDBR.
2018-07-17 16:08:47 -07:00
H.J. Lu
562837c002 x86: Add _CET_ENDBR to functions in dl-tlsdesc.S
Add _CET_ENDBR to functions in dl-tlsdesc.S, which are called indirectly,
to support IBT.

Tested on i686 and x86-64.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	* sysdeps/i386/dl-tlsdesc.S (_dl_tlsdesc_return): Add
	_CET_ENDBR.
	(_dl_tlsdesc_undefweak): Likewise.
	(_dl_tlsdesc_dynamic): Likewise.
	(_dl_tlsdesc_resolve_abs_plus_addend): Likewise.
	(_dl_tlsdesc_resolve_rel): Likewise.
	(_dl_tlsdesc_resolve_rela): Likewise.
	(_dl_tlsdesc_resolve_hold): Likewise.
	* sysdeps/x86_64/dl-tlsdesc.S (_dl_tlsdesc_return): Likewise.
	(_dl_tlsdesc_undefweak): Likewise.
	(_dl_tlsdesc_dynamic): Likewise.
	(_dl_tlsdesc_resolve_rela): Likewise.
	(_dl_tlsdesc_resolve_hold): Likewise.
2018-07-17 16:07:17 -07:00
H.J. Lu
124bcde683 x86: Add _CET_ENDBR to functions in crti.S
Add _CET_ENDBR to functions in crti.S, which are called indirectly, to
support IBT.

Tested on i686 and x86-64.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	* sysdeps/i386/crti.S (_init): Add _CET_ENDBR.
	(_fini): Likewise.
	* sysdeps/x86_64/crti.S (_init): Likewise.
	(_fini): Likewise.
2018-07-17 16:05:18 -07:00
H.J. Lu
ba2ea23d05 x86: Always include <dl-cet.h>/cet-tunables.h> for --enable-cet
Always include <dl-cet.h> and cet-tunables.h> when CET is enabled.
Otherwise, configure glibc with --enable-cet --disable-tunables will
fail to build.

	* sysdeps/x86/cpu-features.c: Always include <dl-cet.h> and
	cet-tunables.h> when CET is enabled.
2018-07-17 04:16:35 -07:00
H.J. Lu
f753fa7dea x86: Support IBT and SHSTK in Intel CET [BZ #21598]
Intel Control-flow Enforcement Technology (CET) instructions:

https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-en
forcement-technology-preview.pdf

includes Indirect Branch Tracking (IBT) and Shadow Stack (SHSTK).

GNU_PROPERTY_X86_FEATURE_1_IBT is added to GNU program property to
indicate that all executable sections are compatible with IBT when
ENDBR instruction starts each valid target where an indirect branch
instruction can land.  Linker sets GNU_PROPERTY_X86_FEATURE_1_IBT on
output only if it is set on all relocatable inputs.

On an IBT capable processor, the following steps should be taken:

1. When loading an executable without an interpreter, enable IBT and
lock IBT if GNU_PROPERTY_X86_FEATURE_1_IBT is set on the executable.
2. When loading an executable with an interpreter, enable IBT if
GNU_PROPERTY_X86_FEATURE_1_IBT is set on the interpreter.
  a. If GNU_PROPERTY_X86_FEATURE_1_IBT isn't set on the executable,
     disable IBT.
  b. Lock IBT.
3. If IBT is enabled, when loading a shared object without
GNU_PROPERTY_X86_FEATURE_1_IBT:
  a. If legacy interwork is allowed, then mark all pages in executable
     PT_LOAD segments in legacy code page bitmap.  Failure of legacy code
     page bitmap allocation causes an error.
  b. If legacy interwork isn't allowed, it causes an error.

GNU_PROPERTY_X86_FEATURE_1_SHSTK is added to GNU program property to
indicate that all executable sections are compatible with SHSTK where
return address popped from shadow stack always matches return address
popped from normal stack.  Linker sets GNU_PROPERTY_X86_FEATURE_1_SHSTK
on output only if it is set on all relocatable inputs.

On a SHSTK capable processor, the following steps should be taken:

1. When loading an executable without an interpreter, enable SHSTK if
GNU_PROPERTY_X86_FEATURE_1_SHSTK is set on the executable.
2. When loading an executable with an interpreter, enable SHSTK if
GNU_PROPERTY_X86_FEATURE_1_SHSTK is set on interpreter.
  a. If GNU_PROPERTY_X86_FEATURE_1_SHSTK isn't set on the executable
     or any shared objects loaded via the DT_NEEDED tag, disable SHSTK.
  b. Otherwise lock SHSTK.
3. After SHSTK is enabled, it is an error to load a shared object
without GNU_PROPERTY_X86_FEATURE_1_SHSTK.

To enable CET support in glibc, --enable-cet is required to configure
glibc.  When CET is enabled, both compiler and assembler must support
CET.  Otherwise, it is a configure-time error.

To support CET run-time control,

1. _dl_x86_feature_1 is added to the writable ld.so namespace to indicate
if IBT or SHSTK are enabled at run-time.  It should be initialized by
init_cpu_features.
2. For dynamic executables:
   a. A l_cet field is added to struct link_map to indicate if IBT or
      SHSTK is enabled in an ELF module.  _dl_process_pt_note or
      _rtld_process_pt_note is called to process PT_NOTE segment for
      GNU program property and set l_cet.
   b. _dl_open_check is added to check IBT and SHSTK compatibilty when
      dlopening a shared object.
3. Replace i386 _dl_runtime_resolve and _dl_runtime_profile with
_dl_runtime_resolve_shstk and _dl_runtime_profile_shstk, respectively if
SHSTK is enabled.

CET run-time control can be changed via GLIBC_TUNABLES with

$ export GLIBC_TUNABLES=glibc.tune.x86_shstk=[permissive|on|off]
$ export GLIBC_TUNABLES=glibc.tune.x86_ibt=[permissive|on|off]

1. permissive: SHSTK is disabled when dlopening a legacy ELF module.
2. on: IBT or SHSTK are always enabled, regardless if there are IBT or
SHSTK bits in GNU program property.
3. off: IBT or SHSTK are always disabled, regardless if there are IBT or
SHSTK bits in GNU program property.

<cet.h> from CET-enabled GCC is automatically included by assembly codes
to add GNU_PROPERTY_X86_FEATURE_1_IBT and GNU_PROPERTY_X86_FEATURE_1_SHSTK
to GNU program property.  _CET_ENDBR is added at the entrance of all
assembly functions whose address may be taken.  _CET_NOTRACK is used to
insert NOTRACK prefix with indirect jump table to support IBT.  It is
defined as notrack when _CET_NOTRACK is defined in <cet.h>.

	 [BZ #21598]
	* configure.ac: Add --enable-cet.
	* configure: Regenerated.
	* elf/Makefille (all-built-dso): Add a comment.
	* elf/dl-load.c (filebuf): Moved before "dynamic-link.h".
	Include <dl-prop.h>.
	(_dl_map_object_from_fd): Call _dl_process_pt_note on PT_NOTE
	segment.
	* elf/dl-open.c: Include <dl-prop.h>.
	(dl_open_worker): Call _dl_open_check.
	* elf/rtld.c: Include <dl-prop.h>.
	(dl_main): Call _rtld_process_pt_note on PT_NOTE segment.  Call
	_rtld_main_check.
	* sysdeps/generic/dl-prop.h: New file.
	* sysdeps/i386/dl-cet.c: Likewise.
	* sysdeps/unix/sysv/linux/x86/cpu-features.c: Likewise.
	* sysdeps/unix/sysv/linux/x86/dl-cet.h: Likewise.
	* sysdeps/x86/cet-tunables.h: Likewise.
	* sysdeps/x86/check-cet.awk: Likewise.
	* sysdeps/x86/configure: Likewise.
	* sysdeps/x86/configure.ac: Likewise.
	* sysdeps/x86/dl-cet.c: Likewise.
	* sysdeps/x86/dl-procruntime.c: Likewise.
	* sysdeps/x86/dl-prop.h: Likewise.
	* sysdeps/x86/libc-start.h: Likewise.
	* sysdeps/x86/link_map.h: Likewise.
	* sysdeps/i386/dl-trampoline.S (_dl_runtime_resolve): Add
	_CET_ENDBR.
	(_dl_runtime_profile): Likewise.
	(_dl_runtime_resolve_shstk): New.
	(_dl_runtime_profile_shstk): Likewise.
	* sysdeps/linux/x86/Makefile (sysdep-dl-routines): Add dl-cet
	if CET is enabled.
	(CFLAGS-.o): Add -fcf-protection if CET is enabled.
	(CFLAGS-.os): Likewise.
	(CFLAGS-.op): Likewise.
	(CFLAGS-.oS): Likewise.
	(asm-CPPFLAGS): Add -fcf-protection -include cet.h if CET
	is enabled.
	(tests-special): Add $(objpfx)check-cet.out.
	(cet-built-dso): New.
	(+$(cet-built-dso:=.note)): Likewise.
	(common-generated): Add $(cet-built-dso:$(common-objpfx)%=%.note).
	($(objpfx)check-cet.out): New.
	(generated): Add check-cet.out.
	* sysdeps/x86/cpu-features.c: Include <dl-cet.h> and
	<cet-tunables.h>.
	(TUNABLE_CALLBACK (set_x86_ibt)): New prototype.
	(TUNABLE_CALLBACK (set_x86_shstk)): Likewise.
	(init_cpu_features): Call get_cet_status to check CET status
	and update dl_x86_feature_1 with CET status.  Call
	TUNABLE_CALLBACK (set_x86_ibt) and TUNABLE_CALLBACK
	(set_x86_shstk).  Disable and lock CET in libc.a.
	* sysdeps/x86/cpu-tunables.c: Include <cet-tunables.h>.
	(TUNABLE_CALLBACK (set_x86_ibt)): New function.
	(TUNABLE_CALLBACK (set_x86_shstk)): Likewise.
	* sysdeps/x86/sysdep.h (_CET_NOTRACK): New.
	(_CET_ENDBR): Define if not defined.
	(ENTRY): Add _CET_ENDBR.
	* sysdeps/x86/dl-tunables.list (glibc.tune): Add x86_ibt and
	x86_shstk.
	* sysdeps/x86_64/dl-trampoline.h (_dl_runtime_resolve): Add
	_CET_ENDBR.
	(_dl_runtime_profile): Likewise.
2018-07-16 14:08:27 -07:00
Rogerio Alves
52b2a80fae powerpc64: Always restore TOC on longjmp [BZ #21895]
This patch changes longjmp to always restore the TOC pointer (r2 register)
to the caller frame on powerpc64 and powerpc64le.  This is related to bug
21895 that reports a situation where you have a static longjmp to a
shared object file.

	[BZ #21895]
	* sysdeps/powerpc/powerpc64/__longjmp-common.S: Remove condition code for
	restoring r2 in longjmp.
	* sysdeps/powerpc/powerpc64/Makefile: Added tst-setjmp-bug21895-static to
	test list.
	Added rules to build test tst-setjmp-bug21895-static.
	Added module setjmp-bug21895 and rules to build a shared object from it.
	* sysdeps/powerpc/powerpc64/setjmp-bug21895.c: New test file.
	* sysdeps/powerpc/powerpc64/tst-setjmp-bug21895-static.c: New test file.

Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
2018-07-16 16:08:41 -03:00
H.J. Lu
31e422ca96 x86_64: Undef SHADOW_STACK_POINTER_OFFSET last
Since SHADOW_STACK_POINTER_OFFSET is defined in jmp_buf-ssp.h, we must
undef SHADOW_STACK_POINTER_OFFSET after including <jmp_buf-ssp.h>.

	* sysdeps/unix/sysv/linux/x86_64/____longjmp_chk.S: Undef
	SHADOW_STACK_POINTER_OFFSET after including <jmp_buf-ssp.h>.
2018-07-15 08:48:54 -07:00
H.J. Lu
faaee1f07e x86: Support shadow stack pointer in setjmp/longjmp
Save and restore shadow stack pointer in setjmp and longjmp to support
shadow stack in Intel CET.  Use feature_1 in tcbhead_t to check if
shadow stack is enabled before saving and restoring shadow stack pointer.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	* sysdeps/i386/__longjmp.S: Include <jmp_buf-ssp.h>.
	(__longjmp): Restore shadow stack pointer if shadow stack is
	enabled, SHADOW_STACK_POINTER_OFFSET is defined and __longjmp
	isn't defined for __longjmp_cancel.
	* sysdeps/i386/bsd-_setjmp.S: Include <jmp_buf-ssp.h>.
	(_setjmp): Save shadow stack pointer if shadow stack is enabled
	and SHADOW_STACK_POINTER_OFFSET is defined.
	* sysdeps/i386/bsd-setjmp.S: Include <jmp_buf-ssp.h>.
	(setjmp): Save shadow stack pointer if shadow stack is enabled
	and SHADOW_STACK_POINTER_OFFSET is defined.
	* sysdeps/i386/setjmp.S: Include <jmp_buf-ssp.h>.
	(__sigsetjmp): Save shadow stack pointer if shadow stack is
	enabled and SHADOW_STACK_POINTER_OFFSET is defined.
	* sysdeps/unix/sysv/linux/i386/____longjmp_chk.S: Include
	<jmp_buf-ssp.h>.
	(____longjmp_chk): Restore shadow stack pointer if shadow stack
	is enabled and SHADOW_STACK_POINTER_OFFSET is defined.
	* sysdeps/unix/sysv/linux/x86/Makefile (gen-as-const-headers):
	Remove jmp_buf-ssp.sym.
	* sysdeps/unix/sysv/linux/x86_64/____longjmp_chk.S: Include
	<jmp_buf-ssp.h>.
	(____longjmp_chk): Restore shadow stack pointer if shadow stack
	is enabled and SHADOW_STACK_POINTER_OFFSET is defined.
	* sysdeps/x86/Makefile (gen-as-const-headers): Add
	jmp_buf-ssp.sym.
	* sysdeps/x86/jmp_buf-ssp.sym: New dummy file.
	* sysdeps/x86_64/__longjmp.S: Include <jmp_buf-ssp.h>.
	(__longjmp): Restore shadow stack pointer if shadow stack is
	enabled, SHADOW_STACK_POINTER_OFFSET is defined and __longjmp
	isn't defined for __longjmp_cancel.
	* sysdeps/x86_64/setjmp.S: Include <jmp_buf-ssp.h>.
	(__sigsetjmp): Save shadow stack pointer if shadow stack is
	enabled and SHADOW_STACK_POINTER_OFFSET is defined.
2018-07-14 05:59:53 -07:00
H.J. Lu
ebff9c5cfa x86: Rename __glibc_reserved1 to feature_1 in tcbhead_t [BZ #22563]
feature_1 has X86_FEATURE_1_IBT and X86_FEATURE_1_SHSTK bits for CET
run-time control.

CET_ENABLED, IBT_ENABLED and SHSTK_ENABLED are defined to 1 or 0 to
indicate that if CET, IBT and SHSTK are enabled.

<tls-setup.h> is added to set up thread-local data.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

	[BZ #22563]
	* nptl/pthread_create.c: Include <tls-setup.h>.
	(__pthread_create_2_1): Call tls_setup_tcbhead.
	* sysdeps/generic/tls-setup.h: New file.
	* sysdeps/x86/nptl/tls-setup.h: Likewise.
	* sysdeps/i386/nptl/tcb-offsets.sym (FEATURE_1_OFFSET): New.
	* sysdeps/x86_64/nptl/tcb-offsets.sym (FEATURE_1_OFFSET):
	Likewise.
	* sysdeps/i386/nptl/tls.h (tcbhead_t): Rename __glibc_reserved1
	to feature_1.
	* sysdeps/x86_64/nptl/tls.h (tcbhead_t): Likewise.
	* sysdeps/x86/sysdep.h (X86_FEATURE_1_IBT): New.
	(X86_FEATURE_1_SHSTK): Likewise.
	(CET_ENABLED): Likewise.
	(IBT_ENABLED): Likewise.
	(SHSTK_ENABLED): Likewise.
2018-07-14 05:56:46 -07:00
Florian Weimer
fd70af4552 Add the statx function 2018-07-10 16:14:45 +02:00
Adhemerval Zanella
b444e8f7cc Comment tst-ofdlocks-compat expected failure in some Linux releases
As pointed out in a libc-alpha thread [1], the misc/tst-ofdlocks-compat
may fail in some specific Linux releases.  This patch adds a comment
along with a link to discussion in the test source code.

No changes are expected.

	* sysdeps/unix/sysv/linux/tst-ofdlocks-compat.c: Add a comment about
	a kernel issue which lead to test failure in some cases.

[1] https://sourceware.org/ml/libc-alpha/2018-07/msg00243.html
2018-07-10 10:28:39 -03:00
Szabolcs Nagy
397c54c1af aarch64: add HWCAP_ATOMICS to HWCAP_IMPORTANT
This enables searching shared libraries in atomics/ when the hardware
supports LSE atomics of armv8.1 so one can provide optimized variants
of libraries in a portable way.

LSE atomics does not affect library abi, the new instructions can
interoperate with old ones.

I considered the earlier comments on the patch

https://sourceware.org/ml/libc-alpha/2018-04/msg00400.html
https://sourceware.org/ml/libc-alpha/2018-04/msg00625.html

It turns out that the way glibc dynamic linker decides on the search
path is not very flexible: it wants to use hwcap bits and associated
strings.  So some targets reuse hwcap bits for glibc internal purposes
to affect the search logic.  But hwcap is an interface with the kernel,
glibc should not allocate bits in it for its internal logic as that
limits future hwcap extensions and confusing to users who expect to see
hwcap bits in ifunc resolvers.  Instead of rewriting the dynamic linker
path logic (which affects all targets) this patch just uses the existing
mechanism, however this means that the path name has to be the hwcap
name "atomics" and cannot be changed to something more meaningful to
users.

It is hard to tell how much performance benefit this can give, in
principle armv8.1 atomics can be better optimized in the hardware, so it
can make a difference for synchronization heavy code.  On some systems
such multilib setup may be the only viable way to get optimized
libraries used.

	* sysdeps/unix/sysv/linux/aarch64/dl-procinfo.h (HWCAP_IMPORTANT): Add
	HWCAP_ATOMICS.
2018-07-06 15:58:16 +01:00
Szabolcs Nagy
d0cd798071 aarch64: Remove HWCAP_CPUID from HWCAP_IMPORTANT
This partially reverts

commit f82e9672ad
Author:     Siddhesh Poyarekar <siddhesh@sourceware.org>

    aarch64: Allow overriding HWCAP_CPUID feature check using HWCAP_MASK

The idea was to make it possible to disable cpuid based ifunc resolution
in glibc by changing the hwcap mask which the user could already control.

However the hwcap mask has an orthogonal role: it specifies additional
library search paths for the dynamic linker.  So "cpuid" got added to
the search paths when it was set in the default mask (HWCAP_IMPORTANT),
which is not useful behaviour, the hwcap masking should not be reused
in the cpu features code.

Meanwhile there is a tunable to set the cpu explicitly so it is possible
to disable the cpuid based dispatch without using a hwcap mask:

  GLIBC_TUNABLES=glibc.tune.cpu=generic

	* sysdeps/unix/sysv/linux/aarch64/cpu-features.c (init_cpu_features):
	Use dl_hwcap without masking.
	* sysdeps/unix/sysv/linux/aarch64/dl-procinfo.h (HWCAP_IMPORTANT):
	Remove HWCAP_CPUID.
2018-07-06 15:57:18 +01:00
Amit Pawar
bce5911b67 Use AVX_Fast_Unaligned_Load from Zen onwards.
From Zen onwards this will be enabled. It was disabled for the
Excavator case and will remain disabled.

Reviewd-by: Carlos O'Donell <carlos@redhat.com>
2018-07-06 09:55:36 -04:00
Maciej W. Rozycki
9449a295ad libc-abis: Define ABSOLUTE ABI [BZ #19818][BZ #23307]
Define a new ABSOLUTE ABI for static linker's use with EI_ABIVERSION
where correct absolute (SHN_ABS) symbol run-time load semantics is
required.  This way it can be ensured at static link time that a program
or DSO will not suffer from previous semantics where absolute symbols
were relocated by the base address, or symbols whose `st_value' is zero
silently ignored leading to a confusing "undefined symbol" error message
at load time, and instead "ELF file ABI version invalid" is printed with
old dynamic loaders, making it clear that there is an ABI version
incompatibility.

	[BZ #19818]
	[BZ #23307]
	* libc-abis (ABSOLUTE): New ABI.
	* sysdeps/unix/sysv/linux/mips/libc-abis (ABSOLUTE): New ABI.
	* NEWS: Mention the new ABI.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-07-05 18:06:43 +01:00
Florian Weimer
d6da5cb6a8 Add renameat2 function [BZ #17662]
The implementation falls back to renameat if renameat2 is not available
in the kernel (or in the kernel headers) and the flags argument is zero.
Without kernel support, a non-zero argument returns EINVAL, not ENOSYS.
This mirrors what the kernel does for invalid renameat2 flags.
2018-07-05 19:00:10 +02:00
Adhemerval Zanella
7279af007c Fix hurd expected fcntl version
Different than Linux, hurd does not need the OFD locks fix from
06ab719d30 (since OFD locks are current Linux specific). This in
turn allows hurd to not provide a fcntl compat symbol.

Checked on a i686-gnu with check-abi.

	* sysdeps/mach/hurd/i386/libc.abilist [GLIBC_2.28] (fcntl): Remove
	symbol.
2018-07-03 10:02:06 -03:00
Gabriel F. T. Gomes
b7b88cea41 ldbl-128ibm-compat: Add printf_size
Since the addition of the _Float128 API, strfromf128 and printf_size use
__printf_fp to print _Float128 values.  This is achieved by setting the
'is_binary128' member of the 'printf_info' structure to one.  Now that
the format of long double on powerpc64le is getting a third option, this
mechanism is reused for long double values that have binary128 format
(i.e.: when -mabi=ieeelongdouble).

This patch adds __printf_sizeieee128 as an exported symbol, but doesn't
provide redirections from printf_size, yet.  All redirections will be
installed in a future commit, once all other functions that print or
read long double values with binary128 format are ready.  In
__printf_fp, when 'is_binary128' is one, the floating-point argument is
treated as if it was of _Float128 type, regardless of the value of
'is_long_double', thus __printf_sizeieee128 sets 'is_binary128' to the
same value of 'is_long_double'.  Otherwise, double values would not be
printed correctly.

Tested for powerpc64le.
2018-07-02 10:51:01 -03:00
Szabolcs Nagy
2b445206a1 Use uint32_t sign in single precision math error handling functions
Ideally sign should be bool, but sometimes (e.g. in powf) it's more
efficient to pass a non-zero value than 1 to indicate that the sign
should be set.  The fixed size int is less ambigous than unsigned
long.

	* sysdeps/ieee754/flt-32/e_powf.c (__powf): Use uint32_t.
	(exp2f_inline): Likewise.
	* sysdeps/ieee754/flt-32/math_config.h (__math_oflowf): Likewise.
	(__math_uflowf): Likewise.
	(__math_may_uflowf): Likewise.
	(__math_divzerof): Likewise.
	(__math_invalidf): Likewise.
	* sysdeps/ieee754/flt-32/math_errf.c (xflowf): Likewise.
	(__math_oflowf): Likewise.
	(__math_uflowf): Likewise.
	(__math_may_uflowf): Likewise.
	(__math_divzerof): Likewise.
	(__math_invalidf): Likewise.
2018-07-02 09:29:04 +01:00
Carlos O'Donell
2827ab990a libc: Extend __libc_freeres framework (Bug 23329).
The __libc_freeres framework does not extend to non-libc.so objects.
This causes problems in general for valgrind and mtrace detecting
unfreed objects in both libdl.so and libpthread.so.  This change is
a pre-requisite to properly moving the malloc hooks out of malloc
since such a move now requires precise accounting of all allocated
data before destructors are run.

This commit adds a proper hook in libc.so.6 for both libdl.so and
for libpthread.so, this ensures that shm-directory.c which uses
freeit () to free memory is called properly.  We also remove the
nptl_freeres hook and fall back to using weak-ref-and-check idiom
for a loaded libpthread.so, thus making this process similar for
all DSOs.

Lastly we follow best practice and use explicit free calls for
both libdl.so and libpthread.so instead of the generic hook process
which has undefined order.

Tested on x86_64 with no regressions.

Signed-off-by: DJ Delorie <dj@redhat.com>
Signed-off-by: Carlos O'Donell <carlos@redhat.com>
2018-06-29 22:39:06 -04:00
Siddhesh Poyarekar
0aec4c1d18 aarch64,falkor: Use vector registers for memcpy
Vector registers perform better than scalar register pairs for copying
data so prefer them instead.  This results in a time reduction of over
50% (i.e. 2x speed improvemnet) for some smaller sizes for memcpy-walk.
Larger sizes show improvements of around 1% to 2%.  memcpy-random shows
a very small improvement, in the range of 1-2%.

	* sysdeps/aarch64/multiarch/memcpy_falkor.S (__memcpy_falkor):
	Use vector registers.
2018-06-29 22:45:59 +05:30
Siddhesh Poyarekar
ce76a5cb8d aarch64,falkor: Use vector registers for memmove
Vector registers perform much better for moves compared to pairs of
registers on falkor, so use them instead.  This results in a time
reduction of up to 50% (i.e. 2x improvement) for a lot of the smaller
sizes, i.e. up to 1K in memmove-walk.  Improvements for larger sizes are
smaller, at about 1%-2%.

	* sysdeps/aarch64/multiarch/memmove_falkor.S
	(__memcpy_falkor): Use vector registers.
2018-06-29 22:45:07 +05:30
Daniel Alvarez
c1f86a33ca getifaddrs: Don't return ifa entries with NULL names [BZ #21812]
A lookup operation in map_newlink could turn into an insert because of
holes in the interface part of the map.  This leads to incorrectly set
the name of the interface to NULL when the interface is not present
for the address being processed (most likely because the interface was
added between the RTM_GETLINK and RTM_GETADDR calls to the kernel).
When such changes are detected by the kernel, it'll mark the dump as
"inconsistent" by setting NLM_F_DUMP_INTR flag on the next netlink
message.

This patch checks this condition and retries the whole operation.
Hopes are that next time the interface corresponding to the address
entry is present in the list and correct name is returned.
2018-06-29 09:44:55 +02:00
Rajalakshmi Srinivasaraghavan
86a0f56158 ldbl-128ibm-compat: Introduce ieee128 symbols
This patch adds __*ieee128 symbols for strfrom, strtold, strtold_l, wcstold
and wcstold_l functions.  Redirection from *l to *ieee128 will be handled
in separate patch once we start building these new files.

2018-06-28  Rajalakshmi Srinivasaraghavan  <raji@linux.vnet.ibm.com>

	* sysdeps/ieee754/ldbl-128ibm-compat/Versions: Add __strfromieee128,
	 __strtoieee128, __strtoieee128_l,__wcstoieee128 and __wcstoieee128_l.
	* sysdeps/ieee754/ldbl-128ibm-compat/strfromf128.c: New file.
	* sysdeps/ieee754/ldbl-128ibm-compat/strtof128.c: New file.
	* sysdeps/ieee754/ldbl-128ibm-compat/strtof128_l.c: New file.
	* sysdeps/ieee754/ldbl-128ibm-compat/wcstof128.c: New file.
	* sysdeps/ieee754/ldbl-128ibm-compat/wcstof128_l.c: New file.
2018-06-28 13:57:50 +05:30
Florian Weimer
890c2ced35 gethostid (Linux variant): Switch to struct scratch_buffer [BZ #18023]
Previously, extend_alloca was used without alloca accounting,
which could have been problematic with large NSS results.
2018-06-27 17:55:38 +02:00
Adhemerval Zanella
95f8ae8c83 Revert hurd errno.h changes
Patch 06ab719d30 wrongly added mach errno.h changes.
2018-06-26 20:13:34 -03:00
Adhemerval Zanella
06ab719d30 Fix Linux fcntl OFD locks for non-LFS architectures (BZ#20251)
This patch fixes the OFD ("file private") locks for architectures that
support non-LFS flock definition (__USE_FILE_OFFSET64 not defined). The
issue in this case is both F_OFD_{GETLK,SETLK,SETLKW} and
F_{SET,GET}L{W}K64 expects a flock64 argument and when using old
F_OFD_* flags with a non LFS flock argument the kernel might interpret
the underlying data wrongly.  Kernel idea originally was to avoid using
such flags in non-LFS syscall, but since GLIBC uses fcntl with LFS
semantic as default it is possible to provide the functionality and
avoid the bogus struct kernel passing by adjusting the struct manually
for the required flags.

The idea follows other LFS interfaces that provide two symbols:

  1. A new LFS fcntl64 is added on default ABI with the usual macros to
     select it for FILE_OFFSET_BITS=64.

  2. The Linux non-LFS fcntl use a stack allocated struct flock64 for
     F_OFD_{GETLK,SETLK,SETLKW} copy the results on the user provided
     struct.

  3. Keep a compat symbol with old broken semantic for architectures
     that do not define __OFF_T_MATCHES_OFF64_T.

So for architectures which defines __USE_FILE_OFFSET64, fcntl64 will
aliased to fcntl and no adjustment would be required.  So to actually
use F_OFD_* with LFS support the source must be built with LFS support
(_FILE_OFFSET_BITS=64).

Also F_OFD_SETLKW command is handled a cancellation point, as for
F_SETLKW{64}.

Checked on x86_64-linux-gnu and i686-linux-gnu.

	[BZ #20251]
	* NEWS: Mention fcntl64 addition.
	* csu/check_fds.c: Replace __fcntl_nocancel by __fcntl64_nocancel.
	* login/utmp_file.c: Likewise.
	* sysdeps/posix/fdopendir.c: Likewise.
	* sysdeps/posix/opendir.c: Likewise.
	* sysdeps/unix/pt-fcntl.c: Likewise.
	* include/fcntl.h (__libc_fcntl64, __fcntl64,
	__fcntl64_nocancel_adjusted): New prototype.
	(__fcntl_nocancel_adjusted): Remove prototype.
	* io/Makefile (routines): Add fcntl64.
	(CFLAGS-fcntl64.c): New rule.
	* io/Versions [GLIBC_2.28] (fcntl64): New symbol.
	[GLIBC_PRIVATE] (__libc_fcntl): Rename to __libc_fcntl64.
	* io/fcntl.h (fcntl64): Add prototype and redirect if
	__USE_FILE_OFFSET64 is defined.
	* io/fcntl64.c: New file.
	* manual/llio.text: Add a note for which commands fcntl acts a
	cancellation point.
	* nptl/Makefile (CFLAGS-fcntl64.c): New rule.
	* sysdeps/mach/hurd/fcntl.c: Alias fcntl to fcntl64 symbols.
	* sysdeps/mach/hurd/i386/libc.abilist [GLIBC_2.28] (fcntl, fcntl64):
	New symbols.
	* sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Fix F_GETLK64,
	F_OFD_GETLK, F_SETLK64, F_SETLKW64, F_OFD_SETLK, and F_OFD_SETLKW for
	non-LFS case.
	* sysdeps/unix/sysv/linux/fcntl64.c: New file.
	* sysdeps/unix/sysv/linux/fcntl_nocancel.c (__fcntl_nocancel): Rename
	to __fcntl64_nocancel.
	(__fcntl_nocancel_adjusted): Rename to __fcntl64_nocancel_adjusted.
	* sysdeps/unix/sysv/linux/not-cancel.h (__fcntl_nocancel): Rename
	to __fcntl64_nocancel.
	* sysdeps/unix/sysv/linux/tst-ofdlocks.c: New file.
	* sysdeps/unix/sysv/linux/tst-ofdlocks-compat.c: Likewise.
	* sysdeps/unix/sysv/linux/Makefile (tests): Add tst-ofdlocks.
	(tests-internal): Add tst-ofdlocks-compat.
	* sysdeps/unix/sysv/linux/aarch64/libc.abilist [GLIBC_2.28]
	(fcntl64): New symbol.
	* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
	* sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/arm/libc.abilist [GLIBC_2.28] (fcntl,
	fcntl64): Likewise.
	* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/i386/libc.abilis: Likewise.
	* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
	Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist:
	Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
2018-06-26 13:22:53 -03:00
Florian Weimer
124e025864 Run thread shutdown functions in an explicit order
This removes the __libc_thread_subfreeres hook in favor of explict
calls.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>
2018-06-26 15:27:12 +02:00
Tulio Magno Quites Machado Filho
d93f4ff16b m68k: Reorganize log1p and significand implementations
Commit 5e79e0292b broke m68k after
s_significand.c became available in the build directory.  All m68k
implementations of log1p and significand were including s_significand.c
and stopped working after the inclusion of the the auto-generated file.

This patch reorganizes the implementation of log1p and significand for
m680x0 in order to avoid hitting this problem.

	* sysdeps/m68k/m680x0/fpu/s_log1p.c: Set as the generic file for
	all log1p and significand functions on m680x0.
	* sysdeps/m68k/m680x0/fpu/s_log1pf.c: Include s_log1p.c instead
	of s_significand.c..
	* sysdeps/m68k/m680x0/fpu/s_log1pl.c: Likewise.
	* sysdeps/m68k/m680x0/fpu/s_significandf.c: Likewise.
	* sysdeps/m68k/m680x0/fpu/s_significandl.c: Likewise.
	* sysdeps/m68k/m680x0/fpu/s_significand.c: Move all the code to
	s_log1p.c and include it..

Signed-off-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
2018-06-22 21:34:28 -03:00
Florian Weimer
f496b28e61 math: Set 387 and SSE2 rounding mode for tgamma on i386 [BZ #23253]
Previously, only the SSE2 rounding mode was set, so the assembler
implementations using 387 were not following the expecting rounding
mode.
2018-06-21 08:04:29 +02:00
Tulio Magno Quites Machado Filho
209ae17c60 ldbl-128ibm-compat: Create libm-alias-float128.h
Add a new libm-alias-float128.h in order to provide the __*ieee128
aliases for the existing *f128 that do not have a globally exported
symbol.

	* sysdeps/ieee754/ldbl-128ibm-compat/Versions: New file.
	* sysdeps/ieee754/ldbl-128ibm-compat/libm-alias-float128.h: New file.
2018-06-20 19:02:07 -03:00
Tulio Magno Quites Machado Filho
5e79e0292b Add a generic significand implementation
Create a template for significand.

	* math/Makefile (libm-calls): Move s_significandF to...
	(gen-libm-calls): ... here.
	* math/s_significand_template.c: New file.
	* math/s_significand.c: Removed.
	* math/s_significandf.c: Removed.
	* math/s_significandl.c: Removed.
	* sysdeps/ieee754/ldbl-opt/s_significand.c: Removed.
	* sysdeps/ieee754/ldbl-opt/s_significandl.c: Removed.

Signed-off-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
2018-06-20 18:15:06 -03:00
Tulio Magno Quites Machado Filho
badba23cac Move declare_mgen_finite_alias definition
Move declare_mgen_finite_alias, declare_mgen_finite_alias_s and
declare_mgen_finite_alias_x to a shared place in order to reuse them in
other files that also declare _finite aliases.

	* math/e_exp2_template.c (declare_mgen_finite_alias,
	declare_mgen_finite_alias_s, declare_mgen_finite_alias_x): Move to...
	* sysdeps/generic/math-type-macros.h (declare_mgen_finite_alias,
	declare_mgen_finite_alias_s, declare_mgen_finite_alias_x): ... here.

Signed-off-by: Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
2018-06-20 17:42:47 -03:00
Samuel Thibault
152fbfb3ad hurd: fix localplt.data format
* sysdeps/mach/hurd/i386/localplt.data: Drop empty line.
2018-06-19 02:37:23 +02:00
Samuel Thibault
c77e4dc0f5 hurd: Fix "Missing required PLT reference"
* sysdeps/mach/hurd/localplt.data: Move to...
	* sysdeps/mach/hurd/i386/localplt.data: new file. Add + REL
	R_386_GLOB_DAT like on Linux i386.
2018-06-19 02:32:41 +02:00
Joseph Myers
f2857da7cd Add SHM_STAT_ANY from Linux 4.17 to bits/shm.h.
Linux 4.17 adds a SHM_STAT_ANY constant (ipcs command).  This patch
adds it to the relevant bits/shm.h headers.

Tested for x86_64.

	* sysdeps/unix/sysv/linux/alpha/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): New macro.
	* sysdeps/unix/sysv/linux/arm/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/generic/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/hppa/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/ia64/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/mips/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/powerpc/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/s390/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/sh/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/sparc/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
	* sysdeps/unix/sysv/linux/x86/bits/shm.h [__USE_MISC]
	(SHM_STAT_ANY): Likewise.
2018-06-18 13:37:57 +00:00