Commit Graph

90 Commits

Author SHA1 Message Date
Rajalakshmi Srinivasaraghavan
6850e9c6ba powerpc: Replace lxvd2x/stxvd2x with lvx/stvx in P7's memcpy/memmove
POWER9 DD2.1 and earlier has an issue where some cache inhibited
vector load traps to the kernel, causing a performance degradation.  To
handle this in memcpy and memmove, lvx/stvx is used for aligned
addresses instead of lxvd2x/stxvd2x.

Reference: https://patchwork.ozlabs.org/patch/814059/

	* sysdeps/powerpc/powerpc64/power7/memcpy.S: Replace
	lxvd2x/stxvd2x with lvx/stvx.
	* sysdeps/powerpc/powerpc64/power7/memmove.S: Likewise.

Reviewed-by: Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>

(cherry picked from commit 63da5cd4a0)
2017-11-21 22:32:20 +05:30
Zack Weinberg
af85385f31 Use locale_t, not __locale_t, throughout glibc
<locale.h> is specified to define locale_t in POSIX.1-2008, and so are
all of the headers that define functions that take locale_t arguments.
Under _GNU_SOURCE, the additional headers that define such functions
have also always defined locale_t.  Therefore, there is no need to use
__locale_t in public function prototypes, nor in any internal code.

	* ctype/ctype-c99_l.c, ctype/ctype.h, ctype/ctype_l.c
	* include/monetary.h, include/stdlib.h, include/time.h
	* include/wchar.h, locale/duplocale.c, locale/freelocale.c
	* locale/global-locale.c, locale/langinfo.h, locale/locale.h
	* locale/localeinfo.h, locale/newlocale.c
	* locale/nl_langinfo_l.c, locale/uselocale.c
	* localedata/bug-usesetlocale.c, localedata/tst-xlocale2.c
	* stdio-common/vfscanf.c, stdlib/monetary.h, stdlib/stdlib.h
	* stdlib/strfmon_l.c, stdlib/strtod_l.c, stdlib/strtof_l.c
	* stdlib/strtol.c, stdlib/strtol_l.c, stdlib/strtold_l.c
	* stdlib/strtoll_l.c, stdlib/strtoul_l.c, stdlib/strtoull_l.c
	* string/strcasecmp.c, string/strcoll_l.c, string/string.h
	* string/strings.h, string/strncase.c, string/strxfrm_l.c
	* sysdeps/ieee754/float128/strtof128_l.c
	* sysdeps/ieee754/float128/wcstof128.c
	* sysdeps/ieee754/float128/wcstof128_l.c
	* sysdeps/ieee754/ldbl-128ibm/strtold_l.c
	* sysdeps/ieee754/ldbl-64-128/strtold_l.c
	* sysdeps/ieee754/ldbl-opt/nldbl-compat.c
	* sysdeps/ieee754/ldbl-opt/nldbl-strfmon_l.c
	* sysdeps/ieee754/ldbl-opt/nldbl-strtold_l.c
	* sysdeps/ieee754/ldbl-opt/nldbl-wcstold_l.c
	* sysdeps/powerpc/powerpc32/power7/strcasecmp.S
	* sysdeps/powerpc/powerpc64/power7/strcasecmp.S
	* sysdeps/x86_64/strcasecmp_l-nonascii.c
	* sysdeps/x86_64/strncase_l-nonascii.c, time/strftime_l.c
	* time/strptime_l.c, time/time.h, wcsmbs/mbsrtowcs_l.c
	* wcsmbs/wchar.h, wcsmbs/wcscasecmp.c, wcsmbs/wcsncase.c
	* wcsmbs/wcstod.c, wcsmbs/wcstod_l.c, wcsmbs/wcstof.c
	* wcsmbs/wcstof_l.c, wcsmbs/wcstol_l.c, wcsmbs/wcstold.c
	* wcsmbs/wcstold_l.c, wcsmbs/wcstoll_l.c, wcsmbs/wcstoul_l.c
	* wcsmbs/wcstoull_l.c, wctype/iswctype_l.c
	* wctype/towctrans_l.c, wctype/wcfuncs_l.c
	* wctype/wctrans_l.c, wctype/wctype.h, wctype/wctype_l.c:
	Change all uses of __locale_t to locale_t.
2017-06-20 20:30:06 -04:00
Alan Modra
d5b411854f PowerPC64 ENTRY_TOCLESS
A number of functions in the sysdeps/powerpc/powerpc64/ tree don't use
or change r2, yet declare a global entry that sets up r2.  This patch
fixes that problem, and consolidates the ENTRY and EALIGN macros.

	* sysdeps/powerpc/powerpc64/sysdep.h: Formatting.
	(NOPS, ENTRY_3): New macros.
	(ENTRY): Rewrite.
	(ENTRY_TOCLESS): Define.
	(EALIGN, EALIGN_W_0, EALIGN_W_1, EALIGN_W_2, EALIGN_W_4, EALIGN_W_5,
	EALIGN_W_6, EALIGN_W_7, EALIGN_W_8): Delete.
	* sysdeps/powerpc/powerpc64/a2/memcpy.S: Replace EALIGN with ENTRY.
	* sysdeps/powerpc/powerpc64/dl-trampoline.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_ceil.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_ceilf.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_floor.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_floorf.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_nearbyint.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_nearbyintf.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_rint.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_rintf.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_round.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_roundf.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_trunc.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_truncf.S: Likewise.
	* sysdeps/powerpc/powerpc64/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/fpu/s_finite.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/fpu/s_isinf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/fpu/s_isnan.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strstr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/e_expf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_cosf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_sinf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strcasestr.S: Likewise.
	* sysdeps/powerpc/powerpc64/addmul_1.S: Use ENTRY_TOCLESS.
	* sysdeps/powerpc/powerpc64/cell/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_copysign.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_copysignl.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_fabsl.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_isnan.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_llrint.S: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/s_llrintf.S: Likewise.
	* sysdeps/powerpc/powerpc64/lshift.S: Likewise.
	* sysdeps/powerpc/powerpc64/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/mul_1.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/memcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_ceil.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_ceilf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_floor.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_floorf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_llround.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_round.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_roundf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_trunc.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5+/fpu/s_truncf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power5/fpu/s_isnan.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6/fpu/s_copysign.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6/fpu/s_isnan.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6x/fpu/s_isnan.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6x/fpu/s_llrint.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6x/fpu/s_llround.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/add_n.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memmove.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/mempcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memrchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/rawmemchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strcasecmp.S (strcasecmp_l):
	Likewise.
	* sysdeps/powerpc/powerpc64/power7/strchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strchrnul.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strncpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strnlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strrchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_finite.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_isinf.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_isnan.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_llrint.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/fpu/s_llround.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/memcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strncpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strnlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strrchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strspn.S: Likewise.
	* sysdeps/powerpc/powerpc64/power9/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power9/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/strchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/strlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/ppc-mcount.S: Store LR earlier.  Don't
	add nop when SHARED.
	* sysdeps/powerpc/powerpc64/start.S: Fix comment.
	* sysdeps/powerpc/powerpc64/multiarch/strrchr-power8.S (ENTRY): Don't
	define.
	(ENTRY_TOCLESS): Define.
	* sysdeps/powerpc/powerpc32/sysdep.h (ENTRY_TOCLESS): Define.
	* sysdeps/powerpc/fpu/s_fma.S: Use ENTRY_TOCLESS.
	* sysdeps/powerpc/fpu/s_fmaf.S: Likewise.
2017-06-14 10:45:50 +09:30
Alan Modra
de7ee73d6f PowerPC64 strncpy, stpncpy and strstr fixes
Makes __stpncpy_power8 call __memset_power8 directly rather than via an
IFUNC.  Fixes a missing _mcount, and removes some redundant NOPS.  The
*_is_local defines are also used in a followup patch.

	* sysdeps/powerpc/powerpc64/multiarch/strncpy-power7.S: Define
	MEMSET_is_local.
	* sysdeps/powerpc/powerpc64/multiarch/strncpy-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/stpncpy-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/stpncpy-power8.S: Likewise.
	Define MEMSET.
	* sysdeps/powerpc/powerpc64/multiarch/strstr-power7.S: Define
	STRLEN_is_local, STRNLEN_is_local, and STRCHR_is_local.
	* sysdeps/powerpc/powerpc64/power7/strstr.S: Likewise.  Don't add
	nop after local calls.
	* sysdeps/powerpc/powerpc64/power7/strncpy.S: Define MEMSET_is_local.
	Don't add nop after local call.
	* sysdeps/powerpc/powerpc64/power8/strncpy.S: Likewise.  Add missing
	CALL_MCOUNT.
2017-06-14 10:44:59 +09:30
Wainer dos Santos Moschetta
b6a6622209 powerpc: refactor memcmp and memmove IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/memcmp-power4.S: Define the
	implementation-specific function name and remove unneeded
	macros definition.
	* sysdeps/powerpc/powerpc64/multiarch/memcmp-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memmove-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/memcmp.S: Set a default function
	name if not defined and pass as parameter to macros accordingly.
	* sysdeps/powerpc/powerpc64/power7/memcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memmove.S: Likewise.
2017-04-11 17:13:56 -03:00
Wainer dos Santos Moschetta
72fd128a08 powerpc: refactor memcpy and mempcpy IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/memcpy-a2.S: Define the
	implementation-specific function name and remove unneeded
	macros definition.
	* sysdeps/powerpc/powerpc64/multiarch/memcpy-cell.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memcpy-power4.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memcpy-power6.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memcpy-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memcpy-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/mempcpy-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/a2/memcpy.S: Set a default function
	name if not defined and pass as parameter to macros accordingly.
	* sysdeps/powerpc/powerpc64/cell/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/mempcpy.S: Likewise.
2017-04-11 17:13:56 -03:00
Wainer dos Santos Moschetta
ff06a55aef powerpc: refactor memchr, memrchr, and rawmemchr IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/memchr-power7.S: Define the
	implementation-specific function name and remove unneeded macros
	definition.
	* sysdeps/powerpc/powerpc64/multiarch/memrchr-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/rawmemchr-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memchr.S: Set a default
	function name if not defined and pass as parameter to macros
	accordingly.
	* sysdeps/powerpc/powerpc64/power7/memrchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/rawmemchr.S: Likewise.
2017-04-11 17:13:55 -03:00
Wainer dos Santos Moschetta
18e0054bf7 powerpc: refactor memset IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/memset-power4.S: Define the
	implementation-specific function name and remove unneeded macros
	definition.
	* sysdeps/powerpc/powerpc64/multiarch/memset-power6.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memset-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memset-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/memset-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/memset.S: Set a default function name if
	not defined and pass as parameter to macros accordingly.
	* sysdeps/powerpc/powerpc64/power4/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power6/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/memset.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/memset.S: Likewise.
2017-04-11 17:13:55 -03:00
Wainer dos Santos Moschetta
f0748b70a8 powerpc: refactor strcasestr and strstr IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/strcasestr-power8.S: Define the
	strcasestr implementation name and remove unneeded macros definition.
	* sysdeps/powerpc/powerpc64/multiarch/strstr-power7.S: Define
	strstr implementation name and remove unneeded macros definition.
	* sysdeps/powerpc/powerpc64/power7/strstr.S: Set a default function
	name if not defined and pass as parameter to macros accordingly.
	* sysdeps/powerpc/powerpc64/power8/strcasestr.S: Likewise.
2017-04-11 17:13:55 -03:00
Wainer dos Santos Moschetta
6d15a5c2e9 powerpc: refactor strchr, strchrnul, and strrchr IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/strchr-power7.S: Define the
	implementation-specific function name and remove unneeded macros
	definition.
	* sysdeps/powerpc/powerpc64/multiarch/strchr-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strchr-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strchrnul-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strchrnul-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strrchr-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strchr.S: Set a default
	function name if not defined and pass as parameter to macros
	accordingly.
	* sysdeps/powerpc/powerpc64/power7/strchrnul.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strrchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strchr.S: Likewise.
	* sysdeps/powerpc/powerpc64/strchr.S: Likewise.
2017-04-11 17:13:54 -03:00
Wainer dos Santos Moschetta
001649fd18 powerpc: refactor strnlen and strlen IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/strlen-power7.S: Define
	the strlen implementation name and remove unneeded macros definition.
	* sysdeps/powerpc/powerpc64/multiarch/strlen-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strlen-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strnlen-power7.S: Define
	the strnlen implementation name and remove unneeded macros definition.
	* sysdeps/powerpc/powerpc64/power7/strlen.S: Set a default function
	name if not defined and pass as parameter to macros accordingly.
	* sysdeps/powerpc/powerpc64/power7/strnlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strlen.S: Likewise.
	* sysdeps/powerpc/powerpc64/strlen.S: Likewise.
2017-04-11 17:13:54 -03:00
Wainer dos Santos Moschetta
3bc426e156 powerpc: refactor strcasecmp, strcmp, and strncmp IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/strcasecmp_l-power7.S: Define
	the implementation-specific function name and remove unneeded
	macros definition.
	* sysdeps/powerpc/powerpc64/multiarch/strcmp-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcmp-power8.S Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcmp-power9.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcmp-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncmp-power4.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncmp-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncmp-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncmp-power9.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncmp-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/power4/strncmp.S: Set a default function
	name if not defined and pass as parameter to macros accordingly.
	* sysdeps/powerpc/powerpc64/power7/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power9/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/power9/strncmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/strcmp.S: Likewise.
	* sysdeps/powerpc/powerpc64/strncmp.S: Likewise.
2017-04-11 17:13:54 -03:00
Wainer dos Santos Moschetta
dbcc7d0893 powerpc: refactor stpcpy, stpncpy, strcpy, and strncpy IFUNC.
Clean up the IFUNC implementations for powerpc in order to remove
unneeded macro definitions.

Tested on ppc64le with and without --disable-multi-arch flag.

	* sysdeps/powerpc/powerpc64/multiarch/stpcpy-power8.S: Define the
	implementation-specific function name and remove unneeded macros
	definition.
	* sysdeps/powerpc/powerpc64/multiarch/stpncpy-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/stpncpy-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcpy-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncpy-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strncpy-power8.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strncpy.S: Set a default
	function name if not defined.
	* sysdeps/powerpc/powerpc64/power8/strcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power8/strncpy.S: Likewise.
2017-04-11 17:13:31 -03:00
Joseph Myers
bfff8b1bec Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
Adhemerval Zanella
b224637928 Fix powerpc64/power7 memchr for large input sizes
Current optimized powercp64/power7 memchr uses a strategy to check for
p versus align(p+n) (where 'p' is the input char pointer and n the
maximum size to check for the byte) without taking care for possible
overflow on the pointer addition in case of large 'n'.

It was triggered by 3038145ca2 where default rawmemchr (used to
created ppc64 rawmemchr in ifunc selection) now uses memchr (p, c, (size_t)-1)
on its implementation.

This patch fixes it by implement a satured addition where overflows
sets the maximum pointer size to UINTPTR_MAX.

Checked on powerpc64le-linux-gnu.

	[BZ# 20971]
	* sysdeps/powerpc/powerpc64/power7/memchr.S (__memchr): Avoid
	overflow in pointer addition.
	* string/test-memchr.c (do_test): Add an argument to pass as
	the size on memchr.
	(test_main): Add check for SIZE_MAX.
2016-12-16 11:30:20 -02:00
Rajalakshmi Srinivasaraghavan
869d7180dd powerpc: Rearrange cfi_offset calls
This patch rearranges cfi_offset() calls after the last store
so as to avoid extra DW_CFA_advance opcodes in unwind information.
2016-03-11 11:31:58 -03:00
Joseph Myers
f7a9f785e5 Update copyright dates with scripts/update-copyrights. 2016-01-04 16:05:18 +00:00
Rajalakshmi Srinivasaraghavan
fe7faec3e5 powerpc: Handle worstcase behavior in strstr() for POWER7
Instead of checking needle length, constant 'n' number of comparisons
is checked to fall back to default implementation.  This patch is tested
on powerpc64 and powerpc64le.

2015-08-25  Rajalakshmi Srinivasaraghavan  <raji@linux.vnet.ibm.com>

	* sysdeps/powerpc/powerpc64/power7/strstr.S: Handle worst case.
2015-08-25 13:45:56 -03:00
Adhemerval Zanella
6f714aa4ad powerpc: Fix PPC64/POWER7 conform tests
When building with --disable-multi-arch the memmove and strstr POWER7
optimization create and uses symbols that conflict with expect conform
tests.

	* sysdeps/powerpc/powerpc64/power7/memmove.S (bcopy): Changing to
	__bcopy and add a weak_alias to bcopy.
	* sysdeps/powerpc/powerpc64/power7/strstr.S (strstr): Use __strnlen
	for static build.
2015-08-11 10:03:10 -03:00
Adhemerval Zanella
142e0a9953 powerpc: Use default strcpy optimization for POWER7
This patches uses the default strcpy/stpcpy implementation for
POWER7/PPC64.  This is faster in mostly inputs for benchtests
and for multiarch the implementation uses the POWER7 strlen and
memcpy.

	* string/stpcpy.c (__stpcpy): Use STPCPY to redefine symbol name and
	cleanup macro usage.
	* string/strcpy.c (strcpt): Use STRCPY to redefine symbol name.
	* sysdeps/powerpc/powerpc64/multiarch/stpcpy-power7.S: Remove file.
	* sysdeps/powerpc/powerpc64/multiarch/stpcpy-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcpy-power7.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcpy-ppc64.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/stpcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strcpy.c: Likewise.
	* sysdeps/powerpc/powerpc64/stpcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/strcpy.S: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/stpcpy.c
	[SHARED && IS_IN (libc)]: Include <string/strcpy.c>.
	* sysdeps/powerpc/powerpc64/multiarch/stpcpy.c
	[SHARED && IS_IN (libc)]: Include <string/stpcpy.c>.
	* sysdeps/powerpc/powerpc64/multiarch/stpcpy-power7.c: New file.
	* sysdeps/powerpc/powerpc64/multiarch/stpcpy-ppc64.c: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcpy-power7.c: Likewise.
	* sysdeps/powerpc/powerpc64/multiarch/strcpy-ppc64.c: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strcpy.c: Likewise.
2015-08-11 10:03:10 -03:00
Adhemerval Zanella
14362ef154 powerpc: Fix strnlen/power7 build
This patch fixes the strnlen.S build with --disable-multi-arch option.
2015-08-11 10:03:09 -03:00
Adhemerval Zanella
357bb400f1 powerpc: Fix strstr/power7 build
This patch fixes the strstr build with --disable-multi-arch option.
The optimization calls the __strstr_ppc symbol, which always build
for multiarch config but not if it is disable.  This patch fixes it
by adding the default C implementation object with the expected
symbol name.

	* sysdeps/powerpc/powerpc64/power7/Makefile [$(subdir) = string]
	(sysdep_routines): Add strstr-ppc64.
	* sysdeps/powerpc/powerpc64/power7/strstr-ppc64.c: New file.
2015-08-11 10:03:09 -03:00
Rajalakshmi Srinivasaraghavan
b42f8cad52 powerpc: strstr optimization
This patch optimizes strstr function for power >= 7 systems.  Performance
gain is obtained using aligned memory access and usage of cmpb
instruction for quicker comparison.  The average improvement of this
optimization is ~40%.  Tested on ppc64 and ppc64le.

2015-07-16  Rajalakshmi Srinivasaraghavan  <raji@linux.vnet.ibm.com>

	* sysdeps/powerpc/powerpc64/multiarch/Makefile: Add strstr().
	* sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c: Likewise.
	* sysdeps/powerpc/powerpc64/power7/strstr.S: New File.
	* sysdeps/powerpc/powerpc64/multiarch/strstr-power7.S: New File.
	* sysdeps/powerpc/powerpc64/multiarch/strstr-ppc64.c: New File.
	* sysdeps/powerpc/powerpc64/multiarch/strstr.c: New File.
2015-07-16 13:43:51 -03:00
Rajalakshmi Srinivasaraghavan
98408b95b1 powerpc: POWER7 strncpy optimization for unaligned string
This patch optimizes strncpy for power7 for unaligned source or
destination address. The source or destination address is aligned
to doubleword and data is shifted based on the alignment and
added with the previous loaded data to be written as a doubleword.
For each load, cmpb instruction is used for faster null check.

The new optimization shows 10 to 70% of performance improvement
for longer string though it does not show big difference on string
size less than 16 due to additional checks.Hence this new algorithm
is restricted to string greater than 16.
2015-02-12 13:16:08 -05:00
Adhemerval Zanella
ce6615c9c6 powerpc: Fix POWER7/PPC64 performance regression on LE
This patch fixes a performance regression on the POWER7/PPC64 memcmp
porting for Little Endian.  The LE code uses 'ldbrx' instruction to read
the memory on byte reversed form, however ISA 2.06 just provide the indexed
form which uses a register value as additional index, instead of a fixed value
enconded in the instruction.

And the port strategy for LE uses r0 index value and update the address
value on each compare loop interation.  For large compare size values,
it adds 8 more instructions plus some more depending of trailing
size.  This patch fixes it by adding pre-calculate indexes to remove the
address update on loops and tailing sizes.

For large sizes it shows a considerable gain, with double performance
pairing with BE.
2015-01-13 14:35:40 -05:00
Rajalakshmi Srinivasaraghavan
72607db038 powerpc: Optimize POWER7 strcmp trailing checks
This patch optimized the POWER7 trailing check by avoiding using byte
read operations and instead use the doubleword already readed with
bitwise operations.
2015-01-13 14:35:40 -05:00
Adhemerval Zanella
9f2f36e5a9 powerpc: Optimized strncat for POWER7/PPC64
With 3eb38795db (Simplify strncat) the generic algorithms uses
strlen, strnlen, and memcpy.  This is faster than POWER7 current
implementation, especially for unaligned strings (where POWER7 code
uses byte-byte operations).

This patch removes the assembly implementation and uses a multiarch
specialization based on default algorithm calling optimized POWER7
symbols.
2015-01-13 11:28:40 -05:00
Joseph Myers
b168057aaa Update copyright dates with scripts/update-copyrights. 2015-01-02 16:29:47 +00:00
Rajalakshmi Srinivasaraghavan
f59ad976ed powerpc: POWER7 strcpy optimization for unaligned strings
This patch optimizes strcpy for ppc64/power7 for unaligned source or
destination address.  The source or destination address is aligned
to doubleword and data is shifted based on the alignment and
added with the previous loaded data to be written as a doubleword.
For each load, cmpb instruction is used for faster null check.

The word aligned optimization is also removed, since the new unaligned
code path shows better results handling word-aligned strings.

More combination of unaligned inputs is also added in benchtest
to measure the improvement.The new optimization shows 2 to 80% of
performance improvement for longer string though it does not show
big difference on string size less than 16 due to additional checks.
2014-12-31 14:35:59 -05:00
Adhemerval Zanella
0f0a1c82f5 powerpc: Add powerpc64 strpbrk optimization
This patch makes the POWER7 optimized strpbrk generic by using
default doubleword stores to zero the hash, instead of VSX
instructions.  Performance on POWER7/POWER8 does not change.
2014-12-02 13:34:02 -05:00
Adhemerval Zanella
bb2542e0ae powerpc: Add powerpc64 strcspn optimization
This patch makes the POWER7 optimized strcspn generic by using
default doubleword stores to zero the hash, instead of VSX
instructions.  Performance on POWER7/POWER8 does not change.
2014-12-02 07:16:24 -05:00
Adhemerval Zanella
2e8a2de2da powerpc: Add powerpc64 strspn optimization
This patch makes the POWER7 optimized strspn generic by using
default doubleword stores to zero the hash, instead of VSX
instructions. Performance on POWER7/POWER8 machines does not changed.
2014-12-02 07:15:58 -05:00
Siddhesh Poyarekar
a109996ef9 Remove IS_IN_libm
Replace with IS_IN (libm). Generated code unchanged on x86_64.

        * include/math.h: Use IS_IN instead of IS_IN_libm.
        * sysdeps/alpha/fpu/s_copysign.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_copysignl.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_finitel.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_fmal.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_frexpl.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_isinfl.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_isnanl.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_modfl.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_scalbnl.c: Likewise.
        * sysdeps/ieee754/ldbl-128ibm/s_signbitl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_copysignl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_finitel.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_frexpl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_isinfl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_isnanl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_modfl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_scalbnl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/s_signbitl.c: Likewise.
        * sysdeps/ieee754/ldbl-64-128/w_scalblnl.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_copysign.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_finite.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_frexp.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_isinf.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_isnan.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_ldexp.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_ldexpl.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_modf.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_scalbln.c: Likewise.
        * sysdeps/ieee754/ldbl-opt/s_scalbn.c: Likewise.
        * sysdeps/powerpc/power5+/fpu/s_modf.c: Likewise.
        * sysdeps/powerpc/powerpc32/fpu/s_copysign.S: Likewise.
        * sysdeps/powerpc/powerpc32/fpu/s_copysignl.S: Likewise.
        * sysdeps/powerpc/powerpc32/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_copysign.c: Likewise.
        * sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_finite.c: Likewise.
        * sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_isinf.c: Likewise.
        * sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_isnan.c: Likewise.
        * sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_modf.c: Likewise.
        * sysdeps/powerpc/powerpc32/power5/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc32/power6/fpu/s_copysign.S: Likewise.
        * sysdeps/powerpc/powerpc32/power6/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc32/power7/fpu/s_finite.S: Likewise.
        * sysdeps/powerpc/powerpc32/power7/fpu/s_isinf.S: Likewise.
        * sysdeps/powerpc/powerpc32/power7/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/multiarch/s_copysign.c: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/multiarch/s_finite.c: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/multiarch/s_isinf.c: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/multiarch/s_isnan.c: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/multiarch/s_modf.c: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/s_copysign.S: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/s_copysignl.S: Likewise.
        * sysdeps/powerpc/powerpc64/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc64/power5/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc64/power6/fpu/s_copysign.S: Likewise.
        * sysdeps/powerpc/powerpc64/power6/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc64/power6x/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc64/power7/fpu/s_finite.S: Likewise.
        * sysdeps/powerpc/powerpc64/power7/fpu/s_isinf.S: Likewise.
        * sysdeps/powerpc/powerpc64/power7/fpu/s_isnan.S: Likewise.
        * sysdeps/powerpc/powerpc64/power8/fpu/s_finite.S: Likewise.
        * sysdeps/powerpc/powerpc64/power8/fpu/s_isinf.S: Likewise.
        * sysdeps/powerpc/powerpc64/power8/fpu/s_isnan.S: Likewise.
        * sysdeps/sparc/sparc32/fpu/s_signbitl.S: Likewise.
        * sysdeps/sparc/sparc32/sparcv9/fpu/s_isnan.S: Likewise.
        * sysdeps/unix/sysv/linux/alpha/fraiseexcpt.S: Likewise.
2014-11-24 11:41:47 +05:30
Adhemerval Zanella
3b473fecdf PowerPC: multiarch bzero cleanup for PPC64
This patch cleanups the multiarch bzero for powerpc64 by remove
the multiarch objects and use instead the the memset embedded
implementation presented in each multiarch optimization.  The
code generate is essentially the same, but the TB_TOCLESS (which
is not essential).
2014-09-10 07:39:46 -04:00
Adhemerval Zanella
87868c2418 PowerPC: Align power7 memcpy using VSX to quadword
This patch changes power7 memcpy to use VSX instructions only when
memory is aligned to quardword.  It is to avoid unaligned kernel traps
on non-cacheable memory (for instance, memory-mapped I/O).
2014-07-07 15:41:27 -05:00
Adhemerval Zanella
17762f6625 PowerPC: optimized memmove for POWER7/PPC64
This patch adds an optimized memmove optimization for POWER7/powerpc64.
Basically the idea is to use the memcpy for POWER7 on non-overlapped
memory regions and a optimized backward memcpy for memory regions
that overlap (similar to the idea of string/memmove.c).

The backward memcpy algorithm used is similar the one use for memcpy for
POWER7, with adjustments done for alignment.  The difference is memory
is always aligned to 16 bytes before using VSX/altivec instructions.
2014-07-07 15:41:21 -05:00
Vidya Ranganathan
e23d3d2690 PowerPC: Optimized strcmp for PPC64/POWER7
Optimization is achieved on 8 byte aligned strings with double word
comparison using cmpb instruction. On unaligned strings loop unrolling
is applied for Power7 gain.
2014-06-11 08:39:31 -05:00
Adhemerval Zanella
ed36bfa18f PowerPC: Fix optimized strncat strlen call
This patch fixes the optimized ppc64/power7 strncat strlen call for
static build without ifunc enabled.  The strlen symbol to call in such
situation is just strlen, instead of __GI_strlen (since the __GI_
alias is just created for shared objects).
2014-06-06 09:37:07 -05:00
Vidya Ranganathan
f360f94a05 PowerPC: strncpy/stpncpy optimization for PPC64/POWER7
The optimization is achieved by following techniques:
  > data alignment [gain from aligned memory access on read/write]
  > POWER7 gains performance with loop unrolling/unwinding
    [gain by reduction of branch penalty].
  > zero padding done by calling optimized memset
2014-05-06 09:54:25 -05:00
Adhemerval Zanella
de21c33c06 PowerPC: Fix --disable-multi-arch builds
This patch fixes some powerpc32 and powerpc64 builds with
--disable-multi-arch option along with different --with-cpu=powerN.
It cleanups the Implies directories by removing the multiarch
folder for non multiarch config and also fixing two assembly
implementations: powerpc64/power7/strncat.S that is calling the
wrong strlen; and power8/fpu/s_isnan.S that misses the hidden_def and
weak_alias directives.
2014-04-09 06:22:53 -05:00
Alan Modra
af6b17973c Correct prefetch hint in power7 memrchr.
Typo fix.

	* sysdeps/powerpc/powerpc64/power7/memrchr.S: Correct stream hint.
2014-04-02 13:42:27 +10:30
Adhemerval Zanella
6f23d0939e PowerPC: optimized strpbrk for POWER7
This patch add an optimized strpbrk for POWER7 by using a different
algorithm than default implementation: it constructs a table based on
the 'accept' argument and use this table to check for any occurance on
the input string. The idea is similar as x86_64 uses.
For PowerPC some tunings were added, such as unroll loops and memory
clear using VSX instructions.
2014-03-20 19:46:13 -05:00
Adhemerval Zanella
6eaf95cbfa PowerPC: optimized strcspn for PPC64/POWER7
This patch add a optimized strcspn for POWER7 by using a different
algorithm than default implementation: it constructs a table based on
the 'accept' argument and use this table to check for any occurance
on the input string. The idea is similar as x86_64 uses.
For PowerPC some tunings were added, such as unroll loops and align
stack memory to table to 16 bytes (so VSX clean can ran without
alignment issues).
2014-03-20 11:24:52 -05:00
Vidya Ranganathan
e65caf1f1d PowerPC: strspn optimization for PPC64/POWER7
The optimization is achieved by following techniques:
  > hashing of needle.
  > hashing avoids scanning of duplicate entries in needle across the string.
  > initializing the hash table with Vector instructions (VSX) by quadword access.
  > unrolling when scanning for character in string across hash table.
2014-03-11 08:54:33 -05:00
Adhemerval Zanella
ba9cc0714e PowerPC: strncat optimization for PPC64
The optimization is achieved by following techniques:
1. Doubleword aligned memory access and compares using
   cmpb instruction.
2. Loop unrolling for byte load/store.
3. CPU pre-fetch to avoid cache miss.
2014-03-10 07:25:09 -05:00
Rajalakshmi Srinivasaraghavan
c7debbdfac PowerPC: strrchr optimization for POWER7/PPC64
This patch optimizes strrchr() for ppc64. It uses aligned memory
access along with cmpb instruction and CPU prefetch to avoid
cache misses for speed improvement.
2014-03-03 08:06:41 -06:00
Allan McRae
d4697bc93d Update copyright notices with scripts/update-copyrights 2014-01-01 22:00:23 +10:00
Andreas Schwab
83f5c32d21 Fix uses of CALL_MCOUNT in ppc64 assembler sources 2013-12-19 17:06:48 +01:00
Adhemerval Zanella
69bbc63d88 PowerPC: Adjust multiarch Implies for PowerPC64
This patch adds Implies files on multiarch folder for POWER chips so
multirach is enabled when building with --with-cpu and powerN
option.
2013-12-13 14:58:02 -05:00
Adhemerval Zanella
8a29a3d00b PowerPC: multiarch memset/bzero for PowerPC64 2013-12-13 14:33:16 -05:00