AuroraMiddleware/glibc
1
0
Fork 0
mirror of https://sourceware.org/git/glibc.git synced 2024-11-26 15:00:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
67c0579669
glibc/sysdeps/x86_64/multiarch
History
Leonardo Sandoval 1457016337 x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 
Optimize x86-64 strcmp/wcscmp and strncmp/wcsncmp with AVX2. It uses vector
comparison as much as possible. Peak performance observed on a SkyLake
machine: 9x, 3x, 2.5x and 5.5x for strcmp, strncmp, wcscmp and wcsncmp,
respectively. The larger the comparison length, the more benefit using
avx2 functions, except on the strcmp, where peak is observed at length
== 32 bytes. Select AVX2 strcmp/wcscmp on AVX2 machines where vzeroupper
is preferred and AVX unaligned load is fast.

NB: It uses TZCNT instead of BSF since TZCNT produces the same result
as BSF for non-zero input.  TZCNT is faster than BSF and is executed
as BSF if machine doesn't support TZCNT.

	* sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Add
	strcmp-avx2, strncmp-avx2, wcscmp-avx2, wcscmp-sse2, wcsncmp-avx2 and
	wcsncmp-sse2.
	* sysdeps/x86_64/multiarch/ifunc-impl-list.c
	(__libc_ifunc_impl_list): Add tests for __strcmp_avx2,
	__strncmp_avx2,	__wcscmp_avx2, __wcsncmp_avx2, __wcscmp_sse2
	and __wcsncmp_sse2.
	* sysdeps/x86_64/multiarch/strcmp.c (OPTIMIZE (avx2)):
	(IFUNC_SELECTOR): Return OPTIMIZE (avx2) on AVX 2 machines if
	AVX unaligned load is fast and vzeroupper is preferred.
	* sysdeps/x86_64/multiarch/strncmp.c: Likewise.
	* sysdeps/x86_64/multiarch/strcmp-avx2.S: New file.
	* sysdeps/x86_64/multiarch/strncmp-avx2.S: Likewise.
	* sysdeps/x86_64/multiarch/wcscmp-avx2.S: Likewise.
	* sysdeps/x86_64/multiarch/wcscmp-sse2.S: Likewise.
	* sysdeps/x86_64/multiarch/wcscmp.c: Likewise.
	* sysdeps/x86_64/multiarch/wcsncmp-avx2.S: Likewise.
	* sysdeps/x86_64/multiarch/wcsncmp-sse2.c: Likewise.
	* sysdeps/x86_64/multiarch/wcsncmp.c: Likewise.
	* sysdeps/x86_64/wcscmp.S (__wcscmp): Add alias only if __wcscmp
	is undefined.
2018-06-01 16:32:43 -05:00
..
bcopy.S Use IFUNC memmove/memset in x86-64 bcopy/bzero 2012-10-11 13:58:16 -07:00
ifunc-avx2.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ifunc-impl-list.c x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
ifunc-memcmp.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ifunc-memmove.h x86-64: Check Prefer_FSRM in ifunc-memmove.h 2018-05-21 16:54:59 -07:00
ifunc-memset.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ifunc-sse4_2.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ifunc-strcasecmp.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ifunc-unaligned-ssse3.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ifunc-wmemset.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
Makefile x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
memchr-avx2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memchr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcmp-avx2-movbe.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcmp-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcmp-sse4.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcmp-ssse3.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcpy_chk-nonshared.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcpy_chk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcpy-ssse3-back.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcpy-ssse3.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memmove_chk-nonshared.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memmove_chk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memmove-avx512-no-vzeroupper.S Don't write beyond destination in __mempcpy_avx512_no_vzeroupper (bug 23196) 2018-05-23 09:50:57 +02:00
memmove-avx512-unaligned-erms.S Require binutils 2.24 to build x86-64 glibc [BZ #20139] 2016-07-01 06:03:05 -07:00
memmove-avx-unaligned-erms.S X86-64: Use non-temporal store in memcpy on large data 2016-04-12 08:10:47 -07:00
memmove-sse2-unaligned-erms.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memmove-ssse3-back.S Improve 64bit memcpy/memmove for Atom, Core 2 and Core i7 2010-06-30 08:26:11 -07:00
memmove-ssse3.S Improve 64bit memcpy/memmove for Atom, Core 2 and Core i7 2010-06-30 08:26:11 -07:00
memmove-vec-unaligned-erms.S x86-64: Skip zero length in __mem[pcpy|move|set]_erms 2018-05-23 11:25:42 -07:00
memmove.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
mempcpy_chk-nonshared.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
mempcpy_chk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
mempcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memrchr-avx2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memrchr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memset_chk-nonshared.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memset_chk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memset-avx2-unaligned-erms.S x86-64: Optimize wmemset with SSE2/AVX2/AVX512 2017-06-05 11:09:59 -07:00
memset-avx512-no-vzeroupper.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memset-avx512-unaligned-erms.S x86-64: Optimize wmemset with SSE2/AVX2/AVX512 2017-06-05 11:09:59 -07:00
memset-sse2-unaligned-erms.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memset-vec-unaligned-erms.S x86-64: Skip zero length in __mem[pcpy|move|set]_erms 2018-05-23 11:25:42 -07:00
memset.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
rawmemchr-avx2.S x86-64: Optimize memchr/rawmemchr/wmemchr with SSE2/AVX2 2017-06-09 05:13:31 -07:00
rawmemchr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
rawmemchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
sched_cpucount.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
stpcpy-sse2-unaligned.S Improved st{r,p}{,n}cpy for SSE2 and SSSE3 on x86-64 2011-06-24 15:14:22 -04:00
stpcpy-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
stpcpy-ssse3.S Improved st{r,p}{,n}cpy for SSE2 and SSSE3 on x86-64 2011-06-24 15:14:22 -04:00
stpcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
stpncpy-c.c x86-64: Implement strcpy family IFUNC selectors in C 2017-06-12 09:06:09 -07:00
stpncpy-sse2-unaligned.S Improved st{r,p}{,n}cpy for SSE2 and SSSE3 on x86-64 2011-06-24 15:14:22 -04:00
stpncpy-ssse3.S Improved st{r,p}{,n}cpy for SSE2 and SSSE3 on x86-64 2011-06-24 15:14:22 -04:00
stpncpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp_l-avx.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp_l-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp_l-sse4_2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp_l-ssse3.S Fix x86-64 build without multiarch. 2010-08-14 14:56:32 -07:00
strcasecmp_l.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcat-sse2-unaligned.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcat-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcat-ssse3.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcat.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchr-avx2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchr-sse2-no-bsf.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchrnul-avx2.S x86-64: Optimize strchr/strchrnul/wcschr with AVX2 2017-06-09 05:42:29 -07:00
strchrnul-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchrnul.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcmp-avx2.S x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
strcmp-sse2-unaligned.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcmp-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcmp-sse4_2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcmp-sse42.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcmp-ssse3.S Remove NOT_IN_libc 2014-11-24 15:03:45 +05:30
strcmp.c x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
strcpy-sse2-unaligned.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcpy-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcpy-ssse3.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcspn-c.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcspn-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcspn.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strlen-avx2.S x86-64: Remove the unnecessary testl in strlen-avx2.S 2018-05-14 03:41:35 -07:00
strlen-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase_l-avx.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase_l-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase_l-sse4_2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase_l-ssse3.S Add optimized strncasecmp versions for x86-64. 2010-08-14 22:04:01 -07:00
strncase_l.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncat-c.c x86-64: Use IFUNC strncat inside libc.so 2018-05-16 09:04:35 -07:00
strncat-sse2-unaligned.S Improve 64 bit strcat functions with SSE2/SSSE3 2011-07-19 17:11:54 -04:00
strncat-ssse3.S Improve 64 bit strcat functions with SSE2/SSSE3 2011-07-19 17:11:54 -04:00
strncat.c x86-64: Use IFUNC strncat inside libc.so 2018-05-16 09:04:35 -07:00
strncmp-avx2.S x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
strncmp-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncmp-sse4_2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncmp-ssse3.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncmp.c x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
strncpy-c.c x86-64: Implement strcpy family IFUNC selectors in C 2017-06-12 09:06:09 -07:00
strncpy-sse2-unaligned.S Improved st{r,p}{,n}cpy for SSE2 and SSSE3 on x86-64 2011-06-24 15:14:22 -04:00
strncpy-ssse3.S Improved st{r,p}{,n}cpy for SSE2 and SSSE3 on x86-64 2011-06-24 15:14:22 -04:00
strncpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strnlen-avx2.S x86-64: Optimize strlen/strnlen/wcslen/wcsnlen with AVX2 2017-06-09 05:18:18 -07:00
strnlen-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strnlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strpbrk-c.c x86-64: Implement strcspn/strpbrk/strspn IFUNC selectors in C 2017-06-15 08:59:05 -07:00
strpbrk-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strpbrk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strrchr-avx2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strrchr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strspn-c.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strspn-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strspn.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strstr-sse2-unaligned.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strstr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-multiarch.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
varshift.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
varshift.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcschr-avx2.S x86-64: Optimize strchr/strchrnul/wcschr with AVX2 2017-06-09 05:42:29 -07:00
wcschr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcschr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcscmp-avx2.S x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
wcscmp-sse2.S x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
wcscmp.c x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
wcscpy-c.c Remove NOT_IN_libc 2014-11-24 15:03:45 +05:30
wcscpy-ssse3.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcscpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcslen-avx2.S x86-64: Optimize strlen/strnlen/wcslen/wcsnlen with AVX2 2017-06-09 05:18:18 -07:00
wcslen-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcslen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcsncmp-avx2.S x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
wcsncmp-sse2.c x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
wcsncmp.c x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 2018-06-01 16:32:43 -05:00
wcsnlen-avx2.S x86-64: Optimize strlen/strnlen/wcslen/wcsnlen with AVX2 2017-06-09 05:18:18 -07:00
wcsnlen-c.c x86-64: Move wcsnlen.S to multiarch/wcsnlen-sse4_1.S 2017-06-06 06:12:32 -07:00
wcsnlen-sse4_1.S x86-64: Move wcsnlen.S to multiarch/wcsnlen-sse4_1.S 2017-06-06 06:12:32 -07:00
wcsnlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcsrchr-avx2.S x86-64: Optimize strrchr/wcsrchr with AVX2 2017-06-09 05:45:52 -07:00
wcsrchr-sse2.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wcsrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wmemchr-avx2.S x86-64: Optimize memchr/rawmemchr/wmemchr with SSE2/AVX2 2017-06-09 05:13:31 -07:00
wmemchr-sse2.S x86-64: Optimize memchr/rawmemchr/wmemchr with SSE2/AVX2 2017-06-09 05:13:31 -07:00
wmemchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wmemcmp-avx2-movbe.S x86-64: Optimize memcmp/wmemcmp with AVX2 and MOVBE 2017-06-05 12:52:55 -07:00
wmemcmp-c.c Remove NOT_IN_libc 2014-11-24 15:03:45 +05:30
wmemcmp-sse4.S Optimized memcmp and wmemcmp for x86-64 and x86-32 2011-10-15 11:10:08 -04:00
wmemcmp-ssse3.S Optimized memcmp and wmemcmp for x86-64 and x86-32 2011-10-15 11:10:08 -04:00
wmemcmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wmemset_chk-nonshared.S Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wmemset_chk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
wmemset.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00