mirror of
https://sourceware.org/git/glibc.git
synced 2024-12-24 19:51:11 +00:00
b22be8c368
6 Commits
Author | SHA1 | Message | Date | |
---|---|---|---|---|
H.J. Lu
|
c867597bff |
X86-64: Remove previous default/SSE2/AVX2 memcpy/memmove
Since the new SSE2/AVX2 memcpy/memmove are faster than the previous ones, we can remove the previous SSE2/AVX2 memcpy/memmove and replace them with the new ones. No change in IFUNC selection if SSE2 and AVX2 memcpy/memmove weren't used before. If SSE2 or AVX2 memcpy/memmove were used, the new SSE2 or AVX2 memcpy/memmove optimized with Enhanced REP MOVSB will be used for processors with ERMS. The new AVX512 memcpy/memmove will be used for processors with AVX512 which prefer vzeroupper. Since the new SSE2 memcpy/memmove are faster than the previous default memcpy/memmove used in libc.a and ld.so, we also remove the previous default memcpy/memmove and make them the default memcpy/memmove, except that non-temporal store isn't used in ld.so. Together, it reduces the size of libc.so by about 6 KB and the size of ld.so by about 2 KB. [BZ #19776] * sysdeps/x86_64/memcpy.S: Make it dummy. * sysdeps/x86_64/mempcpy.S: Likewise. * sysdeps/x86_64/memmove.S: New file. * sysdeps/x86_64/memmove_chk.S: Likewise. * sysdeps/x86_64/multiarch/memmove.S: Likewise. * sysdeps/x86_64/multiarch/memmove_chk.S: Likewise. * sysdeps/x86_64/memmove.c: Removed. * sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S: Likewise. * sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S: Likewise. * sysdeps/x86_64/multiarch/memmove-avx-unaligned.S: Likewise. * sysdeps/x86_64/multiarch/memmove-sse2-unaligned-erms.S: Likewise. * sysdeps/x86_64/multiarch/memmove.c: Likewise. * sysdeps/x86_64/multiarch/memmove_chk.c: Likewise. * sysdeps/x86_64/multiarch/Makefile (sysdep_routines): Remove memcpy-sse2-unaligned, memmove-avx-unaligned, memcpy-avx-unaligned and memmove-sse2-unaligned-erms. * sysdeps/x86_64/multiarch/ifunc-impl-list.c (__libc_ifunc_impl_list): Replace __memmove_chk_avx512_unaligned_2 with __memmove_chk_avx512_unaligned. Remove __memmove_chk_avx_unaligned_2. Replace __memmove_chk_sse2_unaligned_2 with __memmove_chk_sse2_unaligned. Remove __memmove_chk_sse2 and __memmove_avx_unaligned_2. Replace __memmove_avx512_unaligned_2 with __memmove_avx512_unaligned. Replace __memmove_sse2_unaligned_2 with __memmove_sse2_unaligned. Remove __memmove_sse2. Replace __memcpy_chk_avx512_unaligned_2 with __memcpy_chk_avx512_unaligned. Remove __memcpy_chk_avx_unaligned_2. Replace __memcpy_chk_sse2_unaligned_2 with __memcpy_chk_sse2_unaligned. Remove __memcpy_chk_sse2. Remove __memcpy_avx_unaligned_2. Replace __memcpy_avx512_unaligned_2 with __memcpy_avx512_unaligned. Remove __memcpy_sse2_unaligned_2 and __memcpy_sse2. Replace __mempcpy_chk_avx512_unaligned_2 with __mempcpy_chk_avx512_unaligned. Remove __mempcpy_chk_avx_unaligned_2. Replace __mempcpy_chk_sse2_unaligned_2 with __mempcpy_chk_sse2_unaligned. Remove __mempcpy_chk_sse2. Replace __mempcpy_avx512_unaligned_2 with __mempcpy_avx512_unaligned. Remove __mempcpy_avx_unaligned_2. Replace __mempcpy_sse2_unaligned_2 with __mempcpy_sse2_unaligned. Remove __mempcpy_sse2. * sysdeps/x86_64/multiarch/memcpy.S (__new_memcpy): Support __memcpy_avx512_unaligned_erms and __memcpy_avx512_unaligned. Use __memcpy_avx_unaligned_erms and __memcpy_sse2_unaligned_erms if processor has ERMS. Default to __memcpy_sse2_unaligned. (ENTRY): Removed. (END): Likewise. (ENTRY_CHK): Likewise. (libc_hidden_builtin_def): Likewise. Don't include ../memcpy.S. * sysdeps/x86_64/multiarch/memcpy_chk.S (__memcpy_chk): Support __memcpy_chk_avx512_unaligned_erms and __memcpy_chk_avx512_unaligned. Use __memcpy_chk_avx_unaligned_erms and __memcpy_chk_sse2_unaligned_erms if if processor has ERMS. Default to __memcpy_chk_sse2_unaligned. * sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S Change function suffix from unaligned_2 to unaligned. * sysdeps/x86_64/multiarch/mempcpy.S (__mempcpy): Support __mempcpy_avx512_unaligned_erms and __mempcpy_avx512_unaligned. Use __mempcpy_avx_unaligned_erms and __mempcpy_sse2_unaligned_erms if processor has ERMS. Default to __mempcpy_sse2_unaligned. (ENTRY): Removed. (END): Likewise. (ENTRY_CHK): Likewise. (libc_hidden_builtin_def): Likewise. Don't include ../mempcpy.S. (mempcpy): New. Add a weak alias. * sysdeps/x86_64/multiarch/mempcpy_chk.S (__mempcpy_chk): Support __mempcpy_chk_avx512_unaligned_erms and __mempcpy_chk_avx512_unaligned. Use __mempcpy_chk_avx_unaligned_erms and __mempcpy_chk_sse2_unaligned_erms if if processor has ERMS. Default to __mempcpy_chk_sse2_unaligned. |
||
Joseph Myers
|
2969121014 | Remove bounded-pointers handling from x86_64 assembly sources. | ||
Ulrich Drepper
|
bfe6f5fa89 |
* sysdeps/unix/sysv/linux/x86_64/sysconf.c: Move cache information
handling to ... * sysdeps/x86_64/cacheinfo.c: ... here. New file. * sysdeps/x86_64/Makefile [subdir=string] (sysdep_routines): Add cacheinfo. * sysdeps/x86_64/memcpy.S: Complete rewrite. * sysdeps/x86_64/mempcpy.S: Adjust appropriately. Patch by Evandro Menezes <evandro.menezes@amd.com>. * sysdeps/unix/sysv/linux/i386/epoll_pwait.S: New file. |
||
Ulrich Drepper
|
b5cc329c4f |
2004-10-15 Jakub Jelinek <jakub@redhat.com>
* elf/dl-minimal.c (__chk_fail): New. Add rtld_hidden_def. * sysdeps/unix/sysv/linux/readonly-area.c: New file. * sysdeps/i386/i686/memmove.S (__memmove_chk): Add checking routine. * sysdeps/i386/i686/memcpy.S (__memcpy_chk): Likewise. * sysdeps/i386/i686/mempcpy.S (__mempcpy_chk): Likewise. * sysdeps/i386/i686/memset.S (__memset_chk): Likewise. * sysdeps/i386/i686/memmove-chk.S: New file. * sysdeps/i386/i686/memcpy-chk.S: Likewise. * sysdeps/i386/i686/mempcpy-chk.S: Likewise. * sysdeps/i386/i686/memset-chk.S: Likewise. * sysdeps/generic/strcat-chk.c (__strcat_chk): Don't __chk_fail if exactly fitting into buffer. * sysdeps/generic/strncat-chk.c (__strncat_chk): Likewise. * sysdeps/generic/readonly-area.c: New file. * sysdeps/generic/strncpy-chk.c (__strncpy_chk): Only test destlen once. * sysdeps/x86_64/memset.S (__memset_chk): Add checking routine. * sysdeps/x86_64/memcpy.S (__memcpy_chk): Likewise. * sysdeps/x86_64/mempcpy.S (__memcpy_chk): Define to __mempcpy_chk. * sysdeps/x86_64/memcpy-chk.S: New file. * sysdeps/x86_64/mempcpy-chk.S: Likewise. * sysdeps/x86_64/memset-chk.S: Likewise. * sysdeps/x86_64/strcpy-chk.S: Likewise. * sysdeps/x86_64/stpcpy-chk.S: Likewise. * argp/argp-xinl.c (__OPTIMIZE__): Define to 1 instead of nothing. * argp/argp-fs-xinl.c (__OPTIMIZE__): Likewise. * debug/tst-chk1.c: New test. * debug/tst-chk2.c: Likewise. * debug/tst-chk3.c: Likewise. * debug/test-strcpy_chk.c: Likewise. * debug/test-stpcpy_chk.c: Likewise. * debug/vsprintf_chk.c (__vsprintf_chk): If flags > 0, request _IO_FLAGS2_CHECK_PERCENT_N. Add libc_hidden_def. * debug/Makefile (routines): Add printf_chk, fprintf_chk, vprintf_chk, vfprintf_chk, gets_chk and readonly-area. (CFLAGS-*_chk.c): Set. (tests): Add tst-chk1, tst-chk2, tst-chk3, test-strcpy_chk and test-stpcpy_chk. * debug/vprintf_chk.c: New file. * debug/printf_chk.c: Likewise. * debug/vfprintf_chk.c: Likewise. * debug/fprintf_chk.c: Likewise. * debug/gets_chk.c: Likewise. * debug/chk_fail.c (__chk_fail): Add libc_hidden_def. * debug/snprintf_chk.c (__snprintf_chk): Fix order of arguments passed to __vsnprintf_chk. * debug/Versions (libc): Export __printf_chk, __fprintf_chk, __vprintf_chk, __vfprintf_chk and __gets_chk @GLIBC_2.3.4. * debug/vsnprintf_chk.c (__vsnprintf_chk): Don't call __vsnprintf, instead create a temporary file with _IO_strn_jumps jumptable. If flags > 0, request _IO_FLAGS2_CHECK_PERCENT_N. Add libc_hidden_def. * libio/Makefile (headers): Add bits/stdio2.h. * libio/stdio.h: Include <bits/stdio2.h> if __USE_FORTIFY_LEVEL. (sprintf, snprintf, vsprintf, vsnprintf): Remove defines. * libio/strfile.h (_IO_strnfile): New type. (_IO_strn_jumps): New extern. * libio/vsnprintf.c (_IO_strnfile): Remove. (_IO_strn_jumps): Remove static. * libio/bits/stdio2.h: New file. * libio/vswprintf.c (_IO_strnfile): Rename type to... (_IO_wstrnfile): ...this. Adjust all uses. * libio/libio.h (_IO_FLAGS2_CHECK_PERCENT_N): Define. * stdio-common/vfprintf.c (STR_LEN): Define. (vfprintf): Add readonly_format variable. Handle _IO_FLAGS2_CHECK_PERCENT_N. (buffered_vfprintf): Copy _flags2. * include/stdio.h (__sprintf_chk, __snprintf_chk, __vsprintf_chk, __vsnprintf_chk, __printf_chk, __fprintf_chk, __vprintf_chk, __vfprintf_chk): New prototypes. (__vsprintf_chk, __vsnprintf_chk): Add libc_hidden_proto. * include/string.h (__memcpy_chk, __memmove_chk, __mempcpy_chk, __memset_chk, __strcpy_chk, __stpcpy_chk, __strncpy_chk, __strcat_chk, __strncat_chk): New prototypes. * include/bits/string3.h: New file. * include/sys/cdefs.h (__chk_fail): Add libc_hidden_proto and rtld_hidden_proto. * string/Makefile (headers): Add bits/string3.h. * string/bits/string3.h (bcopy, bzero): New defines. (memset, memcpy, memmove, strcpy, strncpy, strcat, strncat): Change macros so that inlines are used only if unknown destination size or side-effects in destination argument. (mempcpy, stpcpy): Likewise. Protect with #ifdef __USE_GNU. 2004-09-16 Ulrich Drepper <drepper@redhat.com> * debug/Makefile (routines): Add *_chk. * debug/Versions (libc): Export __chk_fail, __memcpy_chk, __memmove_chk, __mempcpy_chk, __memset_chk, __stpcpy_chk, __strcat_chk, __strcpy_chk, __strncat_chk, __strncpy_chk, __sprintf_chk, __vsprintf_chk, __snprintf_chk, __vsnprintf_chk @GLIBC_2.3.4. * debug/chk_fail.c: New file. * debug/snprintf_chk.c: Likewise. * debug/sprintf_chk.c: Likewise. * debug/vsnprintf_chk.c: Likewise. * debug/vsprintf_chk.c: Likewise. * include/features.h (_FORTIFY_SOURCE): Document, handle. (__USE_FORTIFY_LEVEL): Define. (__GNUC_PREREQ): Move to earlier location. * include/sys/cdefs.h (__chk_fail): New prototype. * libio/bits/stdio.h (sprintf, vsprintf, snprintf, vsnprintf): Define if __USE_FORTIFY_LEVEL. * misc/sys/cdefs.h (__bos, __bos0): Define. * string/string.h: Include <bits/string3.h> if __USE_FORTIFY_LEVEL. * bits/string/string3.h: New header. * sysdeps/generic/memcpy_chk.c: New file. * sysdeps/generic/memmove_chk.c: Likewise. * sysdeps/generic/mempcpy_chk.c: Likewise. * sysdeps/generic/memset_chk.c: Likewise. * sysdeps/generic/stpcpy_chk.c: Likewise. * sysdeps/generic/strcat_chk.c: Likewise. * sysdeps/generic/strcpy_chk.c: Likewise. * sysdeps/generic/strncat_chk.c: Likewise. * sysdeps/generic/strncpy_chk.c: Likewise. 2004-10-15 Jakub Jelinek <jakub@redhat.com> * elf/dl-minimal.c (__chk_fail): New. Add rtld_hidden_def. * sysdeps/unix/sysv/linux/readonly-area.c: New file. * sysdeps/i386/i686/memmove.S (__memmove_chk): Add checking routine. * sysdeps/i386/i686/memcpy.S (__memcpy_chk): Likewise. * sysdeps/i386/i686/mempcpy.S (__mempcpy_chk): Likewise. * sysdeps/i386/i686/memset.S (__memset_chk): Likewise. * sysdeps/i386/i686/memmove-chk.S: New file. * sysdeps/i386/i686/memcpy-chk.S: Likewise. * sysdeps/i386/i686/mempcpy-chk.S: Likewise. * sysdeps/i386/i686/memset-chk.S: Likewise. * sysdeps/generic/strcat-chk.c (__strcat_chk): Don't __chk_fail if exactly fitting into buffer. * sysdeps/generic/strncat-chk.c (__strncat_chk): Likewise. * sysdeps/generic/readonly-area.c: New file. * sysdeps/generic/strncpy-chk.c (__strncpy_chk): Only test destlen once. * sysdeps/x86_64/memset.S (__memset_chk): Add checking routine. * sysdeps/x86_64/memcpy.S (__memcpy_chk): Likewise. * sysdeps/x86_64/mempcpy.S (__memcpy_chk): Define to __mempcpy_chk. * sysdeps/x86_64/memcpy-chk.S: New file. * sysdeps/x86_64/mempcpy-chk.S: Likewise. * sysdeps/x86_64/memset-chk.S: Likewise. * sysdeps/x86_64/strcpy-chk.S: Likewise. * sysdeps/x86_64/stpcpy-chk.S: Likewise. * argp/argp-xinl.c (__OPTIMIZE__): Define to 1 instead of nothing. * argp/argp-fs-xinl.c (__OPTIMIZE__): Likewise. * debug/tst-chk1.c: New test. * debug/tst-chk2.c: Likewise. * debug/tst-chk3.c: Likewise. * debug/test-strcpy_chk.c: Likewise. * debug/test-stpcpy_chk.c: Likewise. * debug/vsprintf_chk.c (__vsprintf_chk): If flags > 0, request _IO_FLAGS2_CHECK_PERCENT_N. Add libc_hidden_def. * debug/Makefile (routines): Add printf_chk, fprintf_chk, vprintf_chk, vfprintf_chk, gets_chk and readonly-area. (CFLAGS-*_chk.c): Set. (tests): Add tst-chk1, tst-chk2, tst-chk3, test-strcpy_chk and test-stpcpy_chk. * debug/vprintf_chk.c: New file. * debug/printf_chk.c: Likewise. * debug/vfprintf_chk.c: Likewise. * debug/fprintf_chk.c: Likewise. * debug/gets_chk.c: Likewise. * debug/chk_fail.c (__chk_fail): Add libc_hidden_def. * debug/snprintf_chk.c (__snprintf_chk): Fix order of arguments passed to __vsnprintf_chk. * debug/Versions (libc): Export __printf_chk, __fprintf_chk, __vprintf_chk, __vfprintf_chk and __gets_chk @GLIBC_2.3.4. * debug/vsnprintf_chk.c (__vsnprintf_chk): Don't call __vsnprintf, instead create a temporary file with _IO_strn_jumps jumptable. If flags > 0, request _IO_FLAGS2_CHECK_PERCENT_N. Add libc_hidden_def. * libio/Makefile (headers): Add bits/stdio2.h. * libio/stdio.h: Include <bits/stdio2.h> if __USE_FORTIFY_LEVEL. (sprintf, snprintf, vsprintf, vsnprintf): Remove defines. * libio/strfile.h (_IO_strnfile): New type. (_IO_strn_jumps): New extern. * libio/vsnprintf.c (_IO_strnfile): Remove. (_IO_strn_jumps): Remove static. * libio/bits/stdio2.h: New file. * libio/vswprintf.c (_IO_strnfile): Rename type to... (_IO_wstrnfile): ...this. Adjust all uses. * libio/libio.h (_IO_FLAGS2_CHECK_PERCENT_N): Define. * stdio-common/vfprintf.c (STR_LEN): Define. (vfprintf): Add readonly_format variable. Handle _IO_FLAGS2_CHECK_PERCENT_N. (buffered_vfprintf): Copy _flags2. * include/stdio.h (__sprintf_chk, __snprintf_chk, __vsprintf_chk, __vsnprintf_chk, __printf_chk, __fprintf_chk, __vprintf_chk, __vfprintf_chk): New prototypes. (__vsprintf_chk, __vsnprintf_chk): Add libc_hidden_proto. * include/string.h (__memcpy_chk, __memmove_chk, __mempcpy_chk, __memset_chk, __strcpy_chk, __stpcpy_chk, __strncpy_chk, __strcat_chk, __strncat_chk): New prototypes. * include/bits/string3.h: New file. * include/sys/cdefs.h (__chk_fail): Add libc_hidden_proto and rtld_hidden_proto. * string/Makefile (headers): Add bits/string3.h. * string/bits/string3.h (bcopy, bzero): New defines. (memset, memcpy, memmove, strcpy, strncpy, strcat, strncat): Change macros so that inlines are used only if unknown destination size or side-effects in destination argument. (mempcpy, stpcpy): Likewise. Protect with #ifdef __USE_GNU. 2004-09-16 Ulrich Drepper <drepper@redhat.com> * debug/Makefile (routines): Add *_chk. * debug/Versions (libc): Export __chk_fail, __memcpy_chk, __memmove_chk, __mempcpy_chk, __memset_chk, __stpcpy_chk, __strcat_chk, __strcpy_chk, __strncat_chk, __strncpy_chk, __sprintf_chk, __vsprintf_chk, __snprintf_chk, __vsnprintf_chk @GLIBC_2.3.4. * debug/chk_fail.c: New file. * debug/snprintf_chk.c: Likewise. * debug/sprintf_chk.c: Likewise. * debug/vsnprintf_chk.c: Likewise. * debug/vsprintf_chk.c: Likewise. * include/features.h (_FORTIFY_SOURCE): Document, handle. (__USE_FORTIFY_LEVEL): Define. (__GNUC_PREREQ): Move to earlier location. * include/sys/cdefs.h (__chk_fail): New prototype. * libio/bits/stdio.h (sprintf, vsprintf, snprintf, vsnprintf): Define if __USE_FORTIFY_LEVEL. * misc/sys/cdefs.h (__bos, __bos0): Define. * string/string.h: Include <bits/string3.h> if __USE_FORTIFY_LEVEL. * bits/string/string3.h: New header. * sysdeps/generic/memcpy_chk.c: New file. * sysdeps/generic/memmove_chk.c: Likewise. * sysdeps/generic/mempcpy_chk.c: Likewise. * sysdeps/generic/memset_chk.c: Likewise. * sysdeps/generic/stpcpy_chk.c: Likewise. * sysdeps/generic/strcat_chk.c: Likewise. * sysdeps/generic/strcpy_chk.c: Likewise. * sysdeps/generic/strncat_chk.c: Likewise. * sysdeps/generic/strncpy_chk.c: Likewise. |
||
Ulrich Drepper
|
3dbfd8117c |
Update.
2004-05-27 Jakub Jelinek <jakub@redhat.com> * sysdeps/i386/fpu/bits/mathinline.h (__expm1_code): Define using __builtin_expm1l for GCC 3.5+. (__expl): Define using __builtin_expl for GCC 3.4+. (exp, expf, expl): Don't define for GCC 3.4+. (tan, tanf, tanl): Don't define for GCC 3.5+. (__atan2l): Define using __builtin_atan2l for GCC 3.4+. (atan2, atan2f, atan2l): Don't define for GCC 3.4+ or !__FAST_MATH__. (fmod, fmodf, fmodl): Don't define for GCC 3.5+ or !__FAST_MATH__. (fabsf, fabsl): Only provide if __USE_MISC or __USE_ISOC99. (sin, sinf, sinl, cos, cosf, cosl, log, logf, logl): Don't define for GCC 3.4+. (log10, log10f, log10l, asin, asinf, asinl, acos, acosf, acosl): Don't define for GCC 3.5+. (atan, atanf, atanl): Don't define for GCC 3.4+ or !__FAST_MATH__. (log1p, log1pf, log1pl, logb, logbf, logbl, log2, log2f, log2l): Don't define for GCC 3.5+. (drem, dremf, dreml): Don't define for GCC 3.5+ or !__FAST_MATH__. * sysdeps/sparc/fpu/bits/mathinline.h (sqrt, sqrtf, sqrtl): Don't define for GCC 3.2+. 2004-05-27 Jakub Jelinek <jakub@redhat.com> * string/bits/string2.h (__bzero): Define even for GCC 3.0+. * sysdeps/alpha/stpcpy.S (stpcpy): Add libc_hidden_builtin_def. * sysdeps/alpha/alphaev67/stpcpy.S (stpcpy): Likewise. * sysdeps/powerpc/powerpc32/stpcpy.S (stpcpy): Likewise. * sysdeps/powerpc/powerpc64/stpcpy.S (stpcpy): Likewise. * sysdeps/sparc/sparc32/stpcpy.S (stpcpy): Likewise. * sysdeps/sparc/sparc64/stpcpy.S (stpcpy): Likewise. * sysdeps/i386/stpcpy.S (stpcpy): Likewise. * sysdeps/i386/i586/stpcpy.S (stpcpy): Likewise. * sysdeps/generic/stpcpy.c (stpcpy): Likewise. * sysdeps/x86_64/stpcpy.S (stpcpy): Likewise. * sysdeps/i386/i586/memcpy.S (memcpy): Remove libc_hidden_builtin_def if MEMPCPY_P. * sysdeps/x86_64/memcpy.S (memcpy): Likewise. * sysdeps/i386/i686/mempcpy.S (mempcpy): Add libc_hidden_builtin_def. * sysdeps/i386/i586/mempcpy.S (mempcpy): Likewise. * sysdeps/generic/mempcpy.c (mempcpy): Likewise. * sysdeps/x86_64/mempcpy.S (mempcpy): Likewise. |
||
Andreas Jaeger
|
78df0fcb80 |
Update.
* sysdeps/x86_64/dl-machine.h (elf_machine_runtime_setup): Declare external functions with hidden attribute. (elf_machine_rela): Optimize. * sysdeps/x86_64/memset.S: New file. * sysdeps/x86_64/bzero.S: New file. * sysdeps/x86_64/stpcpy.S: New file. * sysdeps/x86_64/strcat.S: New file. * sysdeps/x86_64/strchr.S: New file. * sysdeps/x86_64/strcpy.S: New file. * sysdeps/x86_64/strcspn.S: New file. * sysdeps/x86_64/strlen.S: New file. * sysdeps/x86_64/strpbrk.S: New file. * sysdeps/x86_64/strspn.S: New file. * sysdeps/x86_64/strcmp.S: New file. * sysdeps/x86_64/strtok_r.S: New file. * sysdeps/x86_64/strtok.S: New file. * sysdeps/x86_64/memcpy.S: New file. * sysdeps/x86_64/mempcpy.S: New file. |