glibc/string
Wilco Dijkstra b58bace7ce Improve performance of memmem
This patch significantly improves performance of memmem using a novel
modified Horspool algorithm.  Needles up to size 256 use a bad-character
table indexed by hashed pairs of characters to quickly skip past mismatches.
Long needles use a self-adapting filtering step to avoid comparing the whole
needle repeatedly.

By limiting the needle length to 256, the shift table only requires 8 bits
per entry, lowering preprocessing overhead and minimizing cache effects.
This limit also implies worst-case performance is linear.

Small needles up to size 2 use a dedicated linear search.  Very long needles
use the Two-Way algorithm (to avoid increasing stack size or slowing down
the common case, inlining is disabled).

The performance gain is 6.6 times on English text on AArch64 using random
needles with average size 8.

Tested against GLIBC testsuite and randomized tests.

Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>

	* string/memmem.c (__memmem): Rewrite to improve performance.

(cherry picked from commit 680942b016)
2019-09-13 15:19:26 +01:00
..
bits Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
_strerror.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-addsep.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-append.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-count.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-create.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-ctsep.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-delete.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-extract.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-insert.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-next.c Use libc_hidden_* for argz_next, __argz_next (bug 15105). 2018-02-15 21:00:02 +00:00
argz-replace.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz-stringify.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
argz.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
basename.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
bcopy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
bug-envz1.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
bug-strcoll1.c Update. 2001-04-26 20:45:18 +00:00
bug-strcoll2.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
bug-strncat1.c Fix string/bug-strncat1.c build with GCC 8. 2017-11-14 17:50:36 +00:00
bug-strpbrk1.c Fix bug-strspn1.c, bug-strpbrk1.c build with GCC mainline. 2018-06-20 22:18:22 +00:00
bug-strspn1.c Fix bug-strspn1.c, bug-strpbrk1.c build with GCC mainline. 2018-06-20 22:18:22 +00:00
bug-strtok1.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
byteswap.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
bzero.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
Depend Update. 2001-03-19 21:40:15 +00:00
endian.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
envz.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
envz.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
explicit_bzero.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ffs.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
ffsll.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
inl-tester.c Update. 1997-09-11 12:09:10 +00:00
Makefile Only build libm with -fno-math-errno (bug 24024) 2019-01-08 21:06:25 +01:00
memccpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memfrob.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memmem.c Improve performance of memmem 2019-09-13 15:19:26 +01:00
memmove.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memory.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
mempcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
memset.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
noinl-tester.c Update. 1997-09-16 00:42:43 +00:00
rawmemchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
stpcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
stpncpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
str-two-way.h Improve performance of strstr 2019-09-13 15:19:19 +01:00
stratcliff.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp_l.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasecmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcasestr.c Simplify and speedup strstr/strcasestr first match 2019-09-13 15:15:36 +01:00
strcat.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strchrnul.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcoll_l.c Fix another -Os strcoll build issue. 2018-02-26 18:38:01 +00:00
strcoll.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strcspn.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strdup.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strerror_l.c libc: Extend __libc_freeres framework (Bug 23329). 2018-06-29 22:39:06 -04:00
strerror.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strfry.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
string-inlines.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
string.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strings.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase_l.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncase.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncat.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strncpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strndup.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strnlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strpbrk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strsep.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strsignal.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strspn.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strstr.c Improve performance of strstr 2019-09-13 15:19:19 +01:00
strtok_r.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strtok.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strverscmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strxfrm_l.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
strxfrm.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
swab.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-bcopy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-bzero.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-endian-file-scope.c Unify and simplify bits/byteswap.h, bits/byteswap-16.h headers (bug 14508, bug 15512, bug 17082, bug 20530). 2018-02-06 21:55:08 +00:00
test-endian-sign-conversion.c Unify and simplify bits/byteswap.h, bits/byteswap-16.h headers (bug 14508, bug 15512, bug 17082, bug 20530). 2018-02-06 21:55:08 +00:00
test-endian-types.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-explicit_bzero.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-ffs.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-memccpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-memchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-memcmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-memcpy.c Add a test case for [BZ #23196] 2018-05-23 04:00:11 -07:00
test-memmem.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-memmove.c Fix i386 memmove issue (bug 22644). 2018-03-23 16:19:45 +01:00
test-mempcpy.c Don't write beyond destination in __mempcpy_avx512_no_vzeroupper (bug 23196) 2018-05-23 09:50:57 +02:00
test-memrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-memset.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-rawmemchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-stpcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-stpncpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strcasecmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strcasestr.c Improve strstr performance 2018-07-16 17:51:52 +01:00
test-strcat.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strchrnul.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strcmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strcpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strcspn.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strerror-errno.c Only build libm with -fno-math-errno (bug 24024) 2019-01-08 21:06:25 +01:00
test-string.h Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strncasecmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strncat.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strncmp.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strncpy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strnlen.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strpbrk.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strrchr.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strspn.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
test-strstr.c Fix strstr bug with huge needles (bug 23637) 2018-09-21 12:09:38 +01:00
testcopy.c Move string/testcopy.c to test-driver.c and xmalloc (bug 19667). 2018-02-06 21:43:20 +00:00
tester.c Ignore -Wrestrict for one strncat test. 2018-06-14 14:20:00 +00:00
tst-bswap.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
tst-cmp.c Fix tst-cmp.c build with GCC mainline. 2018-06-20 22:19:50 +00:00
tst-endian.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-inlcall.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
tst-strcoll-overflow.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
tst-strfry.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-strlen.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-strtok_r.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
tst-strtok.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-strxfrm2.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-strxfrm.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-svc2.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-svc.c Update string tests to use the support test driver. 2017-03-23 11:32:29 -03:00
tst-svc.expect * string/strverscmp.c (__strverscmp): Fix last cleanups. 2009-04-07 06:51:59 +00:00
tst-svc.input * string/strverscmp.c (__strverscmp): Fix last cleanups. 2009-04-07 06:51:59 +00:00
tst-xbzero-opt.c Fix string/tst-xbzero-opt if build with gcc head. 2018-07-26 17:09:44 +02:00
Versions New string function explicit_bzero (from OpenBSD). 2016-12-16 16:21:54 -05:00
wordcopy.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00
xpg-strerror.c Update copyright dates with scripts/update-copyrights. 2018-01-01 00:32:25 +00:00