qt5base-lts/tests/benchmarks
Ahmad Samir a6ad755734 QStringList: add filter(QStringMatcher) overload
Now that users can pass a QStringMatcher to do the matching, change the
existing overload to not use QStringMatcher.

Thanks to Giuseppe D'Angelo for the idea of passing a QStringMatcher to
filter instead of using a magic number to decide whether to use
QStringMatcher or not.

Results of running filter() and filter_stringMatcher, times are in msecs
and this was compiled with gcc -O3:

              Without       With QStringMatcher
list10        0.00022       0.000089
list20        0.00040       0.00014
list30        0.00058       0.00018
list40        0.000770      0.00023
list50        0.00094       0.00027
list70        0.0012        0.00037
list80        0.0014        0.00041
list100       0.0018        0.00050
list300       0.0054        0.0014
list500       0.0091        0.0023
list700       0.012         0.0032
list900       0.016         0.0041
list10000     0.17          0.045

Drive-by change: optimize tst_QStringList::populateList().

[ChangeLog][QtCore][QStringList] Added filter(const QStringMatcher &)
overload, which may be faster for large lists and/or lists with very
long strings.

[ChangeLog][Possible Performance Changes][QtCore][QStringList] Changed
the implementation of filter(QStringView) overload to not use
QStringMatcher by default. Using QStringMatcher adds overhead, so it is
beneficial/faster when searching for a pattern in large lists and/or
lists with long strings, otherwise using plain string comparison is
faster. If using QStringMatcher makes a difference in your code, you can
use the newly added filter(QStringMatcher) overload.

Change-Id: I7bb1262706d673f0ce0d9b7699f03c995ce28677
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-10-22 14:37:33 +03:00
..
corelib QStringList: add filter(QStringMatcher) overload 2023-10-22 14:37:33 +03:00
dbus tests: Remove remains of qmake conversion from CMakeLists.txt files 2023-02-17 21:56:49 +01:00
gui Mark all of Qt as free of Q_FOREACH, except where it isn't 2023-08-19 05:19:42 +00:00
network QTestEventLoop: add enterLoop(std::chrono::milliseconds) overload 2023-03-03 21:36:48 +02:00
plugins/imageformats/jpeg tests: Remove remains of qmake conversion from CMakeLists.txt files 2023-02-17 21:56:49 +01:00
sql SQL/Benchmarks: cleanup 2023-04-05 05:37:32 +01:00
testlib tests: Remove remains of qmake conversion from CMakeLists.txt files 2023-02-17 21:56:49 +01:00
widgets QtWidgets benchmarks: port remaining users away from Q_FOREACH 2023-08-14 23:11:54 +03:00
CMakeLists.txt tests: Remove remains of qmake conversion from CMakeLists.txt files 2023-02-17 21:56:49 +01:00
README Whitespace cleanup: remove trailing whitespace 2013-03-16 20:22:50 +01:00

The most reliable way of running benchmarks is to do it in an otherwise idle
system. On a busy system, the results will vary according to the other tasks
demanding attention in the system.

We have managed to obtain quite reliable results by doing the following on
Linux (and you need root):

 - switching the scheduler to a Real-Time mode
 - setting the processor affinity to one single processor
 - disabling the other thread of the same core

This should work rather well for CPU-intensive tasks. A task that is in Real-
Time mode will simply not be preempted by the OS. But if you make OS syscalls,
especially I/O ones, your task will be de-scheduled. Note that this includes
page faults, so if you can, make sure your benchmark's warmup code paths touch
most of the data.

To do this you need a tool called schedtool (package schedtool), from
http://freequaos.host.sk/schedtool/

From this point on, we are using CPU0 for all tasks:

If you have a Hyperthreaded multi-core processor (Core-i5 and Core-i7), you
have to disable the other thread of the same core as CPU0. To discover which
one it is:

$ cat /sys/devices/system/cpu/cpu0/topology/thread_siblings_list

This will print something like 0,4, meaning that CPUs 0 and 4 are sibling
threads on the same core. So we'll turn CPU 4 off:

(as root)
# echo 0 > /sys/devices/system/cpu/cpu4/online

To turn it back on, echo 1 into the same file.

To run a task on CPU 0 exclusively, using FIFO RT priority 10, you run the
following:

(as root)
# schedtool -F -p 10 -a 1 -e ./taskname

For example:
# schedtool -F -p 10 -a 1 -e ./tst_bench_qstring -tickcounter

Warning: if your task livelocks or takes far too long to complete, your system
may be unusable for a long time, especially if you don't have other cores to
run stuff on. To prevent that, run it before schedtool and time it.

You can also limit the CPU time that the task is allowed to take. Run in the
same shell as you'll run schedtool:

$ ulimit -s 300
To limit to 300 seconds (5 minutes)

If your task runs away, it will get a SIGXCPU after consuming 5 minutes of CPU
time (5 minutes running at 100%).

If your app is multithreaded, you may want to give it more CPUs, like CPU0 and
CPU1 with -a 3  (it's a bitmask).

For best results, you should disable ALL other cores and threads of the same
processor. The new Core-i7 have one processor with 4 cores,
each core can run 2 threads; the older Mac Pros have two processors with 4
cores each. So on those Mac Pros, you'd disable cores 1, 2 and 3, while on the
Core-i7, you'll need to disable all other CPUs.

However, disabling just the sibling thread seems to produce very reliable
results for me already, with variance often below 0.5% (even though there are
some measurable spikes).

Other things to try:

Running the benchmark with highest priority, i.e. "sudo nice -19"
usually produces stable results on some machines. If the benchmark also
involves displaying something on the screen (on X11), running it with
"-sync" is a must. Though, in that case the "real" cost is not correct,
but it is useful to discover regressions.

Also; not many people know about ionice (1)
      ionice - get/set program io scheduling class and priority