The old implementation was either using CRC32 on modern processors
or a trivial, but rather slow implementation.
We can't continue with CRC32, as that implementation can only
give us 32bit hashes, where we now need to support 64bit in Qt 6.
Change the implementation to use MurmurHash, as public domain
implementation that is both very fast and leads to well distributed hashes.
This hash function is about as fast as the SSE optimized CRC32 implementation
but works everywhere and gives us 64 bit hash values.
Here are some numbers (time for 10M hashes):
14 char 16 char
QByteArray QString float
old qHash (non CRC32) 127ms 134ms 48ms
old qHash (using SSE CRC32 instructions 60ms 62ms 46ms
new qHash 52ms 43ms 46ms
Unfortunately MurmurHash is not safe against hash table DoS attacks, as
potential hash collisions are indepenent of the seed. This will get
addressed in followup commit, where we use SipHash or an SSE optimized
AES based hashing algorithm that does not have those issues.
Change-Id: I4fbc0ac299215b6db78c7a0a2a1d7689b0ea848b
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
At the same time use the opportunity to refactor the
insertion code inside the implementation of QHash to
avoid copy and move constructors as much as possible
and always construct nodes in place.
Change-Id: I951b4cf2c77a17f7db825c6a776aae38c2662d23
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This is required, so that QHash and QSet can hold more
than 2^32 items on 64 bit platforms.
The actual hashing functions for strings are still 32bit, this will
be changed in a follow-up commit.
Change-Id: I4372125252486075ff3a0b45ecfa818359fe103b
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Make use of the new features available in QHash and do a more
performant implementation than the old one.
Change-Id: Ie74b3cdcc9871cd241aca205672093dc395d04a7
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
A brand new QHash implementation using a faster and more memory efficient data
structure than the old QHash.
A new implementation for QHash. Instead of a node based approach as the old
QHash, this implementation now uses a two stage lookup table. The total
amount of buckets in the table are divided into spans of 128 entries.
Inside each span, we use an array of chars to index into a storage area
for the span.
The storage area for each span is a simple array, that gets (re-)allocated
with size increments of 16 items. This gives an average memory overhead of
8*sizeof(struct{ Key; Value; }) + 128*sizeof(char) + 16 for each span.
To give good performance and avoid too many collisions, the array keeps its
load factor between .25 and .5 (and grows and rehashes if the load factor goes
above .5).
This design allows us to keep the memory overhead of the Hash very small, while
at the same time giving very good performance. The calculated overhead for a
QHash<int, int> comes to 1.7-3.3 bytes per entry and to 2.2-4.3 bytes for
a QHash<ptr, ptr>.
The new implementation also completely splits the QHash and QMultiHash classes.
One behavioral change to note is that the new QHash implementation will not
provide stable references to nodes in the hash when the table needs to grow.
Benchmarking using https://github.com/Tessil/hash-table-shootout shows
very nice performance compared to many different hash table implementation.
Numbers shown below are for a hash<int64, int64> with 1 million entries. These
numbers scale nicely (mostly in a linear fashion with some variation due to
varying load factors) to smaller and larger tables. All numbers are in seconds,
measured with gcc on Linux:
Hash table random random random random reads full
insertion insertion full full after iteration
(reserved) deletes reads deletes
------------------------------------------------------------------------------
std::unordered_map 0,3842 0,1969 0,4511 0,1300 0,1169 0,0708
google::dense_hash_map 0,1091 0,0846 0,0550 0,0452 0,0754 0,0160
google::sparse_hash_map 0,2888 0,1582 0,0948 0,1020 0,1348 0,0112
tsl::sparse_map 0,1487 0,1013 0,0735 0,0448 0,0505 0,0042
old QHash 0,2886 0,1798 0,5065 0,0840 0,0717 0,1387
new QHash 0,0940 0,0714 0,1494 0,0579 0,0449 0,0146
Numbers for hash<std::string, int64>, with the string having 15 characters:
Hash table random random random random reads
insertion insertion full full after
(reserved) deletes reads deletes
--------------------------------------------------------------------
std::unordered_map 0,4993 0,2563 0,5515 0,2950 0,2153
google::dense_hash_map 0,2691 0,1870 0,1547 0,1125 0,1622
google::sparse_hash_map 0,6979 0,3304 0,1884 0,1822 0,2122
tsl::sparse_map 0,4066 0,2586 0,1929 0,1146 0,1095
old QHash 0,3236 0,2064 0,5986 0,2115 0,1666
new QHash 0,2119 0,1652 0,2390 0,1378 0,0965
Memory usage numbers (in MB for a table with 1M entries) also look very nice:
Hash table Key int64 std::string (15 chars)
Value int64 int64
---------------------------------------------------------
std::unordered_map 44.63 75.35
google::dense_hash_map 32.32 80,60
google::sparse_hash_map 18.08 44.21
tsl::sparse_map 20.44 45,93
old QHash 53.95 69,16
new QHash 23.23 51,32
Fixes: QTBUG-80311
Change-Id: I5679734144bc9bca2102acbe725fcc2fa89f0dff
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Conflicts:
examples/opengl/doc/src/cube.qdoc
src/corelib/global/qlibraryinfo.cpp
src/corelib/text/qbytearray_p.h
src/corelib/text/qlocale_data_p.h
src/corelib/time/qhijricalendar_data_p.h
src/corelib/time/qjalalicalendar_data_p.h
src/corelib/time/qromancalendar_data_p.h
src/network/ssl/qsslcertificate.h
src/widgets/doc/src/graphicsview.qdoc
src/widgets/widgets/qcombobox.cpp
src/widgets/widgets/qcombobox.h
tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp
tests/auto/widgets/widgets/qcombobox/tst_qcombobox.cpp
tests/benchmarks/corelib/io/qdiriterator/qdiriterator.pro
tests/manual/diaglib/debugproxystyle.cpp
tests/manual/diaglib/qwidgetdump.cpp
tests/manual/diaglib/qwindowdump.cpp
tests/manual/diaglib/textdump.cpp
util/locale_database/cldr2qlocalexml.py
util/locale_database/qlocalexml.py
util/locale_database/qlocalexml2cpp.py
Resolution of util/locale_database/ are based on:
https://codereview.qt-project.org/c/qt/qtbase/+/294250
and src/corelib/{text,time}/*_data_p.h were then regenerated by
running those scripts.
Updated CMakeLists.txt in each of
tests/auto/corelib/serialization/qcborstreamreader/
tests/auto/corelib/serialization/qcborvalue/
tests/auto/gui/kernel/
and generated new ones in each of
tests/auto/gui/kernel/qaddpostroutine/
tests/auto/gui/kernel/qhighdpiscaling/
tests/libfuzzer/corelib/text/qregularexpression/optimize/
tests/libfuzzer/gui/painting/qcolorspace/fromiccprofile/
tests/libfuzzer/gui/text/qtextdocument/sethtml/
tests/libfuzzer/gui/text/qtextdocument/setmarkdown/
tests/libfuzzer/gui/text/qtextlayout/beginlayout/
by running util/cmake/pro2cmake.py on their changed .pro files.
Changed target name in
tests/auto/gui/kernel/qaction/qaction.pro
tests/auto/gui/kernel/qaction/qactiongroup.pro
tests/auto/gui/kernel/qshortcut/qshortcut.pro
to ensure unique target names for CMake
Changed tst_QComboBox::currentIndex to not test the
currentIndexChanged(QString), as that one does not exist in Qt 6
anymore.
Change-Id: I9a85705484855ae1dc874a81f49d27a50b0dcff7
std::function does not have deduction guides in older libc++ (presumably older
than version 10). Omitting the template parameter isn't essential for the test,
so just give it.
Change-Id: Ia9bb91f961b0928203737ec976913effd06433e0
Reviewed-by: Jüri Valdmann <juri.valdmann@qt.io>
Test fails on MinGW 8.1 x86, but not on MinGW 8.1 x86_64.
Task-number: QTQAINFRA-3304
Task-number: QTBUG-69947
Change-Id: Ie9a35bd6d5a8481028cd0ea426d1cf00bd7cf093
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
A common pattern in declarative is to use the unused bits in linked list
next pointers for additional information storage. The "next" pointer is
typically then a tagged pointer of the containing class, which is not
fully defined yet. Therefore alignof() can't be used at tagged pointer
instantiation time. This patch delays the calls to alignment, etc. until
the corresponding functions are used, as in principle the tagged pointer
is just a quintptr and no additional information should be needed until
operating on it.
Change-Id: I87a3578ee921d471e1b60ed5903b549ef0610b97
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Useful for attaching small bits of information in the alignment bits of
a naked pointer. For use in the new property system as well as in
qtdeclarative (where currently a similar class exists as private API).
Change-Id: Idf9b93e714e15129f302e16425dbeda94bcd207b
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Allow for more than 2^31 items and large offsets.
Change-Id: I42f7bf20ce0e4af43dbb2e2083abf0e232e68282
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
QLinkedList has been deprecated, but we still need to test it. Suppress
the warnings for QLinkedList used in tests. Note, that I had to move
some of the test code, to avoid repeating
QT_WARNING_PUSH/QT_WARNING_POP everywhere.
Change-Id: I4203b3ef50045c4f45475a08638dbdc60f68761d
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
This patch adds the arrow operator to the stl-like key-value
iterator (QKeyValueIterator) for QMap and QHash.
This allows using normal member access syntax it->first and it->second
instead of having to use (*it).first and (*it).second.
[ChangeLog][QtCore][Containers] Added operator-> to the key-value
iterator for QHash/QMap.
Change-Id: I9cfa6480784ebce147fcfbf37fec5ad0080e2899
Reviewed-by: Vitaly Fanaskov <vitaly.fanaskov@qt.io>
QLinkedList has been moved to Qt5Compat. Remove and stop mentioning
it in docs, examples (the docs & examples for QLinkedList itself will
be moved to Qt5Compat) and remove the corresponding tests.
Also remove QT_NO_LINKED_LIST, since it's not needed anymore.
Task-number: QTBUG-81630
Task-number: QTBUG-80312
Change-Id: I4a8f1105cb60aa87e7fd67e901ec1a27c489aa31
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Conflicts:
tests/manual/rhi/hellominimalcrossgfxtriangle/CMakeLists.txt
Hopefully final merge from wip/cmake, and then all cmake changes
should target dev directly.
Change-Id: I29b04c9b0284e97334877c77a32ffdf887dbf95b
Conflicts:
examples/widgets/graphicsview/boxes/scene.h
src/corelib/Qt5CoreMacros.cmake
src/corelib/Qt6CoreMacros.cmake
src/network/ssl/qsslsocket.cpp
src/network/ssl/qsslsocket.h
src/platformsupport/fontdatabases/windows/qwindowsfontenginedirectwrite.cpp
src/testlib/CMakeLists.txt
src/testlib/.prev_CMakeLists.txt
tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp
Disabled building manual tests with CMake for now, because qmake
doesn't do it, and it confuses people.
Done-With: Alexandru Croitor <alexandru.croitor@qt.io>
Done-With: Volker Hilsheimer <volker.hilsheimer@qt.io>
Change-Id: I865ae347bd01f4e59f16d007b66d175a52f1f152
The unspecified segment is between 0 and 1. This is because
QTypeRevision::zero(), as the default revisision needs to be smaller
than any other. At the same time we want explicitly specified non-zero
revisions to be larger than unspecified ones. Breaking this down on a
per segment level gives us the order shown here.
Change-Id: I1cca12f1180eb6f77563fb5b22c3400e118dc5e9
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
This pulls the CMake port, which not only adds CMake files but also
modifies existing code. A brief summary of "seemingly unrelated" changes:
* configure.json was re-formatted to not use multi-line strings. That
is an extension of the Qt JSON parser but not JSON compliant, which
is needed for the configure.json-to-cmake conversion script (python).
* Some moc inclusions were added due to CMake's slightly different way
of handling moc. With the changes the files build with qmake and cmake.
* Since CMake just grep's for the Q_OBJECT macro to determine whether to
call moc (instead of doing pre-processing like qmake), the existing use
of "Q_OBJECT" in our documentation was changed to \Q_OBJECT, which cmake
doesn't see and which is now a qdoc macro.
* QTestLib's qFindTestData was extended to also search in the source
directory known at build time.
What this change also brings is a new way of building modules in Coin by using
YAML configuration files that describe the steps of building and testing in Coin
specific terms. The platform configuration files in qt5 are instructed to use the
old Coin built-in way of testing ("UseLegacyInstructions" feature) but for any
configurations that do not have this, these yaml files in the coin/ sub-directory
are used and shared across repositories.
Change-Id: I1d832c3400e8d6945ad787024ba60e7440225c08
Partially reverts 4f077b7e5f.
Can't overload with forwarding references and lvalue references. Use a single
forwarding reference overload, but take care of not trying to create a
QScopeGuard of reference type and forward instead of moving.
Add tests to ensure calling with both lvalues and rvalues is possible.
Change-Id: Ia034afe0a8feb08246c2c7c154a85cae37421c98
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
QTypeRevision is needed to encode the value of the new two-argument
Q_REVISION(major, minor) macros. Those, in turn are necessary because
the minor version resets to 0, and we need to take the major version
into account when stating revisions for Qt classes.
Task-number: QTBUG-71278
Change-Id: I63eff6eab7d6e4f8f32b359a216767c98947a106
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
A couple of tests in the QHash autotest could iterate beyond
end(), leading to undefined behavior. This is bound to crash
with the new upcoming QHash implementation.
Change-Id: I977fc939e6e472f05b7cb2fa0a79c2d5f8782f45
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Remove support for reverse iteration over a QSet.
Change-Id: I1e9c986a118aea4ebeb7fcdceb41e9ce6593cdb6
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
QHash and QMultiHash are separate classes in the future, and
the iterator is not random access.
Change-Id: I7e1a4162ca964001c8da81a2fd7c41ccae27bdb3
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Split QHash and QMultiHash, and get rid of some compiler warnings.
Change-Id: I48991f097f408ad5c1aa349443e26ab816e0b736
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Prepare the test cases so that QHash and QMultiHash are used
as if they were fully independent classes.
Change-Id: Iaf5d65c8f6321ec2edaef490e657b144619655a0
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
With Class Template Argument Deduction users might want to use the constructor
itself instead of a separate helper function. In both cases it's possible to let
the compiler deduce the template arguments.
Try to make the usefulness of the helper function in the absence of CTAD still
clear in the documentation.
Change-Id: I9b07983c1fb276a6dd9e7ed4c3e606764e9b68ca
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
We don't know which versions these blacklistings actually apply on
unless we actually get macOS 10.14 and 10.15 into the CI and running
tests, so let's start with that, and then granularize the blacklists
after that.
Task-number: QTBUG-75786
Change-Id: Id79642afa50cb20efa2cd209286b6933918d3a4a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Conflicts:
src/corelib/tools/qvector.h
Make QVector(DataPointer dd) public to be able to properly merge
5b4b437b30 from 5.15 into dev.
src/widgets/kernel/qapplication.cpp
tests/auto/tools/moc/allmocs_baseline_in.json
Done-With: Christian Ehrlicher <ch.ehrlicher@gmx.de>
Change-Id: I929ba7c036d570382d0454c2c75f6f0d96ddbc01
Add a light-weight associative container, API-wise very similar to QMap,
that's based on two sorted continuous containers (QVector by default).
The class is internal for now.
Change-Id: Ife12576c4abb39a3ea2acb0a1ba0faca91b3a4c5
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
It is useful to use an array of bit as an integer value.
I add also a prarameter to set endianness when converting
value to UInt32.
[ChangeLog][QtCore][QBitArray] Added toUInt32() to return
the bit array's value as a uint32_t.
Change-Id: I9d8c7a33f11e7ce94cb67aa9a50b11fa42d56168
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
As opposed to unite(), this inserts one map into the other
without duplicating elements.
Task-number: QTBUG-35544
Change-Id: Ie8ab350b29148851a3176cef1007e8a4ca82c273
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
We already detach immediately since change
c2d2757bcc. That basically removes
the main purpose of having QChar/ByteRef, and we can just as well
get rid of those classes for Qt 6.
Change-Id: I8dc566a1948ddc29c0cb8a77ec7310654a7219a4
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
That allows us to pass by value for all fundamental and pointer types.
This requires some magic to remove methods taking a T&& to avoid
ambiguous overloads for QVector<int/qsizetype>. Remove them for all
cases where parameter_type is T, as copying or moving will do
exactly the same thing for those types.
Change-Id: I8133fecd3ac29bb8f6ae57376e680bc3d616afbf
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
