QMultiHash has access to two sizes: one of them is shared with QHash,
stored in QHashPrivate::Data::size, which counts keys; the other, which
is what our public size() function returns, is stored in
QMultiHash::m_size and counts plain (key,value) entries. We forgot to
update it in the non-const operator[] that created a node.
I've reviewed the rest of the code and can't find any more places where
the item count may be changed and m_size isn't updated.
[ChangeLog][QtCore][QMultiHash] Fixed a bug that caused an element that
was created by operator[] to not be counted, resulting in a hash map
with an incorrect element count and which could cause an assertion
failure depending on how the hash was later mutated.
Fixes: QTBUG-112534
Pick-to: 6.2 6.4 6.5
Change-Id: Idd5e1bb52be047d7b4fffffd17527ec274e1d99e
Reviewed-by: Lars Knoll <lars@knoll.priv.no>
It was confusing entry capacity with the bucket capacity. The value
maxNumBuckets() returned was the maximum number of entries. This issue
was harmless: we would just fail to cap the maximum to an allocatable
size. But the array new[] in the Data constructors would have capped the
maximum anyway (by way of throwing std::bad_alloc).
So instead of trying to calculate what the maximum bucket count is so we
can cap at that, simplify the calculation of the next power of 2 while
preventing it from overflowing in our calculations. We continue to rely
on new[] throwing when we return count that is larger than the maximum
allocatable.
This commit changes the load factor for QHashes containing exactly a
number of elements that is exactly a power of two. Previously, it would
be loaded at 50%, now it's at 25%. For this reason, tst_QSet::squeeze
needed to be fixed to depend less on the implementation details.
Pick-to: 6.5
Change-Id: I9671dee8ceb64aa9b9cafffd17415f3856c358a0
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This is a semantic patch using ClangTidyTransformator as in
qtbase/df9d882d41b741fef7c5beeddb0abe9d904443d8, but extended to
handle typedefs and accesses through pointers, too:
const std::string o = "object";
auto hasTypeIgnoringPointer = [](auto type) { return anyOf(hasType(type), hasType(pointsTo(type))); };
auto derivedFromAnyOfClasses = [&](ArrayRef<StringRef> classes) {
auto exprOfDeclaredType = [&](auto decl) {
return expr(hasTypeIgnoringPointer(hasUnqualifiedDesugaredType(recordType(hasDeclaration(decl))))).bind(o);
};
return exprOfDeclaredType(cxxRecordDecl(isSameOrDerivedFrom(hasAnyName(classes))));
};
auto renameMethod = [&] (ArrayRef<StringRef> classes,
StringRef from, StringRef to) {
return makeRule(cxxMemberCallExpr(on(derivedFromAnyOfClasses(classes)),
callee(cxxMethodDecl(hasName(from), parameterCountIs(0)))),
changeTo(cat(access(o, cat(to)), "()")),
cat("use '", to, "' instead of '", from, "'"));
};
renameMethod(<classes>, "count", "size");
renameMethod(<classes>, "length", "size");
except that the on() matcher has been replaced by one that doesn't
ignoreParens().
a.k.a qt-port-to-std-compatible-api V5 with config Scope: 'Container'.
Added two NOLINTNEXTLINEs in tst_qbitarray and tst_qcontiguouscache,
to avoid porting calls that explicitly test count().
Change-Id: Icfb8808c2ff4a30187e9935a51cad26987451c22
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Inserting the same key repeatedly with QMultiHash will not
test rehashing behavior because in Qt6 those entries all
end up in a linked list.
Pick-to: 6.4
Change-Id: I78c45eed0f35a13af6d6da75d7189a6933750f13
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
These functions are marked as deprecated in future Qt releases.
Task-number: QTBUG-104858
Change-Id: I25d2932455d8c9e3e2d722b1c48fc2cfa2d1e679
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
When calling QHash::reserve(), or when creating the
internal QHashPrivate::Data structure, the value 0
for the size parameter is reserved for performing
the squeeze operation.
However commit 8a984ab772
broke it, by using the 0 value in QHashPrivate::Data
constructors as a mark that no resizing needs to be done.
This patch reverts the problematic commit (also applying
some later fixes to the code), and adds the missing
tests for Q[Multi]Hash::squeeze().
Pick-to: 6.3 6.2
Change-Id: Id644df7b2beb008e6a37b2c89b709adfbd893e25
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Calling Q[Multi]Hash::reserve(n) when n is much smaller than the
current amount of elements in the hash, could result in an infinite
loop, because at some point the algorithm could not find a free bucket
for the element.
Fixing it by returning early if the new desired capacity is less than
current.
Fixes: QTBUG-102067
Pick-to: 6.3 6.2
Change-Id: I38ef0b2168c4e2a317eedf91b2155b1fdffb1c27
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Don't mix unsigned and signed types in comparisons.
Pick-to: 6.3
Change-Id: Ia4ba9c114177425a21cadc8cafe8179928315a5d
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
qt_internal_undefine_global_definition disables an internal global
definition that is defined by the qt_internal_add_global_definition
function for a specific target.
Remove the ability to set the custom "undefine" flag for the
definitions since it's hard to control it using the introduced
function.
Pick-to: 6.2 6.3
Task-number: QTBUG-100334
Change-Id: Ic1637d97aa51bbdd06c5b191c57a941aa208d4dc
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Restore the 'QT_NO_JAVA_STYLE_ITERATORS' and
'QT_NO_NARROWING_CONVERSIONS_IN_CONNECT' definitions for Qt
targets.
Add the function that adds global definitions for Qt targets according
to the provided scope and the target property-based switch to disable
the definition for a specific target.
Pick-to: 6.2 6.3
Task-number: QTBUG-100295
Change-Id: I28697e81f9aabc45c48d79aae1e5caea141e04e1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
By using the bucketForHash function we can loop through and find
some appropriate keys to test the edge-case. This will then
automatically keep the test working even if some internals
of QHash changes.
We do this because certain changes which change the bucket the
pre-selected keys would end up in could make the test a no-op,
without warning. And recent and upcoming changes have changed
both this and erase(). We limit the search-space to
the minimum numBuckets * 4, where minimum numBuckets is current
128.
Change-Id: I13b0bce15ee884144e3248846be34667fb5d35cc
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
QHash::operator[] could grow the hash even if the key being
looked up already existed. This in turn invalidated all iterators.
Avoid this by refactoring findOrInsert() to not grow if the key
already exists.
Added advantage is that this should make lookups of existing keys
slightly faster.
Fixes: QTBUG-97752
Pick-to: 6.3 6.2
Change-Id: I9df30459797b42c434ba0ee299fd1d55af8d2313
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
When deleting the last item in a chain, without it being the last item
in the chain, then we re-use the iterator which was passed in as an
argument. This is wrong if we detached earlier in the function, and
means we return an iterator to the previously shared data.
Pick-to: 6.2
Change-Id: I7da6309e23a32073da59e7da0cbfd1d16734f1ca
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
If we detach from a shared hash while holding a reference to a key from
said shared hash then there is no guarantee for how long the reference
is valid (given a multi-thread environment).
Pick-to: 6.2
Change-Id: Ifb610753d24faca63e2c0eb8836c78d55a229001
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
QMultiHash::operator== crashes when comparing two unequal objects.
This patch fixes it.
Pick-to: 6.2
Fixes: QTBUG-98265
Change-Id: Ibf9fef3372a2b4581843be5f25e65cc9a55ef64d
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
QHash::squeeze() was unconditionally calling reserve(0), which is
always allocating memory (even for 0 size).
This was leading to a confusing situation when calling squeeze() on
a default-constructed container with 0 capacity() actually allocated
memory. This is very misleading, as squeeze() is supposed to free
unneeded memory, not to allocate more.
This patch adds a check for non-zero capacity. As a result, nothing
is done for default-constructed container.
Note that this patch also affects the QSet::squeeze() behavior, because
QSet uses QHash as its underlying data type.
Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: Ib1c3c8b7b3de6ddeefea0e70b1ec71803e8fd3b3
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Andreas Buhr <andreas.buhr@qt.io>
Extend tests to explicitly check the behavior of empty
default-constructed containers.
Also add some missing tests for the existing methods
(mostly for QMultiHash) and correct some end()s to cend()s
in comparisons.
Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: Ic9e1b86ef67f6bca2751a65a8589b2f7e0ebb5ea
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The method was never tested, but it failed to compile after
QMultiHash was introduced as a separate class in 6.0.
This patch fixes it and adds some unit-tests to cover the case.
Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: I5dd989d4775efc6a9bb13c5ed1d892e499d95dc2
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Do not detach when find(key, value) is called on an empty QMultiHash.
As a drive-by: fix return value for QMultiHash::remove() in case of
empty QMultiHash.
Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: I1e32f359e7ee9ce8403dae79d02e0b88a20ec4a5
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
When the element you want to erase is the last element AND the
next element (element 0), when rehashed, would be relocated to the last
element, this leads to the state below. Which is similar to a test in
tst_qhash for some seeds.
auto it = hash.begin + (hash.size - 1)
it = hash.erase(it)
it != hash.end
By forcing the iterator to increment if we were erasing the last element
we always end up with a pointer which is equal to hash.end
Befriend the tst_qhash class so we can set the seed to a known-bad one
Pick-to: 6.2 6.1
Change-Id: Ie0b175003a2acb175ef5e3ab5a984e010f65d986
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
As QMultiHash uses a pointer for the data, nullptr dereference is a
thing, so check for valid d before doing anything in count()
Fixes: QTBUG-91704
Pick-to: 6.0 6.1
Change-Id: Ia20440cd7bdc03cb09c77f796fb9c5b52765eac5
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Those serve no purpose anymore, now that the .pro files are gone.
Task-number: QTBUG-88742
Change-Id: I39943327b8c9871785b58e9973e4e7602371793e
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
Remove the qmake project files for most of Qt.
Leave the qmake project files for examples, because we still test those
in the CI to ensure qmake does not regress.
Also leave the qmake project files for utils and other minor parts that
lack CMake project files.
Task-number: QTBUG-88742
Change-Id: I6cdf059e6204816f617f9624f3ea9822703f73cc
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
QMultiHash::equal_range crashes when called in a const member function.
The Data `d` is a NULL pointer when calling equal_range()
before inserting data into an empty QMultiHash.
Then calling`d->find` crashes.
Fixes: QTBUG-89687
Pick-to: 6.0
Change-Id: I10c3d196cbc72aed8c8c922ef16534bba51037b7
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Complete search and replace of QtTest and QtTest/QtTest with QTest, as
QtTest includes the whole module. Replace all such instances with
correct header includes. See Jira task for more discussion.
Fixes: QTBUG-88831
Change-Id: I981cfae18a1cabcabcabee376016b086d9d01f44
Pick-to: 6.0
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
In addition (and as a fallback) from requiring qHash, add support
for std::hash specializations. This catches two birds with one stone:
1) users of Qt can simply specialize std::hash for their datatypes,
and use them in both QHash and stdlib unordered associative containers;
2) we get QHash support for any (stdlib) datatype that is hashable
without having to overload qHash for them.
[ChangeLog][QtCore][QHash] QHash, QMultiHash and QSet now support
for key types anything that can be hashed via std::hash, instead of
always requiring a qHash() overload.
Change-Id: Ib5ecba86e4b376d318389500bd24883ac6534c5f
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
QChar should not be convertible from any integral type except from
char16_t, short and possibly char (since it's a direct superset).
David provided the perfect example:
if (str == 123) { ~~~ }
compiles, with 123 implicitly converted to QChar (str == "123"
was meant instead). But similarly one can construct other
scenarios where QString(123) gets accidentally used (instead of
QString::number(123)), like QString s; s += 123;.
Add a macro to revert to the implicit constructors, for backwards
compatibility.
The breaks are mostly in tests that "abuse" of integers (arithmetic,
etc.). Maybe it's time for user-defined literals for QChar/QString,
but that is left for another commit.
[ChangeLog][Potentially Source-Incompatible Changes][QChar] QChar
constructors from integral types are now by default explicit.
It is recommended to use explicit conversions, QLatin1Char,
QChar::fromUcs4 instead of implicit conversions. The old behavior
can be restored by defining the QT_IMPLICIT_QCHAR_CONSTRUCTION
macro.
Change-Id: I6175f6ab9bcf1956f6f97ab0c9d9d5aaf777296d
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reaches into the internals to avoid erasing one entry at a time from the
QHash.
Change-Id: I47079592d130d2ecd844998dfa31e633e049d4c1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
And add a QMultiHash::unite(const QHash &) method to avoid
a copy of the data when inserting a QHash into a multi hash.
Change-Id: I864aa9d2b9b7b2c367c3c4d140a2ce2f5408ae09
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The old code was trying to convert a multi hash to a QHash. While
that worked in Qt 5 it won't compile in Qt 6 anymore.
QHashCombineCommutative also can't be used with a std::pair.
ADL won't find the correct instance with a namespaced build,
as qHash(std::pair) is defined after QHashCommutative. Fix
the code to compile and work correctly.
Change-Id: Ice2bc3ab4244e310cbbb5e0f31fc11eb14f5faf3
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Modify special case locations to use the new API as well.
Clean up some stale .prev files that are not needed anymore.
Clean up some project files that are not used anymore.
Task-number: QTBUG-86815
Change-Id: I9947da921f98686023c6bb053dfcc101851276b5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Use pro2cmake with '--api-version 2' to force regenerate
projects to use the new prefixed qt_foo APIs.
Change-Id: I055c4837860319e93aaa6b09d646dda4fc2a4069
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
There is no reason for keep using our macro now that we have C++17.
The macro itself is left in for the moment being, as well as its
detection logic, because it's needed for C code (not everything
supports C11 yet). A few more cleanups will arrive in the next few
patches.
Note that this is a mere search/replace; some places were using
double braces to work around the presence of commas in a macro, no
attempt has been done to fix those.
tst_qglobal had just some minor changes to keep testing the macro.
Change-Id: I1c1c397d9f3e63db3338842bf350c9069ea57639
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
At the same time use the opportunity to refactor the
insertion code inside the implementation of QHash to
avoid copy and move constructors as much as possible
and always construct nodes in place.
Change-Id: I951b4cf2c77a17f7db825c6a776aae38c2662d23
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This is required, so that QHash and QSet can hold more
than 2^32 items on 64 bit platforms.
The actual hashing functions for strings are still 32bit, this will
be changed in a follow-up commit.
Change-Id: I4372125252486075ff3a0b45ecfa818359fe103b
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
A brand new QHash implementation using a faster and more memory efficient data
structure than the old QHash.
A new implementation for QHash. Instead of a node based approach as the old
QHash, this implementation now uses a two stage lookup table. The total
amount of buckets in the table are divided into spans of 128 entries.
Inside each span, we use an array of chars to index into a storage area
for the span.
The storage area for each span is a simple array, that gets (re-)allocated
with size increments of 16 items. This gives an average memory overhead of
8*sizeof(struct{ Key; Value; }) + 128*sizeof(char) + 16 for each span.
To give good performance and avoid too many collisions, the array keeps its
load factor between .25 and .5 (and grows and rehashes if the load factor goes
above .5).
This design allows us to keep the memory overhead of the Hash very small, while
at the same time giving very good performance. The calculated overhead for a
QHash<int, int> comes to 1.7-3.3 bytes per entry and to 2.2-4.3 bytes for
a QHash<ptr, ptr>.
The new implementation also completely splits the QHash and QMultiHash classes.
One behavioral change to note is that the new QHash implementation will not
provide stable references to nodes in the hash when the table needs to grow.
Benchmarking using https://github.com/Tessil/hash-table-shootout shows
very nice performance compared to many different hash table implementation.
Numbers shown below are for a hash<int64, int64> with 1 million entries. These
numbers scale nicely (mostly in a linear fashion with some variation due to
varying load factors) to smaller and larger tables. All numbers are in seconds,
measured with gcc on Linux:
Hash table random random random random reads full
insertion insertion full full after iteration
(reserved) deletes reads deletes
------------------------------------------------------------------------------
std::unordered_map 0,3842 0,1969 0,4511 0,1300 0,1169 0,0708
google::dense_hash_map 0,1091 0,0846 0,0550 0,0452 0,0754 0,0160
google::sparse_hash_map 0,2888 0,1582 0,0948 0,1020 0,1348 0,0112
tsl::sparse_map 0,1487 0,1013 0,0735 0,0448 0,0505 0,0042
old QHash 0,2886 0,1798 0,5065 0,0840 0,0717 0,1387
new QHash 0,0940 0,0714 0,1494 0,0579 0,0449 0,0146
Numbers for hash<std::string, int64>, with the string having 15 characters:
Hash table random random random random reads
insertion insertion full full after
(reserved) deletes reads deletes
--------------------------------------------------------------------
std::unordered_map 0,4993 0,2563 0,5515 0,2950 0,2153
google::dense_hash_map 0,2691 0,1870 0,1547 0,1125 0,1622
google::sparse_hash_map 0,6979 0,3304 0,1884 0,1822 0,2122
tsl::sparse_map 0,4066 0,2586 0,1929 0,1146 0,1095
old QHash 0,3236 0,2064 0,5986 0,2115 0,1666
new QHash 0,2119 0,1652 0,2390 0,1378 0,0965
Memory usage numbers (in MB for a table with 1M entries) also look very nice:
Hash table Key int64 std::string (15 chars)
Value int64 int64
---------------------------------------------------------
std::unordered_map 44.63 75.35
google::dense_hash_map 32.32 80,60
google::sparse_hash_map 18.08 44.21
tsl::sparse_map 20.44 45,93
old QHash 53.95 69,16
new QHash 23.23 51,32
Fixes: QTBUG-80311
Change-Id: I5679734144bc9bca2102acbe725fcc2fa89f0dff
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This patch adds the arrow operator to the stl-like key-value
iterator (QKeyValueIterator) for QMap and QHash.
This allows using normal member access syntax it->first and it->second
instead of having to use (*it).first and (*it).second.
[ChangeLog][QtCore][Containers] Added operator-> to the key-value
iterator for QHash/QMap.
Change-Id: I9cfa6480784ebce147fcfbf37fec5ad0080e2899
Reviewed-by: Vitaly Fanaskov <vitaly.fanaskov@qt.io>
This pulls the CMake port, which not only adds CMake files but also
modifies existing code. A brief summary of "seemingly unrelated" changes:
* configure.json was re-formatted to not use multi-line strings. That
is an extension of the Qt JSON parser but not JSON compliant, which
is needed for the configure.json-to-cmake conversion script (python).
* Some moc inclusions were added due to CMake's slightly different way
of handling moc. With the changes the files build with qmake and cmake.
* Since CMake just grep's for the Q_OBJECT macro to determine whether to
call moc (instead of doing pre-processing like qmake), the existing use
of "Q_OBJECT" in our documentation was changed to \Q_OBJECT, which cmake
doesn't see and which is now a qdoc macro.
* QTestLib's qFindTestData was extended to also search in the source
directory known at build time.
What this change also brings is a new way of building modules in Coin by using
YAML configuration files that describe the steps of building and testing in Coin
specific terms. The platform configuration files in qt5 are instructed to use the
old Coin built-in way of testing ("UseLegacyInstructions" feature) but for any
configurations that do not have this, these yaml files in the coin/ sub-directory
are used and shared across repositories.
Change-Id: I1d832c3400e8d6945ad787024ba60e7440225c08
A couple of tests in the QHash autotest could iterate beyond
end(), leading to undefined behavior. This is bound to crash
with the new upcoming QHash implementation.
Change-Id: I977fc939e6e472f05b7cb2fa0a79c2d5f8782f45
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
