qhashfunctions.h defines a catch-all 2-arguments qHash(T, seed)
in order to support datatypes that implement a 1-argument overload
of qHash (i.e. qHash(Type)). The catch-all calls the 1-argument
overload and XORs the result with the seed.
The catch-all is constrained on the existence of such a 1-argument
overload. This is done in order to make the catch-all SFINAE-friendly;
otherwise merely instantiating the catch-all would trigger a hard error.
Such an error would make it impossible to build a type trait that
detects if one can call qHash(T, size_t) for a given type T.
The constraint itself is called HasQHashSingleArgOverload and lives in a
private namespace.
It has been observed that HasQHashSingleArgOverload misbehaves for
some datatypes. For instance, HasQHashSingleArgOverload<int> is actually
false, despite qHash(123) being perfectly callable. (The second argument
of qHash(int, size_t) is defaulted, so the call *is* possible.)
--
Why is HasQHashSingleArgOverload<int> false?
This has to do with how HasQHashSingleArgOverload<T> is implemented: as
a detection trait that checks if qHash(declval<T>()) is callable.
The detection itself is not a problem. Consider this code:
template <typename T>
constexpr bool HasQHashSingleArgOverload = /* magic */;
class MyClass {};
size_t qHash(MyClass);
static_assert(HasQHashSingleArgOverload<MyClass>); // OK
Here, the static_assert passes, even if qHash(MyClass) (and MyClass
itself) were not defined at all when HasQHashSingleArgOverload was
defined.
This is nothing but 2-phase lookup at work ([temp.dep.res]): the
detection inside HasQHashSingleArgOverload takes into account the qHash
overloads available when HasQHashSingleArgOverload was declared, as well
as any other overload declared before the "point of instantiation". This
means that qHash(MyClass) will be visible and detected.
Let's try something slightly different:
template <typename T>
constexpr bool HasQHashSingleArgOverload = /* magic */;
size_t qHash(int);
static_assert(HasQHashSingleArgOverload<int>); // ERROR
This one *does not work*. How is it possible? The answer is that 2-phase
name lookup combines the names found at definition time with the names
_found at instantiation time using argument-dependent lookup only_.
`int` is a fundamental type and does not participate in ADL. In the
example, HasQHashSingleArgOverload has actually no qHash overloads to
even consider, and therefore its detection fails.
You can restore detection by moving the declaration of the qHash(int)
overload *before* the definition of HasQHashSingleArgOverload, so it's
captured at definition time:
size_t qHash(int);
template <typename T>
constexpr bool HasQHashSingleArgOverload = /* magic */;
static_assert(HasQHashSingleArgOverload<int>); // OK!
This is why HasQHashSingleArgOverload<int> is currently returning
`false`: because HasQHashSingleArgOverload is defined *before* all the
qHash(fundamental_type) overloads in qhashfunctions.h.
--
Now consider this variation of the above, where we keep the qHash(int)
overload after the detector (so, it's not found), but also prepend an
Evil class implicitly convertible from int:
struct Evil { Evil(int); };
size_t qHash(Evil);
template <typename T> constexpr bool HasQHashSingleArgOverload = /* magic */;
size_t qHash(int);
static_assert(HasQHashSingleArgOverload<int>); // OK
Now the static_assert passes. HasQHashSingleArgOverload is still not
considering qHash(int) (it's declared after), but it's considering
qHash(Evil). Can you call *that* one with an int? Yes, after a
conversion to Evil.
This is extremely fragile and likely an ODR violation (if not ODR, then
likely falls into [temp.dep.candidate/1]).
--
Does this "really matter" for a type like `int`? The answer is no. If
HasQHashSingleArgOverload<int> is true, then a call like
qHash(42, 123uz);
will have two overloads in its overloads set:
1) qHash(int, size_t)
2) qHash(T, size_t), i.e. the catch-all template. To be pedantic,
qHash<int>(const int &, size_t), that is, the instantiation of the
catch-all after template type deduction for T (= int)
([over.match.funcs.general/8]).
Although it may look like this is ambiguous as both calls have perfect
matches for the arguments, 1) is actually a better match than 2) because
it is not a template specialization ([over.match.best/2.4]).
In other words: qHash(int, size_t) is *always* called when the argument
is `int`, no matter the value of HasQHashSingleArgOverload<int>. The
catch-all template may be added or not to the overload set, but it's
a worse match anyways.
--
Now, let's consider this code:
enum MyEnum { E1, E2, E3 };
qHash(E1, 42uz);
This code compiles, although we do not define any qHash overload
specifically for enumeration types (nor one is defined by MyEnum's
author).
Which qHash overload gets called? Again there are two possible
overloads available:
1) qHash(int, size_t). E1 can be converted to `int` ([conv.prom/3]),
and this overload selected.
2) qHash(T, size_t), which after instantiation, is qHash<MyEnum>(const
MyEnum &, size_t).
In this case, 2) is a better match than 1), because it does not require
any conversion for the arguments.
Is 2) a viable overload? Unfortunately the answer here is "it depends",
because it's subject to what we've learned before: since the catch-all
is constrained by the HasQHashSingleArgOverload trait, names introduced
before the trait may exclude or include the overload.
This code:
#include <qhashfunctions.h>
enum MyEnum { E1, E2, E3 };
qHash(E1, 42uz);
static_assert(HasQHashSingleArgOverload<MyEnum>); // ERROR
will fail the static_assert. This means that only qHash(int, size_t) is
in the overload set.
However, this code:
struct Evil { Evil(int); };
size_t qHash(Evil);
#include <qhashfunctions.h>
enum MyEnum { E1, E2, E3 };
qHash(E1, 42uz);
static_assert(HasQHashSingleArgOverload<MyEnum>); // OK
will pass the static_assert. qHash(Evil) can be called with an object of
type MyEnum after an user-defined conversion sequence
([over.best.ics.general], [over.ics.user]: a standard conversion
sequence, made of a lvalue-to-rvalue conversion + a integral promotion,
followed by a conversion by constructor [class.conv.ctor]).
Therefore, HasQHashSingleArgOverload<MyEnum> is true here; the catch-all
template is added to the overload set; and it's a best match for the
qHash(E1, 42uz) call.
--
Is this a problem? **Yes**, and a huge one: the catch-all template does
not yield the same value as the qHash(int, size_t) overload. This means
that calculating hash values (e.g. QHash, QSet) will have different
results depending on include ordering!
A translation unit TU1 may have
#include <QSet>
#include <Evil>
QSet<MyEnum> calculateSet { /* ... */ }
And another translation unit TU2 may have
#include <Evil>
#include <QSet> // different order
void use() {
QSet<MyEnum> set = calculateSet();
}
And now the two TUs cannot exchange QHash/QSet objects as they would
hash the contents differently.
--
`Evil` actually exists in Qt. The bug report specifies QKeySequence,
which has an implicit constructor from int, but one can concoct infinite
other examples.
--
Congratulations if you've read so far.
=========================
=== PROPOSED SOLUTION ===
=========================
1) Move the HasQHashSingleArgOverload detection after declaring the
overloads for all the fundamental types (which we already do anyways).
This means that HasQHashSingleArgOverload<fundamental_type> will now
be true. It also means that the catch-all becomes available for all
fundamental types, but as discussed before, for all of them we have
better matches anyways.
2) For unscoped enumeration types, this means however an ABI break: the
catch-all template becomes always the best match. Code compiled before
this change would call qHash(int, size_t), and code compiled after this
change would call the catch-all qHash<Enum>(Enum, size_t); as discussed
before, the two don't yield the same results, so mixing old code and new
code will break.
In order to restore the old behavior, add a qHash overload for
enumeration types that forwards the implementation to the integer
overloads (using qToUnderlying¹).
(Here I'm considering the "old", correct behavior the one that one gets
by simply including QHash/QSet, declaring an enumeration and calling
qHash on it. In other words, without having Evil around before including
QHash.)
This avoids an ABI break for most enumeration types, for which one
does not explicitly define a qHash overload. It however *introduces*
an ABI break for enumeration types for which there is a single-argument
qHash(E) overload. This is because
- before this change, the catch-all template was called, and that
in turn called qHash(E) and XOR'ed the result with the seed;
- after this change, the newly introduced qHash overload for
enumerations gets called. It's very likely that it would not give
the same result as before.
I don't have a solution for this, so we'll have to accept the ABI
break.
Note that if one defines a two-arguments overload for an enum type,
then nothing changes there (the overload is still the best match).
3) Make plans to kill the catch-all template, for Qt 7.0 at the latest.
We've asked users to provide a two-args qHash overload for a very long
time, it's time to stop working around that.
4) Make plans to switch from overloading qHash to specializing std::hash
(or equivalent). Specializations don't overload, and we'd get rid of
all these troubles with implicit conversions.
--
¹ To nitpick, qToUnderlying may select a *different* overload than
the one selected by an implicit conversion.
That's because an unscoped enumeration without a fixed underlying type
is allowed to have an underlying type U, and implicitly convert to V,
with U and V being two different types (!).
U is "an integral type that can represent all the enumerator values"
([dcl.enum/7]). V is selected in a specific list in a specific order
([conv.prom]/3). This means that in theory a compiler can take enum E {
E1, E2 }, give it `unsigned long long` as underlying type, and still
allow for a conversion to `int`.
As far as I know, no compiler we use does something as crazy as that,
but if it's a concern, it needs to be fixed.
[ChangeLog][Deprecation Notice] Support for overloads of qHash with only
one argument is going to be removed in Qt 7. Users are encouraged to
upgrade to the two-arguments overload. Please refer to the QHash
documentation for more information.
[ChangeLog][Potentially Binary-Incompatible Changes] If an enumeration
type for which a single-argument qHash overload has been declared is
being used as a key type in QHash, QMultiHash or QSet, then objects of
these types are no longer binary compatible with code compiled against
an earlier version of Qt. It is very unlikely that such qHash overloads
exist, because enumeration types work out of the box as keys Qt
unordered associative containers; users do not need to define qHash
overloads for their custom enumerations. Note that there is no binary
incompatibity if a *two* arguments qHash overload has been declared
instead.
Fixes: QTBUG-108032
Fixes: QTBUG-107033
Pick-to: 6.2 6.4
Change-Id: I2ebffb2820c553e5fdc3a341019433793a58e3ab
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
These functions are marked as deprecated in future Qt releases.
Task-number: QTBUG-104858
Change-Id: I25d2932455d8c9e3e2d722b1c48fc2cfa2d1e679
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
... instead of QT_PREPEND_NAMESPACE(qHash), which is qualified (prepends at least '::'), and therefore disables ADL.
This is not a problem as long as we wrote our qHash() overloads as free functions (incl. non-hidden friends), but it should™ fail for hidden friends, so use the old using-std::swap() trick to bring QT_PREPEND_NAMESPACE(qHash) into scope, proceeding with an unqualified lookup.
Pick-to: 6.2
Change-Id: I00860b2313699849f86bfe3dd9f41db4ce993cd3
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
There were two problems here: first, qHash(std::pair) must be declared
before qHashMulti that might call back to qHash(std::pair) (i.e., a pair
with one element that is also a pair). But moving the declaration above
causes the second problem: the noexcept expression can't refer to qHash
functions that aren't declared yet. So we forward-declare a constexpr
function for that result, but implement it far below.
Fixes: QTBUG-92910
Change-Id: Ia8e48103a54446509e3bfffd16767ed2e29b026c
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Remove the qmake project files for most of Qt.
Leave the qmake project files for examples, because we still test those
in the CI to ensure qmake does not regress.
Also leave the qmake project files for utils and other minor parts that
lack CMake project files.
Task-number: QTBUG-88742
Change-Id: I6cdf059e6204816f617f9624f3ea9822703f73cc
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
Complete search and replace of QtTest and QtTest/QtTest with QTest, as
QtTest includes the whole module. Replace all such instances with
correct header includes. See Jira task for more discussion.
Fixes: QTBUG-88831
Change-Id: I981cfae18a1cabcabcabee376016b086d9d01f44
Pick-to: 6.0
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Modify special case locations to use the new API as well.
Clean up some stale .prev files that are not needed anymore.
Clean up some project files that are not used anymore.
Task-number: QTBUG-86815
Change-Id: I9947da921f98686023c6bb053dfcc101851276b5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Export some private functions from QUtf8 to resolve
undefined symbols in Qt5Compat after moving QStringRef.
Task-number: QTBUG-84437
Change-Id: I9046dcb14ed520d8868a511d79da6e721e26f72b
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Use pro2cmake with '--api-version 2' to force regenerate
projects to use the new prefixed qt_foo APIs.
Change-Id: I055c4837860319e93aaa6b09d646dda4fc2a4069
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
There is no reason for keep using our macro now that we have C++17.
The macro itself is left in for the moment being, as well as its
detection logic, because it's needed for C code (not everything
supports C11 yet). A few more cleanups will arrive in the next few
patches.
Note that this is a mere search/replace; some places were using
double braces to work around the presence of commas in a macro, no
attempt has been done to fix those.
tst_qglobal had just some minor changes to keep testing the macro.
Change-Id: I1c1c397d9f3e63db3338842bf350c9069ea57639
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
The old implementation was either using CRC32 on modern processors
or a trivial, but rather slow implementation.
We can't continue with CRC32, as that implementation can only
give us 32bit hashes, where we now need to support 64bit in Qt 6.
Change the implementation to use MurmurHash, as public domain
implementation that is both very fast and leads to well distributed hashes.
This hash function is about as fast as the SSE optimized CRC32 implementation
but works everywhere and gives us 64 bit hash values.
Here are some numbers (time for 10M hashes):
14 char 16 char
QByteArray QString float
old qHash (non CRC32) 127ms 134ms 48ms
old qHash (using SSE CRC32 instructions 60ms 62ms 46ms
new qHash 52ms 43ms 46ms
Unfortunately MurmurHash is not safe against hash table DoS attacks, as
potential hash collisions are indepenent of the seed. This will get
addressed in followup commit, where we use SipHash or an SSE optimized
AES based hashing algorithm that does not have those issues.
Change-Id: I4fbc0ac299215b6db78c7a0a2a1d7689b0ea848b
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This is required, so that QHash and QSet can hold more
than 2^32 items on 64 bit platforms.
The actual hashing functions for strings are still 32bit, this will
be changed in a follow-up commit.
Change-Id: I4372125252486075ff3a0b45ecfa818359fe103b
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
We have a problem. Our types don't play well with the std unordered
containers, because they do not specialize std::hash. We therefore
force our users to come up with an implementation, hindering
interoperability, since any two developers are unlikely to come up
with compatible implementations. So combining libraries written by
different developers will result in ODR violations.
Now that we depend on C++11, and thus the presence of std::hash, we
still face the problem that the standard does not provide us with a
means to compose new hash functions out of old ones. In particular, we
cannot, yet, depend on C++17's std::hash<std::string_view> to
implement std::hash<QByteArray>, say. There's also no std::hash for
std::tuple, which would allow easy composition by using std::tie().
So piggy-back on the work we have done over the years on qHash()
functions, and implement the std::hash specializations for Qt types
using the existing qHash() functions, with a twist: The standard
allows implementations to provide means against predictable hash
values. Qt has this, too, but the seed is managed by the container and
passed to the qHash() function as a separate argument. The standard
does not have this explicit seed, so any protection must be implicit
in the normal use of std::hash.
To reap whatever protection that std library has on offer, if any, we
calculate a seed value by hashing int(0). This will be subject to
constant folding if there's no actual seed, but will produce a value
dependent on the seed if there is one.
Add some tests.
A question that remains is how to document the specialization. Can we
have a \stdhashable QDoc macro that does everything for us?
Task-number: QTBUG-33428
Change-Id: Idfe775f1661f8489587353c4b148d76611ac76f3
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
That's the only value for which we will guarantee a stable result across
Qt versions and across invocations of the same application on different
architectures is zero. For any other value, we reserve the right to
change the algorithm. We'll now print a warning when we detect that.
Task-number: QTBUG-47566
Change-Id: I27b55fdf514247549455fffd14b1135e10d24ab4
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: David Faure <david.faure@kdab.com>
Remove most type traits from qtypetraits.h, but keep the custom
implementation of is_signed/is_unsigned. This gets rid of
BSD-3 licensed code from Google in a public header (hugh!).
The custom implementations for is_signed/is_unsigned are kept
because the implementations in gcc's standard headers do not
work as we expect for enums - both is_signed and is_unsigned
always returns false there - see also
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=59027
[ChangeLog][QtCore][General] Qt now relies on type traits from
the C++ standard library.
Change-Id: I3f2188b46949f04ca4482a6ac9afd3482103f0e1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Conflicts:
qmake/library/qmakeevaluator.cpp
One side changed the iterator to use ranged-for, the other changed its
body; they only conflicted because the latter had to add braces around
the body, intruding on the for-line. Trivial resolution.
Change-Id: Ib487bc3bd6e3c5225db15f94b9a8f6caaa33456b
In Qt, null QStrings compare equal to empty ones, so add an explicit
check that the corresponding hash values are identical, too.
Ditto for QByteArray.
Change-Id: I190fc95a765305928d9b6b0e4955433865b6b247
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some
exceptions, see
http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/
Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one
(in those files which will be under GPL 3 with exceptions)
Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
We already include <utility> in <qglobal.h>, so we might
as well provide a qHash() overload for std::pair.
[ChangeLog][QtCore] Added qHash(std::pair), defined in
<QHashFunctions>.
Change-Id: I0f61c513e82e05ce9d2e56bcf18f3be9e2da4da9
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
In some cases it's not possible to use QT_HASH_SEED, specially when
we need to set the environment variable from inside the application,
as dynamically loaded libraries or plugins may create static QHash
instances. That would set qt_qhash_seed to a value different from
-1 and skip the env var value.
For those cases, and when we still want to set qt_qhash_seed, we
provide a way to enforce its value.
Auto-tests accessing qt_qhash_seed directly have been updated
accordingly. Usage in qdoc, uic and rcc has been left as is
for the time being.
Change-Id: I3b35b4fa0223c83b1348a6508641905a2a63266f
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
The keyword no longer has a meaning for the new CI.
Change-Id: Ibcea4c7a82fb7f982cf4569fdff19f82066543d1
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
Qt copyrights are now in The Qt Company, so we could update the source
code headers accordingly. In the same go we should also fix the links to
point to qt.io.
Outdated header.LGPL removed (use header.LGPL21 instead)
Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing
combination. New header.LGPL-COMM taken in the use file which were
using old header.LGPL3 (src/plugins/platforms/android/extract.cpp)
Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license
combination
Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe
Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
qHashRange() takes an (input iterator) range and hashes each element, combining
the hash values using the hash combiner from Boost/N1837 with the magic number
0x9e3779b9, as described here:
http://stackoverflow.com/questions/4948780/magic-number-in-boosthash-combine
qHashRangeCommutative() does the same but with a cummutative combiner (unsigned
addition) to create hash values that are order-independent, e.g. for hashed
containers. The obvious combiner, XOR, is a bad one because it eliminates
duplicate elements. Signed addition cannot be used, since signed overflow
leads to undefined behavior.
[ChangeLog][QtCore] Added qHashRange() and qHashRangeCommutative() functions to aid
implementing qHash() overloads for custom types.
Change-Id: I3c2bbc9ce4bd0455262a70e0cf248486525e534f
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This is in preparation of adding more qHash()-related tests.
Change-Id: Iae65bf8b123e1d6ac6d1eb34d74ba4eb9df8173c
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
