This patch refactors the wildcardToRegularExpression method to generate
a simpler regular expression. It also fixes some shortcomings of the
previous implementation.
Tests have been updated to ensure all cases are properly supported.
Change-Id: I454e3fe8fe0bb663b2f319d6fa2fa8aec626c50d
Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
The API originally proposed was flawed in the sense that the setter
function would use a modified version of the parameter given which would
have make it a black box for the user. This patch fixes that by removing
that setter and providing a static method that will return the pattern
suitably modified to be used by QRegularExpression the same way the
escape method does.
[ChangeLog][Core][QRegularExpression] Implemented support for wildcard
patterns through a static method.
Change-Id: I0054bcaffd7525dac569f54fa81f73b7e4544b2e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QTest::addRow expects a format specifier; this usage makes GCC complain
with -Wformat-security. Use the "old" newRow call instead.
Change-Id: Ieed8e4f64ff5e3d0d4e87325629d20ed6839ff9d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
After the move to PCRE2, optimizing patterns has been a thorn in the
side due to the fact that PCRE2's JIT compiler modifies the pattern
object itself (instead of returning a new set of data, like PCRE1
did). To make this fit with the existing behavior, a read/write
lock was introduced, with the read part locking when matching and
the write when compiling (or JIT-compiling) the pattern.
This locking strategy however introduced a performance issue,
as we needed:
* to acquire a write lock to compile/optimize the pattern (incl. the
common case where the pattern was already compiled, so bailing out
immediately);
* to acquire a read lock during the actual match, to prevent
some other thread from optimizing the pattern under our nose.
This was due to the "lazy" optimization policy of QRegularExpression
-- optimize a pattern after a certain number of usages. The
excessive amount of locking effectively limited scalability.
Simplify the code, and drop that policy altogether: since JIT
compiling in PCRE2 is faster and pretty much "always recommended",
just always do it for any pattern (unless it gets disabled via
env variables) when compiling it.
This allows to go back to a plain QMutex, and now the actual
matching doesn't require acquiring any locks any longer. Of course,
there is still a mutex acquired just before matching for checking
whether the pattern needs recompiling in the first place; this can
probably be further optimized via double-checked locking (using
atomics), but not doing it right now.
This shift makes a couple of pattern options controlling
optimization useless, and allows to centralize the 3
QRegularExpression tests (which were actually the very same test,
just setting slightly different optimizations strategies).
While at it, install a stress-test for threading, with the idea
of running it under TSAN or helgrind to catch bugs in
QRegularExpression's locking.
[ChangeLog][Important Behavior Changes][QRegularExpression] Regular
expressions are now automatically optimized (including JIT
compiling) on their first usage. The pattern options
OptimizeOnFirstUsageOption and DontAutomaticallyOptimizeOption no
longer have any effect, and will get removed in a future version of
Qt. QRegularExpression::optimize() can be still used to compile and
optimize the regular expression in advance (before any match), if
needed.
Task-number: QTBUG-66781
Change-Id: Ia0e97208ae78255fe811b78029ed01c204e47bd2
Reviewed-by: David Faure <david.faure@kdab.com>
This method will make QRegularExpression on par with QRegExp and
will allow to replace this class when a wildcard expression can be
set through an API (e.g. QSortFilterProxyModel::setFilterWildcard).
For other use cases, see QTBUG-34052.
[ChangeLog][QRegularExpression] Implemented support for wildcard
patterns.
Warning: QRegularExpression might not give the exact same result
as QRegExp as its implementation follows strictly the glob patterns
definition for the wildcard expressions.
Change-Id: I5ed4617ca679159430c3d46da3449f6b3100e366
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
Use the new qtConfig macro in all pro/pri files.
This required adding some feature entries, and adding
{private,public}Feature to every referenced already existing entry.
Change-Id: I164214dad1154df6ad84e86d99ed14994ef97cf4
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@theqtcompany.com>
- port Q_FOREACH to C++11 range-for
(incl. one case of iterating over QHash::keys())
- port uses of inefficient QLists to QVector
- add Q_DECLARE_TYPEINFO for types held in Qt containers
Fixes errors pointed out by my tree's static checks.
Change-Id: I30eb432528fa3008240b5c217d5f2f9ddc3679be
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some
exceptions, see
http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/
Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one
(in those files which will be under GPL 3 with exceptions)
Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
Use QByteArray/QString addition instead in loops and for
test row names.
Change-Id: Ieffb429efdc14aa5932b3fcdef5a18e13a62d35f
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@theqtcompany.com>
The keyword no longer has a meaning for the new CI.
Change-Id: Ibcea4c7a82fb7f982cf4569fdff19f82066543d1
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
Playing with the offset argument of pcre_exec is not equivalent to
adjusting the pointer to the subject string. In particular, PCRE
can go behind the offset to check for lookbehinds or "transition"
metacharacters (\b, \B, etc.).
This made the code that deals with QStringRefs not matching in behavior
with the corresponding code dealing with QStrings. For instance,
QString subject("Miss");
QRegularExpression re("(?<=M)iss");
re.match(subject.mid(1)); // doesn't match
re.match(subject.midRef(1)); // matches!!!
Instead, actually adjust the pointer to the subject string so that
the behavior is identical. A broken test that relied on the
equivalence is also removed.
Change-Id: If96333241ef59621d7f5a6a170ebd0a186844874
Reviewed-by: Volker Krause <volker.krause@kdab.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QReg*Exp*s can be compared for equality,
so qHash should be overloaded, too.
There was a (poor) private implementation of qHash(QRegExpEngineKey)
already, which has now been replaced with a better one (the old one
didn't take into account all the fields that make up equality,
producing unnecessary collisions).
[ChangeLog][QtCore][QRegExp] Added qHash(QRegExp).
[ChangeLog][QtCore][QRegularExpression] Added qHash(QRegularExpression).
Change-Id: I1d22fbcc0508018a3f94b4c24571b13ba6e07df2
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
Qt copyrights are now in The Qt Company, so we could update the source
code headers accordingly. In the same go we should also fix the links to
point to qt.io.
Outdated header.LGPL removed (use header.LGPL21 instead)
Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing
combination. New header.LGPL-COMM taken in the use file which were
using old header.LGPL3 (src/plugins/platforms/android/extract.cpp)
Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license
combination
Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe
Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
So far we've only tested matching against QStringRefs which were
spanning the entire underlying QString. Instead, use the offset
matching to also perform tests using "proper" substrings.
Change-Id: Ia66e23f738b52997311ed2cf869e595079984292
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
Instead of simply doing a match against a string S at offset O > 0
(and checking the result), test that the results never change when matching
against the substring starting at (O - i) at offset i, 0 <= i <= O.
Change-Id: Ib7fd648ca0a36b93d145267ebfb792c46ca4e4f5
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
[ChangeLog][QtCore][QRegularExpression] Support for matching
using QStringRef as the subject's string type has been added.
Change-Id: Idb956bbbdf4213f9ebe035db32cd37cf3370c6bc
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
Unify the code paths which test the normal match, the partial
match and the global matches. Not only does this remove the duplication,
but it will also enable the corresponding tests to be run over QStringRefs.
Change-Id: Ia2cbe7a540bbe864b0975c9353697bc06c99c397
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
The fact that we kick in a pattern study and possibly a JIT compilation
after an undocumented number of usages is suboptimal, for a number
or reasons: users may want to JIT compile a pattern immediately,
and at the same time they may not want a random delay in the program
(due to the pattern getting optimized at a random usage).
So: add an optimize() call to force an immediate pattern optimization,
and a pattern option to force an optimization on the first usage.
Change-Id: I95efdecfd31f11ca7cceb9c05037df613601a11c
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(?J) inside a pattern string can be used to allow or disallow duplicated
capturing group names in the pattern string itself.
Although PCRE supports duplicated names, in Qt we don't yet.
Change-Id: I21cd0c41273cd7ef42870ced3a0fad6ba7035cbc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
It may be useful to know which named capturing groups are defined
in an regular expression, and for each of them, what's the
corresponding index. This commit adds the needed method
to QRegularExpression.
Note that extracting the information doesn't happen while holding
the mutex in the private -- pcre_fullinfo just reads information
from the compiled pattern, so that's thread-safe.
Task-number: QTBUG-29079
Change-Id: I50c00ee860f06427c2e6ea10417d5c0733cc8303
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This allows to put them in containers, and to enable subsequent
features for QString.
Change-Id: I3b3fe695ffe6930331ed9f670738376722e0fc36
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This match type doesn't do any match at all; it's only necessary to
properly introduce default constructors for QRegularExpressionMatch
and QRegularExpressionMatchIterator (since they return the match type
that created them).
Change-Id: Ibfe92459c7fdd23129cf3afe073cd443c461ddeb
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Check that the regular expression, the match type and the match options
returned by QRegularExpression/QRegularExpressionIterator are the
same ones passed to match / globalMatch.
Change-Id: Ibd282379bb60b53d614b1d56a7e08b3d6c236da6
Reviewed-by: Jason McDonald <macadder1@gmail.com>
These types are either built-in or 'automatically declared' and so
don't need to be explicitly declared as metatypes.
Change-Id: I54523eb854619917123d8816d3cd6c3a1f5b4c55
Reviewed-by: David Faure <david.faure@kdab.com>
Change copyrights and license headers from Nokia to Digia
Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
Qt 5.0 beta requires changing the default to the 5.0 API, disabling
the deprecated code. However, tests should test (and often do) the
compatibility API too, so turn it back on.
Task-number: QTBUG-25053
Change-Id: I8129c3ef3cb58541c95a32d083850d9e7f768927
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
Although passing a null pointer to pcre16_get_stringnumber for
the compiled pattern should simply make it error out, it's actually
an undocumented behaviour, so let's stay safe and add an explicit
check.
Tests for this codepath are added.
Change-Id: Ifd9c87874f6812ba487104ec1a5bbc83c3b16761
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
PCRE's JIT uses by default 32K on the pcre_exec caller's stack. This
is fine for most situations, but in some cases (esp. patterns with
lot of recursion) more memory is required.
Therefore, if a match execution fails due to exhausting JIT memory,
we let PCRE allocate up to 512KB to be used for the JIT's stack.
The pointer to the allocated memory is put in thread local storage
(so it can be reused from the same thread, if needed, and automatically
goes away when the thread dies).
Change-Id: Ica5fb7d517068befff88ebb198a603a26ec5d8a7
Reviewed-by: Bradley T. Hughes <bradley.hughes@nokia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Exporting the counter that controls the optimization of a compiled
pattern lets us to forcibly optimize all patterns. Therefore,
two tests are now run: one with default optimization values
and another one which always optimizes the pattern.
The counter itself was renamed with a qt_ prefix and put
inside the Qt compilation namespace
(thanks to rohanpm for pointing it out).
Change-Id: I56602433d37adc127772b2d0d2cdaf2e49d43c71
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Some minor fixes: obviously, a valid match always come from a
valid regular expression, but a valid regular expression can create
an invalid match (internal error during matching).
Also, testing an invalid iterator should silence the emitted
warnings.
Change-Id: I585bb99a81e22f108601fd66bf30b56e0229d68b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QRegularExpression::captureCount() returns the number of
capturing groups inside the regular expression pattern.
Change-Id: Ib90ce67c67d06ab2966f0c98bd91da21defc156d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Trivial change: compare dpointers first, then the data.
Added test function for operator==.
Change-Id: I33ac64a59db4ccad56c30be17622187e42415f38
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Added QRegularExpression, QRegularExpressionMatch and
QRegularExpressionMatchIterator as PCRE-enabled, regexp classes.
Documentation is included, as well as a first round of autotests.
Task-number: QTBUG-23489
Change-Id: Id47031b80602c913ccd2fd740070e3024ea06abc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
