Commit Graph

951 Commits

Author SHA1 Message Date
Alexandru Croitor
4ed483b0e2 CMake: pro2cmake: Fix errors reported by flake8 and mypy
Change-Id: I842c0f8d7ec1d173fa1dbd74e769e2c370e70518
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-07-31 12:55:33 +02:00
Alexandru Croitor
ca103f357c CMake: pro2cmake: Reformat files using black
Change-Id: Ic06840f9877d6b863f48a5eb97fcd6735ab9bde8
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-07-31 12:55:33 +02:00
Alexandru Croitor
d65283ebd0 CMake: Teach pro2cmake to convert load(qt_app)
Changes pro2cmake to handle load(qt_app) projects and write out
qt_internal_add_app calls.

Also adds handling of macOS and Windows specific resource files for Qt
apps only.

Task-number: QTBUG-85757
Change-Id: I994d8d19ab2ae366a985cab7894b97d6a278a56f
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-07-31 12:55:32 +02:00
Eirik Aavitsland
db0dc4f1db Remove reference to removed module
The openglextensions module is dicontinued and removed in Qt 6.

Change-Id: I0b98f9b6575026828c188786aeb5c1a62de1f8a3
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-07-23 10:18:13 +02:00
Edward Welbourne
d853816307 Fix handling of Suzhou numbering system
This only arises when the system locale tells us to use its zero as
our zero digit, since no CLDR locale uses it by default. Adapt an
MS-specific QLocale::system() test to use Suzhou numbering, so as to
test this.

While updating the locale-restoration code to also restore the digits
being set in that test, add restore code for the long time format,
where previously only the short time format was restored. Add a
comment to make it less likely one of those shall be missed in future.

Fixes: QTBUG-85409
Change-Id: I343324bb563ee0e455dfe77d4825bf8c3082ca30
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-07-17 12:19:01 +02:00
Edward Welbourne
bb6a73260e Support digit-grouping correctly
Read three more values from CLDR and add a byte to the bit-fields at
the end of QLocaleData, indicating the three group sizes. This adds
three new parameters to various low-level formatting functions. At the
same time, rename ThousandsGroup to GroupDigits, more faithfully
expressing what this (internal) option means.

This replaces commit 27d1391280 with a
fuller implementation that handles digit-grouping in any of the ways
that CLDR supports. The formerly "Indian" formatting now also applies
to at least some locales for Bangladesh, Bhutan and Sri Lanka.

Fixed Costa Rica currency formatting test that wrongly put a separator
after the first digit; the locale (in common with several Spanish
locales) requires at least two digits before the first separator.

[ChangeLog][QtCore][Important Behavior Changes] Some locales require
more than one digit before the first grouping separator; others use
group sizes other than three. The latter was partially supported (only
for India) at 5.15 but is now systematically supported; the former is
now also supported.

Task-number: QTBUG-24301
Fixes: QTBUG-81050
Change-Id: I4ea4e331f3254d1f34801cddf51f3c65d3815573
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-07-14 14:52:08 +02:00
Joerg Bornemann
ac14858e85 configurejson2cmake: Use a context manager for file special handling
One can now write
    with special_cased_file("base/dir", "foo.txt") as fh:
        do_something(fh)

This makes the code of processJson a bit clearer, and it allows us to
easily add more files that support the special handling comments.

Change-Id: Ia25d0c0d48df1802c5e2123d05345a88b42a2981
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-07-13 10:51:43 +02:00
Joerg Bornemann
8a0676d5f9 configurejson2cmake: Generalize special case support
The SpecialCaseHandler supported only two file names: "CMakeLists.txt"
and "configure.cmake". Generalize the code to allow for arbitrary file
names. We will use this in a subsequent commit.

Change-Id: I0adada91409a11a369fd1cf2d6ab21cc8f28ba0f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-07-13 10:51:15 +02:00
Dimitrios Apostolou
1e546595e9 Remove unused imports
As found by LGTM.com.

Change-Id: I1704f10f9bab1b11ab22824aca0cfcdcb47fef2f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-07-10 02:36:54 +02:00
Dimitrios Apostolou
3736ebf7c8 Remove duplicate dict entry
As found by LGTM.com.

Change-Id: I9d41fec8a9f80fe9cb1cae960418032f84646a99
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-07-10 02:36:40 +02:00
Dimitrios Apostolou
efdfdfa0fc Remove dead code
Found by LGTM.com.

Change-Id: I661e2287a6964b63d5b03c8913bc4b09f21ffdf1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-07-10 02:36:25 +02:00
Alexandru Croitor
b1f8ca8032 CMake: Fix various issues with the Android build
Certain resources and compiled jar files have to be copied into the
non-prefix build dir location. They were previously only installed,
which doesn't do anything in a non-prefix build.

Change pro2cmake to generate code that places the compiled java
jars into QT_BUILD_DIR, so that non-prefix builds work.

Place the module dependencies xml files into lib folder in non-prefix
builds.

Don't special case the output and install location of the Android QPA
plugin.

Task-number: QTBUG-85399
Change-Id: I4ac9d3929ea8ecc95ec99a77e621ad2121b68832
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-09 11:56:31 +02:00
Jarek Kobus
a7f9d5a7fa Use QList instead of QVector in util
Task-number: QTBUG-84469
Change-Id: I077fb5c32456d438a457c1f73852313ea2ea9ae5
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-07 20:34:48 +02:00
Alexandru Croitor
7ad5c5346f CMake: pro2cmake: Remove dead code regarding resource retaining
qtdeclarative as of 41864db3b61d9e81a9fe4906918d2cd3d6d32a0c always
retains the resources, so there's no manual facility for retaining
left. Remove the code.

Change-Id: If79b66683897b5f430844e13c0230e41a2cf061d
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-07 11:48:46 +02:00
Alexandru Croitor
19b04f6928 CMake: pro2cmake: Fix handling of QT_SKIP_QUICKCOMPILER
If a file was part of a resource that has a non-empty base directory,
the script generated set_source_files_properties calls to the file
path without including the base dir, which means the command did
nothing.

The script should set the QT_SKIP_QUICKCOMPILER on the full file path
if a base dir is present, similar to how the alias handled.

Refactor the alias property writing code to write all properties
into the command that uses the full correct path.

Change-Id: Ic75d51ecf60eef3ada9bd6ca26882de3447896e9
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-07 11:48:45 +02:00
Lars Knoll
a46caf087c Simplify Q_ARRAY_LITERAL
And clean up some unused pieces of code.

Change-Id: I285b6862dc67b7130af66d3e08f652b1a56b990e
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-07-06 21:30:06 +02:00
Tor Arne Vestbø
8dd6957cd7 Make QtPlatformHeaders a private module
The APIs will be reintroduced as part of the new platform interface
API where appropriate.

Clients that still depend on the platform headers can include it
via QT += platformheaders-private.

Change-Id: Ifbd836d833d19f3cf48cd4f933d7fe754c06d2d9
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-07-06 11:52:14 +02:00
Joerg Bornemann
943340222e CMake: Generate qdevice.pri when cross-building
This adds the CMake equivalent to the classical -device-option key=value
configure argument:
-DQT_QMAKE_DEVICE_OPTIONS=key1=value1[;keyN=valueN]
The keys and values get dumped verbatim into qdevice.pri.

This patch also ports the machineTuple configure test. Its result is
written into qdevice.pri as value for the GCC_MACHINE_DUMP variable.

Change-Id: I29f2323fd87639fafaed99ec7446c7ee75504705
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-06-29 12:47:59 +02:00
Alexandru Croitor
7fcc9cf055 CMake: Create a Find module wrapper for OpenGL
Our .prl files embedded an absolute path to the OpenGL.tbd file. This
obviously breaks their usage when used on another machine when no SDK
exists.

To fix that we need to use a "-framework OpenGL" linker flag instead
of linking against the absolute path library.
To convince CMake to do that, we have to create a wrapping
OpenGL target which sets an appropriate INTERFACE_LINK_LIBRARIES
property.

So create a FindWrapOpenGL find module to do that on darwin platforms.
Adjust helper.py and our build system to use it.

This tangentially amends 38cd18384f
because it recreates the FindWrapOpenGL module, but for a different
purpose.

Task-number: QTBUG-85240
Task-number: QTBUG-84781
Change-Id: I3498c19157ae31db5099e6edfb9d71490187f1d3
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-06-29 12:22:49 +02:00
Joerg Bornemann
f6b7b64ed0 CMake: Adjust conversion scripts to iconv changes in qt5compat
Change-Id: I8e06a09768bd3554500d666273060d2e130965b8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-06-28 17:17:02 +02:00
Ulf Hermann
fd52a6fcb9 pro2cmake.py: Parse qmldir import statements with versions
You can write the following in a qmldir file these days:

import QtQuick auto
import QtQml 2.4

pro2cmake should understand and convert these to cmake.

Change-Id: Ica1728de0c8d7a0b2b5aa341d20e60b23dfa7fe1
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2020-06-24 15:49:35 +00:00
Karsten Heimrich
18ec53156e Move QTextCodec support out of QtCore
* Assume UTF-8 on all Unix like systems
* Export some functions to be able to compile QTextCodec once
  moved to Qt5Compat.

Task-number: QTBUG-75665
Change-Id: I52ec47a848bc0ba72e9c7689668b1bcc5d736c29
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-06-20 02:04:38 +02:00
Giuseppe D'Angelo
3e1d03b1ea Port Q_STATIC_ASSERT(_X) to static_assert
There is no reason for keep using our macro now that we have C++17.
The macro itself is left in for the moment being, as well as its
detection logic, because it's needed for C code (not everything
supports C11 yet).  A few more cleanups will arrive in the next few
patches.

Note that this is a mere search/replace; some places were using
double braces to work around the presence of commas in a macro, no
attempt has been done to fix those.

tst_qglobal had just some minor changes to keep testing the macro.

Change-Id: I1c1c397d9f3e63db3338842bf350c9069ea57639
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-06-19 19:38:23 +02:00
Alexandru Croitor
7e03bc39b8 CMake: Improve sanitizer detection
The change fixes the labels to show up correctly in the configure
summary.
It also allows enabling the sanitizer via the feature flags, e.g.
-DFEATURE_sanitize_address=ON.
Finally the qtbase sanitizer option is saved in QtBuildInternalsExtra
so that repos built after qtbase have the same sanitizer options
enabled.

Change-Id: Ic9d9e3ce3c7ebbc244ced2e6d163d1ac8ee06b12
Fixes: QTBUG-84721
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-06-15 18:41:35 +02:00
Alexandru Croitor
a08bf7f00a CMake: Fix usage of gc_binaries feature
The qt_internal_apply_gc_binaries function should apply both compile
and link flags, not just link flags.

The flags should be applied publically to all consumers of Bootstrap
regardless if the gc_binaries feature is enabled.

The flags should be applied publically to Core only in case if the
feature is enabled (aka for static builds only).

Change-Id: Id42af0d9b527004d74c04eff2c9e3c2be1e76aac
Fixes: QTBUG-84461
Task-number: QTBUG-83929
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-06-12 22:34:12 +02:00
Eirik Aavitsland
56f40cdca6 Remove the openglextensions module
[ChangeLog][General] The QtOpenglExtensions module has been
discontinued. It provided only convenience functionality for certain
direct OpenGL usage. With the new RHI graphics API abstraction, that
is no longer a primary use case. Applications that still need to
access that API have a number of alternative options, including
QOpenGLExtraFunctions.

Fixes: QTBUG-84085
Change-Id: I272af61c69ebcec207b576d67d08b59623d485ec
Reviewed-by: Laszlo Agocs <laszlo.agocs@qt.io>
2020-06-10 09:35:04 +02:00
Ulf Hermann
910804a420 pro2cmake: Parse "optional plugin" from qmldir files
Task-number: QTBUG-84639
Change-Id: Iedfa2e53c686a7c7c855efc7b9deee097a15f4b7
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-06-09 14:10:10 +02:00
Tor Arne Vestbø
93a04332eb Move QtAccessibilitySupport into QtGui
Task-number: QTBUG-83255
Change-Id: Ibc1b38e77c3c90030a832c41f4de65c6c38bc91d
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-06-02 23:32:36 +02:00
Jannis Voelker
33fc2ff663 Adds library mappings for the Qt OPC UA module in helper.py
Change-Id: I64bee2a7c149b8e4b14b71103ebc748c32993145
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-06-02 23:12:56 +02:00
Leander Beernaert
5a779a4ad3 CMake: Make it possible to build tools when cross-compiling
This patch allows tools to be built for the target platform when the
QT_BUILD_TOOLS_WHEN_CROSSCOMPILING parameter is set at configuration
time.

To avoid naming conflicts, the target tools are suffixed with "_native".
The qt_get_tool_target_name() function can be used to get the tool name
for both scenarios (cross and non-cross compilation).

Extend pro2cmake to refer to the right target name for tools.
The relevant write_XXX functions have a new target_ref parameter that
will be "${target_name}" for tools and literally the target name for
everything else.

Fixes: QTBUG-81901
Change-Id: If4efbc1fae07a4a3a044dd09c9c06be6d517825e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-06-02 22:42:15 +02:00
Laszlo Agocs
e3361c1614 CMake: Update src\gui\configure.json and the conversion script
The ANGLE removal's manual configure.cmake changes are now replaced
by updating the conversion script's special cases instead.

Change-Id: I06317fe0202b25bd9f9524da14e1bb0ccbb30cc1
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-05-29 23:23:02 +02:00
Alexandru Croitor
c4e09cf267 CMake: pro2cmake: Check existence of QMLTYPES_INSTALL_DIR
When writing the qt6_qml_type_registration call, make sure to write
the set(QT_QML_MODULE_INSTALL_QMLTYPES TRUE) statement only if an
explicit path was provided via
QMLTYPES_INSTALL_DIR / QT_QML_MODULE_INSTALL_DIR.

This will remove the QT_QML_MODULE_INSTALL_QMLTYPES property from
many examples and tests in qtdeclarative, thus not installing the
.qmltypes file into the Qt prefix, thus polluting it.

Task-number: QTBUG-84403
Change-Id: I5c64d42915896d14f2591cbb64d73d912dbf7c7d
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-05-29 17:17:18 +02:00
Volker Hilsheimer
37eb5d38aa Add SvgWidgets library
Change-Id: I243aa35a0628f80219b589a0ba943ec1a25bb972
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-29 16:58:43 +02:00
Tor Arne Vestbø
7172b5112e Move QtPlatformCompositorSupport into QtOpenGL
Task-number: QTBUG-83255
Change-Id: Id9ea654db8efb00b487d53aea03d7f23a7ab1a54
Reviewed-by: Laszlo Agocs <laszlo.agocs@qt.io>
2020-05-28 21:39:48 +02:00
Dimitrios Apostolou
52f218ee83 Change const arrays to constexpr
so that they can be used with Q_STATIC_ASSERT().
Also constify more another array.

Change-Id: I1e7208127e06abb33af9bdc46712657af5dc98fe
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-05-28 10:48:58 +02:00
Dimitrios Apostolou
fa81b3ebe2 Update paths pointing to old locations
Change-Id: I2fb14834b34827ed4e84e9f7dc0791aa810a32ce
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-05-28 10:48:58 +02:00
Dimitrios Apostolou
5b3c3ff97e Improve readability of the autogenerated arrays in qurltlds_p.h
The next time qurltlds_p.h is generated, array sizes shall be explicit.
Also added some helpful comments. And updated copyright date.

Change-Id: I2bc9a78bcc2982845fce75a413e2048b0bfd12a0
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-05-28 10:48:57 +02:00
Friedemann Kleint
9c31acbf47 Move windowsuiautomation API support into QtGui
Task-number: QTBUG-83255
Change-Id: Ibface71931b6384494842ba9744d76f738c5ca85
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-05-27 22:28:41 +02:00
Tor Arne Vestbø
7a1650e343 Move QtVulkanSupport into QtGui and QtOpenGL
Task-number: QTBUG-83255
Change-Id: Ib021cecebda89fa2ab9332752124a8cec0a51a10
Reviewed-by: Laszlo Agocs <laszlo.agocs@qt.io>
2020-05-22 15:38:28 +02:00
Tor Arne Vestbø
f76161d61f Move QRasterBackingStore to QtGui
Task-number: QTBUG-83255
Change-Id: I339173de6e109c5a9b9572972ba894c15053c034
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-05-21 20:28:37 +02:00
Tor Arne Vestbø
42024666a3 Move QMacInternalPasteboardMime to QtGui
Task-number: QTBUG-83255
Change-Id: I00fda24479ad2c04781c5fefaa15fac1118033a8
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-05-21 13:21:53 +02:00
Joerg Bornemann
759da6742a CMake: Generate information about 3rdparty libs in module .pri files
For modules that are not yet ported to CMake and that use
   QMAKE_USE += libfoo
we need to provide the information about libfoo in the qt_lib_XXX.pri
files.

Also, we now generate qt_ext_XXX.pri files for bundled 3rdparty libs.

Task-number: QTBUG-75666
Change-Id: I9e4b057a197554ecb37c294c0bf09e2a2b3aa053
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-19 13:26:53 +02:00
Joerg Bornemann
d5cb44cd3a pro2cmake: Fix Scope.get_string for list values
QMake variables are always string lists, but some variables are used to
represent single string values. People tend to fill those variables with
multiple values and expect them to be joined by spaces.

Exhibit A: qtdeclarative/tools/qml/qml.pro
    QMAKE_TARGET_DESCRIPTION = QML Runtime

QMake supports this for many values. Do the same for Scope.get_string().

Change-Id: I6af22bc1bfed07a4d8eac94d7315051a5bcb7a4d
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-15 08:54:21 +02:00
Joerg Bornemann
286d79d2ed pro2cmake: Handle QMAKE_TARGET_XXX variables
Change-Id: I62151e04d21c20d8c8ad5825464f26074c8abc3d
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-12 14:31:12 +02:00
Alexandru Croitor
64f62105ce CMake: pro2cmake: Fix cmake api detection for plugins
We should search for the values of function names rather than
the mapping key.

Change-Id: Iedef7fcf4d716ff7f6d277d9626d9dc2dd781401
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-05-12 13:03:11 +02:00
Friedemann Kleint
d2ae13b75b cmake: Add MinGW platform check
Task-number: QTBUG-78167
Change-Id: Id203925d69c5b4d7e0841c0dbdd298269a13f06a
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-12 08:57:04 +02:00
Marc Mutz
19e7c0d2b5 QChar/QString: centralize case folding in qchar.cpp
There are (at least) two implementations of the low-level case-folding
algorithm, one of which (for QChar::toLower()) seems to be wrong (it
doesn't deal with special cases which expand to more than one code
point).

The algoithm hidden in QString and entangled with the QString
detaching code makes reusing the code much harder.

At the same time, the dependency of the algorithm on the unicode
tables makes exposing a non-allocating result type in the public API
hard. std::u16string would be an alternative if we can assure that all
implementations use SSO with at least four characters.

So, for the time being, leave this as internal API for use in an
upcoming QStringView::toLower() as well as case-insensitive hashing.

Change-Id: Iabb2611846f6176776aa20e634f44d8464f3305c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-09 06:25:05 +00:00
Joerg Bornemann
95b441d097 pro2cmake: Add library mappings for the restructured ActiveQt module
Change-Id: I643cb2afc0e4b74abcbd67bb9002ab334337be37
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-07 16:23:15 +02:00
Christophe Giboudeaux
5dcaa11cc2 pro2cmake: Honor INSTALL_EXAMPLESDIR if defined
When building examples standalone, 'INSTALL_EXAMPLESDIR' may be
undefined.

The converted .pro files now check whether the variable is
set to prevent installation into unexpected locations.

Fixes: QTBUG-81797
Change-Id: I9f97568923c8103011a41d9af7cfe02dd1e40b56
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-07 15:41:26 +02:00
Mårten Nordheim
9c85078dc2 Support special casing in configure.cmake
Change-Id: Ib899a5446a81e2604ba3642b9ad37aedc18ca650
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-05-05 15:34:15 +02:00
Mårten Nordheim
6ffe9b1c42 pro2cmake.py: Generate .cmake.conf files for versioning
And create one for QtBase at the same time.

Fixes: QTBUG-83835
Change-Id: Icc6b022165a57bd4e22c23bdb0016522b99a5b80
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-30 17:44:03 +00:00
Tor Arne Vestbø
ebc3350be7 pro2cmake: Translate (QT|QMAKE_USE)_FOR_PRIVATE into PRIVATE_MODULE_INTERFACE
Fixes: QTBUG-83063
Change-Id: I5b02cd236ef0fd3a9ed389294c05f8e3024463cd
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-30 14:20:26 +02:00
Tor Arne Vestbø
4618fa6f8d Fix pro2cmake help text
Change-Id: I84980dda981c50da159e4bb81b08190b882d6978
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-30 14:20:26 +02:00
Fabian Kosmale
14c713e759 pro2cmake: convert QMLTYPES_FILENAME to QT_QMLTYPES_FILENAME
Change-Id: I9afc6d4c0c9b3fd87ef28f15da0a984d778b83ac
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-30 10:13:56 +02:00
Joerg Bornemann
062b50abff CMake: Namespace all our IMPORTED targets
CMake IMPORTED targets should be namespaced so that CMake knows that
the name refers to a target and not a file.

Use the existing WrapXXX naming scheme where applicable.

Fixes: QTBUG-83773
Change-Id: I5b0b722c811200c56c260c69e76940a625228769
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-29 23:08:32 +02:00
Marc Mutz
7b04e0012b QUnicodeTables: port to charNN_t
This makes existing calls passing uint or ushort ambiguous, so
fix all the callers. There do not appear to be callers outside
QtBase. In fact, the ...BreakClass() functions appear to be
utterly unused.

Change-Id: I1c2251920beba48d4909650bc1d501375c6a3ecf
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
2020-04-27 13:08:41 +02:00
Marc Mutz
20cdf807b1 QChar: port low-level functions from uint/ushort to char32/16_t
Now that the standard gives us proper types for UTF-16 and UTF-32
characters, use them. Will eventually make the code much easier to
read than today, where uint could be an index as well as a char32_t.

It also ensures that the result of e.g. QChar::highSurrogate() can
still be implicitly converted to a QChar now that the
QChar(non-characater-integral-types) ctors are being made explicit.

[ChangeLog][QtCore][QChar] All low-level functions
(e.g. highSurrogate()) now take and return char16_t instead of ushort
and char32_t instead of uint.

Change-Id: I9cd8ebf6fb998fe1075dae96c7c4484a057f0b91
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
2020-04-24 12:45:53 +02:00
Joerg Bornemann
4545eadd3e CMake: Port the 'static_runtime' feature
This feature is Windows-only and must be turned on manually.
For MSVC it sets the MSVC_RUNTIME_LIBRARY target property.
For MinGW it adds the -static linker flag.

Change-Id: I9da3b88d545b34bc34a3a80301b2dd1b5986fa88
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-23 10:09:15 +02:00
Leander Beernaert
6fbeef4c6b CMake: Add qt6_add_plugin public API
This patch adds a publicly callable qt6_add_plugin() API to create
plugins. This API is meant to cover cases such as the plugandpaint
example.

This patch also renames qt_add_plugin to qt_internal_add_plugin in order
to avoid clashes with the public API. To avoid breaking the existing
projects, a compatibility wrapper function is enabled by default unless
QT_DISABLE_QT_ADD_PLUGIN_COMPATIBILITY is specified.

Fixes: QTBUG-82961
Change-Id: If5b564a8406c90434f1bdad0b8df76d3e6626b5f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-22 09:51:12 +02:00
Joerg Bornemann
2a0585e7cb CMake: Port the 'pkg-config' feature
Change-Id: If571208aaaba8c1d0af834e8ae11869872d42135
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
2020-04-21 09:01:39 +02:00
Joerg Bornemann
3eed6d76b7 CMake: Port the 'ccache' feature
This maps to the CMake variable QT_USE_CCACHE.

Change-Id: I3258027301284d907f6ecde6c65d2c0dde8f0a11
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-17 23:10:11 +02:00
Joerg Bornemann
8a5ffc4f73 CMake: Port the 'ltcg' feature
This maps to the CMake variable CMAKE_INTERPROCEDURAL_OPTIMIZATION.

Change-Id: Id0ce48f176b95c27e74ab80276e89503b1660f79
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-17 23:10:09 +02:00
Joerg Bornemann
9b45ca7411 CMake: Port the 'cross_compile' feature
Change-Id: Iccd1d55e95797740a4a8689462ce9ab1e49a62c1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-17 20:42:42 +02:00
Joerg Bornemann
6f5c91e985 configurejson2cmake: Remove mention of the system-xcb feature
This feature was removed in commit 60588e1a.

Change-Id: I061410dfab13a2210474014892d1bc828a5b21cf
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-17 20:41:44 +02:00
Alexandru Croitor
67ee92f4d8 CMake: Handle automatic rpath embedding correctly
Instead of using CMAKE_INSTALL_RPATH to embed an absolute path
to prefix/libdir into all targets, use the more sophisticated aproach
that qmake does.

For certain targets (modules, plugins, tools) use relative rpaths.
Otherwise embed absolute paths (examples, regular binaries).
Installed tests currently have no rpaths.

On certain platforms rpaths are not used (Windows, Android,
iOS / uikit).

Frameworks, app bundles and shallow bundles should also be handled
correctly.

Additional rpaths can be provided via QT_EXTRA_RPATHS variable
(similar to the -R option that configure takes).

Automatic embedding can be disabled either via QT_FEATURE_rpath=OFF
or QT_DISABLE_RPATH=ON.

Note that installed examples are not relocatable at the moment (due
to always having an absolute path rpath), so this is a missing feature
compared to qmake. This is due to missing information on where
examples will be installed, so a relative rpath can not be computed.

By default a Qt installation is relocatable, so there is no need to
pass -DQT_EXTRA_RPATHS=. like Coin used to do with qmake e.g. -R .

Relative rpaths will have the appropriate 'relative base' prefixed
to them (e.g $ORIGIN on linux and @loader_path on darwin platforms).
There is currently no support for other platforms that might have a
different 'relative base' than the ones mentioned above.

Any extra rpaths are saved to BuildInternalsExtra which are re-used
when building other repositories.

configurejson2cmake modified to include correct conditions for the
rpath feature.

It's very likely that we will need a new qt_add_internal_app()
function for gui apps that are to be installed to prefix/bin.
For example for Assistant from qttools. Currently such apps
use qt_add_executable().
The distinction is necessary to make sure that relative rpaths are
embedded into apps, but not executables (which tests are part of).

Amends e835a6853b

Task-number: QTBUG-83497
Change-Id: I3510f63c0a59489741116cc8ec3ef6a0a7704f25
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-04-17 15:59:25 +02:00
Eskil Abrahamsen Blomfeldt
ad5aee2e34 Get rid of QT_OPENGL_ES*
The QT_OPENGL_ES* macros are leftovers from an earlier,
ad hoc configuration system, which has since been
replaced by QT_CONFIG. To clean things up in Qt 6,
we use the new way instead.

Task-number: QTBUG-83467
Change-Id: I578dc7695bff9d5ee303b22e44f60fee22fe0c28
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-04-17 11:50:24 +02:00
Leander Beernaert
30f69f7ff6 CMake: Add library mapping for xcb-fixes
Change-Id: Ida893674fa0782446acdac4ee5dfec4849d4605f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-15 16:25:19 +02:00
Leander Beernaert
812cc76b02 CMake: Update pro2cmake and friend to convert QtVirtualkeyboard
Expand $$MODULE_BASE_DIR to ${QT_BUILD_DIR}.

Add library mappings for Hunspell.

Add test mappings for QtVirtualKeyboard regarding 3rdparty Hunspell and
T9Write.

Change-Id: Ief007456d0471cbcf9a03d819291edec5f63680f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-15 14:15:52 +00:00
Leander Beernaert
a77cd07b24 CMake: pro2cmake expand variables for qt_add_resource()
Change-Id: Ida1196645dabcb0c3aee210d9bc5ee5632a9ccc5
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2020-04-14 13:19:23 +02:00
Mikhail Svetkin
ae57e4fba5 cmake: Add library mapping for QtHttpServer
Change-Id: I3e12f164f7f6891c60c0d9da700b888850021e53
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-09 20:11:31 +02:00
Lars Knoll
c6cdf38e75 Change qHash() to work with size_t instead of uint
This is required, so that QHash and QSet can hold more
than 2^32 items on 64 bit platforms.

The actual hashing functions for strings are still 32bit, this will
be changed in a follow-up commit.

Change-Id: I4372125252486075ff3a0b45ecfa818359fe103b
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2020-04-09 20:03:25 +02:00
Alexandru Croitor
066e8a4557 CMake: Handle missing packages in project compile tests
If a find_package() in a try_compile project doesn't find a package,
and we then link against a non-existent target, the configuration
failure of the compile test also fails the configuration of the
project.

To avoid that, separate library targets from non-targets, and make sure
to only link against the targets if they exist.

pro2cmake now outputs modified compile test project code which iterates
over targets and non-target libraries, and links against them when
needed.

Change-Id: Ib0f4b5f07af13929c42d01a661df2cabdf9b926b
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2020-04-09 11:26:42 +02:00
Qt Forward Merge Bot
c937ed8af4 Merge "Merge remote-tracking branch 'origin/5.15' into dev" 2020-04-08 22:04:23 +02:00
Alexandru Croitor
e0346df1b2 CMake: Handle finding of OpenSSL headers correctly
In Coin when provisioning for Android, we download and configure
the OpenSSL package, but don't actually build it. This means that
find_package(OpenSSL) can find the headers, but not the library,
and thus the package is marked as not found.

Previously the openssl_headers feature used the result of finding
the OpenSSL package, which led to it being disabled in the above
described Android case.

Introduce 2 new find scripts FindWrapOpenSSL and
FindWrapOpenSSLHeaders. FindWrapOpenSSLHeaders wraps FindOpenSSL,
and checks if the headers were found, regardless of the OpenSSL_FOUND
value, which can be used for implementing the openssl_headers feature.

FindWrapOpenSSL uses FindWrapOpenSSLHeaders, and simply wraps the
OpenSSL target if available.

The find scripts also have to set CMAKE_FIND_ROOT_PATH for Android.
Otherwise when someone passes in an OPENSSL_ROOT_DIR, its value will
always be prepended to the Android sysroot, causing the package not
to be found.

Adjust the mapping in helper.py to use the targets created by these
find scripts. This also replaces the openssl/nolink target.

Adjust the projects and tests to use the new target names.

Adjust the compile tests for dtls and oscp to use the
WrapOpenSSLHeaders target, so that the features can be enabled even
if the library is dlopen-ed (like on Android).

Task-number: QTBUG-83371
Change-Id: I738600e5aafef47a57e1db070be40116ca8ab995
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2020-04-08 22:03:24 +02:00
Alexandru Croitor
0a13c3a3f0 CMake: pro2cmake: Handle $$PWD in included .pri files correct-ish
Previously the $$PWD was evaluated within the including .pro file,
which generated incorrect relative paths to source files.

Now we use a horrible hack to evaluate keys ending with SOURCES
and HEADERS. If such is a case, use a map_file transformer which
will use the included scope, thus creating correct relative paths.

Fixes projects in qtdeclarative like src/qmltypregistrar and qmllint.

Checked that it doesn't break projects in qtdeclarative and qtbase.

Change-Id: I21f1e4c638c2cf8d0f67e94e1a583ebc54c175a2
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2020-04-08 21:58:41 +02:00
Qt Forward Merge Bot
8823bb8d30 Merge remote-tracking branch 'origin/5.15' into dev
Conflicts:
	examples/opengl/doc/src/cube.qdoc
	src/corelib/global/qlibraryinfo.cpp
	src/corelib/text/qbytearray_p.h
	src/corelib/text/qlocale_data_p.h
	src/corelib/time/qhijricalendar_data_p.h
	src/corelib/time/qjalalicalendar_data_p.h
	src/corelib/time/qromancalendar_data_p.h
	src/network/ssl/qsslcertificate.h
	src/widgets/doc/src/graphicsview.qdoc
	src/widgets/widgets/qcombobox.cpp
	src/widgets/widgets/qcombobox.h
	tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp
	tests/auto/widgets/widgets/qcombobox/tst_qcombobox.cpp
	tests/benchmarks/corelib/io/qdiriterator/qdiriterator.pro
	tests/manual/diaglib/debugproxystyle.cpp
	tests/manual/diaglib/qwidgetdump.cpp
	tests/manual/diaglib/qwindowdump.cpp
	tests/manual/diaglib/textdump.cpp
	util/locale_database/cldr2qlocalexml.py
	util/locale_database/qlocalexml.py
	util/locale_database/qlocalexml2cpp.py

Resolution of util/locale_database/ are based on:
https://codereview.qt-project.org/c/qt/qtbase/+/294250
and src/corelib/{text,time}/*_data_p.h were then regenerated by
running those scripts.

Updated CMakeLists.txt in each of
	tests/auto/corelib/serialization/qcborstreamreader/
	tests/auto/corelib/serialization/qcborvalue/
	tests/auto/gui/kernel/
and generated new ones in each of
	tests/auto/gui/kernel/qaddpostroutine/
	tests/auto/gui/kernel/qhighdpiscaling/
	tests/libfuzzer/corelib/text/qregularexpression/optimize/
	tests/libfuzzer/gui/painting/qcolorspace/fromiccprofile/
	tests/libfuzzer/gui/text/qtextdocument/sethtml/
	tests/libfuzzer/gui/text/qtextdocument/setmarkdown/
	tests/libfuzzer/gui/text/qtextlayout/beginlayout/
by running util/cmake/pro2cmake.py on their changed .pro files.

Changed target name in
	tests/auto/gui/kernel/qaction/qaction.pro
	tests/auto/gui/kernel/qaction/qactiongroup.pro
	tests/auto/gui/kernel/qshortcut/qshortcut.pro
to ensure unique target names for CMake

Changed tst_QComboBox::currentIndex to not test the
currentIndexChanged(QString), as that one does not exist in Qt 6
anymore.

Change-Id: I9a85705484855ae1dc874a81f49d27a50b0dcff7
2020-04-08 20:11:39 +02:00
Alexandru Croitor
0bfbe10ff2 CMake: Implement proper exclusion of tools including installing
The previous approach didn't work for prefix builds. While a target
might be excluded from building via EXCLUDE_FROM_ALL property, when
calling make install it would still try to install the target and
thus fail.

It's not possible to modify an install() command in a post-processing
step, so we switch the semantics around.

pro2cmake will now write a
qt_exclude_tool_directories_from_default_target() call before adding
subdirectories. This will set an internal variable with a list
of the given subdirectories, which is checked by qt_add_executable.

If the current source dir matches one of the given subdirectories,
the EXCLUDE_FROM_ALL property is set both for the target and the
qt_install() command.

This should fix the failing Android prefix builds of qttools.

Amends 622894f96e

Change-Id: Ia19323a2ef72a3fb9cb752ad2d4f2742269d11c4
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-04-07 23:22:33 +02:00
Joerg Bornemann
067ae2df45 CMake: Port the 'static' feature
Now that we have the 'shared' feature implemented we can easily port
the 'static' feature.

Change-Id: Ia9b54b68d532d73c3d62d12a86c9e8b83e7909c8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-06 21:42:53 +02:00
Edward Welbourne
727afdf344 Fix parameter order in cldr2qlocalexml.py's usage()
Callers and definition were out of sync.

Change-Id: Icda26887cb64c61c7e373766f25559b0d450d112
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-06 14:29:32 +02:00
Alexandru Croitor
9b21dbbd79 CMake: Reformat python scripts
Change-Id: I1dfac318cdbbc4b4b7c76b113edca7db8f52f56b
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-06 11:00:53 +02:00
Alexandru Croitor
d64a7c9f5a CMake: pro2cmake: Allow specifiying custom output file
Change-Id: If984d2bbc3e4b655a5eb58c68b282e3d13d51218
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2020-04-06 11:00:53 +02:00
Alexandru Croitor
34c7344e2a CMake: configurejson2cmake: Port precompile_header feature
We don't need the test, we can just check the value of BUILD_WITH_PCH
to know whether the feature is enabled.

Regenerate configure.cmake files.

Change-Id: I5691a22af2913bc398f99825e0c41cf2daf5a587
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-06 11:00:53 +02:00
Sona Kurazyan
6a6482cb8d Add Qt Core5Compat to the library mapping
Change-Id: Iad613c75705b09f7ae043cff417b0fcf3f5dd946
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-03 17:45:43 +00:00
Alexandru Croitor
8a0a31ed68 CMake: pro2cmake: Skip conversion of doc snippets
Most of them are hand-written.

Change-Id: Ia3d83cdc9e279420c9b4700993b428e10cf8fb22
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-03 16:05:26 +02:00
Alexandru Croitor
61d9428d8c CMake: Make sure that the library config.test is used for assimp
Change-Id: Ia5b2a2ffca2dda460a323fd3f564c548be84c0aa
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-03 16:05:22 +02:00
Alexandru Croitor
d2931a2626 CMake: Handle standalone config.tests in configure libraries section
Some library entries in configure.json have a test entry.
An example is assimp in qtquick3d.

qmake tries to find the library via the sources section, and then tries
to compile the test found in config.tests/assimp/assimp.pro while
automagically passing it the include and link flags it found for assimp.

We didn't handle that in CMake, and now we kind of do.

configurejson2cmake will now create a corresponding
qt_config_compile_test call where it will pass a list of packages and
libraries to find and link against.

pro2cmake will in turn generate new code for the standalone
config.test project. This code will iterate over packages that need to
be found (like WrapAssimp) and then link against a list of passed-in
targets.

In this way the config.test/assimp/main.cpp file can successfully
use assimp code (due to propagated include headers).

qt_config_compile_test is augmented to take a new PACKAGES argument,
with an example as follows

PACKAGES PACKAGE Foo 6 COMPONENTS Bar
         PACKAGE Baz REQUIRED

The arguments will be parsed and passed to the try_compile project,
to call find_package() on them.

We also need to pass the C/C++ standard values to the try_compile
project, as well as other try_compile specific flags, like the
toolchain, as given by qt_get_platform_try_compile_vars().

Change-Id: I4a3f76c75309c70c78e580b80114b33870b2cf79
Reviewed-by: Leander Beernaert <leander.beernaert@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-03 16:05:19 +02:00
Edward Welbourne
cabd8f860b Ensure we use UTF-8 for the emitted QLocaleXML data file
Python helpfully uses a sensible locale when stdout is a tty but uses
the system (not the filesystem) default encoding, which may be ascii
and unable to encode some of the data we need to save. So brute force
kludge it to ensure emit.encoding is UTF-8 when writing the output
we'll read as UTF-8 anyway.

(This matches dev's commit 0ef79d94f6
for the reworked version of the script.)

Task-number: QTBUG-79902
Change-Id: I60ddc896a308c06e01fa87e8e18e112faa17d601
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:44:06 +01:00
Edward Welbourne
67c0e28789 Purge a stray space from calendar locale data
It was causing all lines after the first, in each calendar's
locale_data[], to be over-indented. This only changes spacing.

Change-Id: Ibfc4986548eecbfdba2902cc18f44a2af669bc6d
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2020-04-02 19:44:01 +01:00
Edward Welbourne
3dfdc9b97a Convert the qlocale2cpp's last few %-formats to modern format() style
I've taken care of all the others in the course of other changes
already ...

Task-number: QTBUG-81344
Change-Id: I44e40a0d1c9f1e1a540a5f4cd252369fdc9b2698
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:50 +01:00
Edward Welbourne
963931550d Check all matches for each XPath when searching
Previously, if we found one element with required attributes, we would
search into it and ignore any later elements also with those required
attributes. This meant that, if the first didn't contain the child
elements we were looking for, we'd fail to find what we sought, if it
was in a later matching element (e.g. with some ignored attributes).
We would then go on to look for a match in a later file, where there
might have been a match we should have found in the earlier file.

Check all matches, rather than only the first match in each file.  Do
the search in each file "in parallel" to save reparsing the XPath.
This clears the search code of rather hard-to-follow break/else
handling in loops; and currently makes no change to the generated
data.

Change-Id: I86b010e65b9a1fc1b79e5fdd45a5aeff1ed5d5d5
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:43 +01:00
Edward Welbourne
89bd12b9ad Change QLocale to use CLDR's accounting formats for currencies
In particular, this changed the US currency formats for negative
amounts to be parenthesised versions of the positive amount forms,
rather than having a minus sign after the $ sign. Test updated.

[ChangeLog][QtCore][QLocale] Currency formats are now based on CLDR's
accounting formats, where they were previously mostly based (more or
less by accident) on standard formats. In particular, this now means
negative currency formats are specified, where available, where they
(mostly) were not previously.

Task-number: QTBUG-79902
Change-Id: Ie0c07515ece8bd518a74a6956bf97ca85e9894eb
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 20:43:34 +02:00
Edward Welbourne
81cf23c7a7 Take CLDR's distinguished attributes into account
When doing XPATH searches, child nodes that have distinguished
attributes that were not asked for should be skipped. This is part of
the LDML spec and matters when resolving locale inheritance. Scan the
LDML DTD (previously only scanned for the CLDR version) to find which
attributes of which tags are ignorable - all others are distinguished
- and take the result into account when performing XPATH searches.

The XPath we were using for currency formats wasn't excluding
currencyFormatLength elements with type="short" and patterns specific
to thousands (and larger multiples); this is fixed by taking
distinguished attributes into account. However, the XPATH also wasn't
specifying the always distinguished attribute type="standard" that
was, in practice, used for nearly all locales that weren't (wrongly)
using short-forms for thousands; so type="standard" is now made
explicit, so as to minimize the diff.

This leaves only twenty-one locales with a negative currency formats.
A later commit shall switch to using accounting by default (it falls
back via an alias to standard, in any case), thereby restoring the two
mentioned below that were using it by accident, but the present change
gives the minimal diff here.

Thousands-specific formats replaced with sensible ones:
* zh_Hant_{HK,MO} (Traditional Mandarin, Hong Kong and Macau)
* eo_001 (Esperanto)
* fr_CA (Canadian French)
* ha_* (Hausa, when not written in Arabic)
* es_{GT,MX,US} (Spanish - Guatemala, Mexico, USA)
* sw_KE (Swahili, Kenya)
* yi_001 (Yiddish)
* mfe_MU (Morisyen, Mauritius)
* lag_TZ (Langi, Tanzania)
* mgh_MZ (Makhuwa Meetto, Mozambique)
* wae_CH (Walser, Switzerland)
* kkj_CM (Kako, Cameroon)
* lkt_US (Lakota, USA)
* pa_Arab_PK (Punjabi, in Arabic script, as used in Pakistan; uses
  arabext number system, whose currency falls back to latn's, for
  which pa_Arab over-rides the thousands-format).

Format changed from an over-ridden type="accounting" to standard (so
these lost a negative-specific form) in:
* en_SI (English, Slovenia)
* es_DO (Spanish, Dominican Republic; same)

For some locales we were picking up over-rides of narrow or short list
formats, or formats for or-lists or unit-lists rather than and-lists,
in place of the standard list format, that these locales don't
over-ride, provided by a parent locale. This changed list formats for:
* en_CA, en_IN (dropped "Oxford" comma before "and")
* qu_* (Quechua; dropped "utaq", presumably meaning "and")
* ur_IN (Urdu, India; was using unit-list formats)

[ChangeLog][QtCore][QLocale] Data used for currency formats in several
locales and list patterns in some locales have changed due to now
parsing the CLDR data more faithfully.

Fixes: QTBUG-81344
Change-Id: I6b95c6c37db92df167153767c1b103becfb0ac98
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:28 +01:00
Edward Welbourne
e5eb0aa428 Take number system into account in currency format look-up
CLDR's currency formats do have number system variation, so take it
into account. (The old xpathlite code clearly intended to do this, but
failed at it due to looking for the wrong component of an XPATH to
fix.) This changes the currency formats in use for
* all Dutch locales (because nl.xml lists a currency format for arab
  before the one for latn, and they differ),
* Punjabi, Urdu - specifically pa_Guru_IN, ur_Arab_PK (both like
  Dutch, arabext before latn; which is correct for pa_Arab_PK and
  ur_Arab_IN),
* Sindi (whose over-ride of latn currency format we were using, where
  we should be using arab's format, supplied by root's default),
* Tatar (which specifies a generic currency format, which we were
  using, before one specific to latn, which we now use),
* Tongan (same as Dutch),
* Konkani (like Dutch, deva before latn) and
* several North African Arabic locales (whose default number system is
  latn, rather than arab, but previously used arab's formats).

Task-number: QTBUG-79902
Change-Id: I18d8ec16bfd3a516d1bcd2f63bc7f7f15179a3f4
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:23 +01:00
Edward Welbourne
be3dfd7a71 Rework cldr2qlocalexml.py's reading of CLDR data
Move the code out to a CldrReader class in cldr.py, expand CldrAccess
with facilities that needs, expand ldml.py to include support for more
features, finally making xpathlite.py redundant. This initial commit
aims, though, to be bug-for-bug compatible with xpathlite in its
reading of the CLDR data.

It turns out we've been using draftier data than we were aware of
(which might not be a bad thing). The xpathlite code appeared to check
for draft attributes, but these only appear on leaf nodes and most
data were fetched by finding a parent and then scanning its children
without the draft check; only am/pm data was actually being excluded
based on draft values.  (We allowed contributed, for am/pm, in
addition to approved, which is all the xpathlite code allows
otherwise.) There are also some less equivocal bugs; I'll deal with
these in later commits.

Simplified number-system data look-ups; the old get_number_in_system()
was taking care of old LDML versions' placement of the number system
attribute; this is no longer needed. (It was also being used for a
currency value to which it was not appropriate, which is now handled
separately; this is one of the bugs mentioned above.) Ditched a
fall-back to nativeZeroDigit, which no longer exists in CLDR.

Change the command-line to take the root of the CLDR data tree, rather
than its common/main/ sub-directory. Support naming the file to which
to write output, as a second command-line argument, instead of always
writing to stdout (which remains the default) and leaving whoever runs
the script to redirect stdout.

Support (internally for now, while adding TODOs to give main() more
command-line options) separating the stderr output into its more and
less interesting parts; for now, continue producing both, but suppress
the least interesting entirely.

Task-number: QTBUG-81344
Change-Id: Ie611b47403a9452b51feaeeaaa0fbc8f7e84dc71
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:18 +01:00
Edward Welbourne
c834dbc6fb Move cldr2qtimezone.py's CLDR-reading to a CldrAccess class
This begins the process of replacing xpathlite.py, adding low-level
DOM-access classes to ldml.py and the CldrAccess class to cldr.py

Moved a format comment from cldr2qtimezone.py's doc-string to the
method of CldrAccess that does the actual reading.

Task-number: QTBUG-81344
Change-Id: I46ae3f402f8207ced6d30a1de5cedaeef47b2bcf
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:13 +01:00
Edward Welbourne
9fab53a513 Rework qlocalexml2cpp.py to use writers based on Transcriber
This saves repetition of temporary-file manipulation code. In the
process, ensure that we tidy away temporary files on failure.

Moved a comment in qlocale.h to *outside* the re-written portion, to
save having to rewrite it every time. Added blank lines to separate
script data from country data in the generated output. Changed 0s in
one comment to zeros, to match another comment.

Isolated use of sys to the __main__ block.
Isolated use of enumdata to the new LocaleHeaderWriter class.
Modernised all the string-formatting I touched.

Task-number: QTBUG-81344
Change-Id: I5768e45d9a8ea23facc303b3dd8af8b3ccbf7ff2
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:56 +01:00
Edward Welbourne
5b1c33cc78 Rework cldr2qtimezone.py into more maintainable form
Broke out the updating of a source file to a ZoneIdWriter helper
class, which enables tidying away the temporary file if we fail.
Collected up the rest of the script into a main() that's now
called from a __name__ == '__main__' block.
Rationalized the imports.

Eliminated an inefficient lookup function by constructing a suitable
dict() before entering the loop that needed it.

Separated the "data you might need to update" tables from the code
that does the work, to make it easier for those adding support for new
zones to see what they're doing.

Removed the spurious $Revision$ from the output and reworded the
premable of the generated file. (It would seem CLDR no longer uses an
RCS-based version-control system.) Generated output is otherwise
unchanged.

Task-number: QTBUG-81344
Change-Id: I7d9de8357ebcb599d154de9f862e25f7ade00390
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:50 +01:00
Edward Welbourne
bb4242341b Add tools to localetools to facilitate source file recreation
For now unused; later commits shall put them to use.
Transcriber -- base, takes care of tempfile and renaming.
SourceFileEditor -- handles copying parts before and after a common delimiter.

Task-number: QTBUG-81344
Change-Id: I28cf977d0a08825fbb873fb330da6823b88ad3ed
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:45 +01:00
Edward Welbourne
c3dea1ffca Move some shared code to a localetools module
The time-zone script was importing two functions from the locale data
generation script. Move them to a separate module, to which I'll
shortly add some more shared utilities. Cleaned up some imports in the
process.

Combined qlocalexml2cpp's and xpathlit's error classes into a new
Error class in the new module and made it a bit more like a proper
python error class.

Task-number: QTBUG-81344
Change-Id: Idbe0139ba9aaa2f823b8f7216dee1d2539c18b75
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:40 +01:00
Edward Welbourne
4d9f1a87de Move qlocalexml2cpp.py's XML-reading to QLocaleXmlReader
This new class mirrors the existing QLocaleXmlWriter and places the
two side-by-side in qlocalexml.py, rather than having the writing and
reading in separate places.

Made judicious use of transformed versions of mappings to save
repeated iteration of a mapping's entries to do lookups on fist
entries of pair-values; several (id, name, code) data-sets are
sometimes indexed by id, sometimes by name.

Reworked the default_map, that the complicated compareLocaleKeys()
used in sorting locale keys, to map IDs instead of names; the function
also needed the locale_map so that it could convert IDs to names,
which we can skip by going directly with IDs.

Task-number: QTBUG-81344
Change-Id: Iff6a97f7f0755b56dda70d8a6796ec074c558910
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:34 +01:00