Commit Graph

125 Commits

Author SHA1 Message Date
Alexandru Croitor
0a96302dc1 More fixes to conversion script while porting qtdeclarative
Hardcode a few cases regarding scopes containing "qtConfig(opengl)".
These arew few, and contain regular expressions, to just manually
check and replace them.

Add a new entry to _qt_library_map for handling QmlModels module.

Fix Scope.children property to recursively access the .children
property on included scopes (instead of just ._children) so that
we get the full list of scopes from included children that include
other scopes. This is needed for nested .pri files.

Fix mapping of "win*" to WIN32.

Change-Id: If949a8051f517683e56cda605733719daadb384a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
Reviewed-by: Qt CMake Build Bot
2019-06-03 14:41:52 +00:00
Simon Hausmann
8fa646dbe2 Add basic support for compiling applications with qmake
* Generate module .pri files
* Generate qconfig.pri
* Propagate MODULE_CONFIG from the .pro files

This enables the basic use-case of simple application builds that for
example use the moc. Omitted from the patch is support for private
module configurations, prl files (should we do this?) and possibly more
hidden gems that need to be implemented to for example support building
Qt modules with qmake.

Task-number: QTBUG-75666
Change-Id: Icbf0d9ccea4cd683e4c38340b9a2320bf7951d0d
Reviewed-by: Qt CMake Build Bot
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-31 12:53:09 +00:00
Leander Beernaert
d908e0a47d Fix uncessary relative dir suffix in pro2cmake.py
When we replace the PWD key from qmake files and both the base and
current directory are the same, there's no need to add an extra './' to
the current working directory.

This also fixes a unit test in qtsvg as it requires the passed in path
to match exactly to the one outputed in the log files.

Change-Id: Ide9ca6a70493e8039d3af84a9e576d8f6a313f2a
Reviewed-by: Qt CMake Build Bot
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-05-29 08:58:52 +00:00
Alexandru Croitor
499771f3ea Enhance the porting scripts with some new functionality
These were some hard requirements while porting QtQml .pro files so
that the generated CMake code is syntactically correct and the result
buildable.

This include handling of a few more different condition scopes
and disabling the c++ make_unique feature test.

Change-Id: Iae875ffaf8d100296e8b56b57d076455e5d72006
Reviewed-by: Qt CMake Build Bot
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-05-28 14:04:40 +00:00
Alexandru Croitor
cbb143e9f1 Improve configurejson2cmake.py to handle non-compliant qmake JSON
Some configure.json files contain new lines inside quoted strings,
which is not conformant with the JSON spec.

Add a new json_parser python module which uses pyparsing to preprocess
the json files to remove the new lines inside the quoted strings, and
then hands over the preprocessed content to the regular json module.

Change-Id: I5f8938492068dda5640465cc78f5a7b6be0e709a
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Qt CMake Build Bot
2019-05-27 07:48:51 +00:00
Alexandru Croitor
bfed22e3b7 Remove all line continuations when processing qmake syntax
We constantly had to adjust the qmake grammar to handle line
continuations (\\\n) in weird places. Instead of doing that,
just do a preprocess step to remove all the LCs like we do with
comments, and simplify the grammar not to take into account the
LCs.

From some manual testing it doesn't look like we get any regressions.

Change-Id: I2017d59396004cf67b6cb54977583db65c65e7d3
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-23 14:19:22 +00:00
Alexandru Croitor
a2e0f19b61 Make debug token output actually work for the pro2cmake qmake parser
setName() and setDebug() need to be called on a parser element before
the parser element is used as a sub-element in another parser element,
otherwise the debug output is not shown.

Hence the "iterate over all locals" approach works only partially.
Instead add a new decorating function add_element() for the
construction of each parser element, and make sure to enable debugging
inside that function.

Unfortunately there is no clean way to avoid duplicating the parser
element name both in the local variable and in the function argument.

Change-Id: Iaa9ed9b7dbb22ec084070b9c049cf51c841d442c
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-23 14:19:11 +00:00
Alexandru Croitor
6dfeb81bcb Improve qmake syntax parser in pro2cmake.py
Some qtdeclarative pro files caused exceptions when trying to parse
them using the script. This included the following:
- handling conditions divided by newlines and backslashes
- handling conditions that have no scope

The parser has been fixed to deal with those cases and relevant
tests were added.

After the change, all qtdeclarative project files are parseable by
the script.

Change-Id: Ib9736423f7fb3bcc1944b26cfb3114306b4db9a7
Reviewed-by: Qt CMake Build Bot
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-23 14:17:46 +00:00
Alexandru Croitor
80271e8280 Improve qmake parser debug output in pro2cmake
Override the default debug actions to be decorated with proper
indentation for easier reading.

The setup only has to be done once, and not on each QMakeParser
creation.

Change-Id: If5f965b462c782c654ee8ebfdd33570e8f94b084
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-23 14:17:34 +00:00
Alexandru Croitor
0425ee0610 Change the default enabled AUTOGEN tools list to contain only moc
Before this patch we enabled AUTOMOC, AUTORCC, AUTOUIC for all targets
that did not opt out.

Aside from being wasteful from a performance point of view,
this also caused issues when trying to build qtimageformats which
does not depend on Widgets which is the package that exposes uic.

To avoid this, enable only AUTOMOC for all targets by default, and
UIC and RCC can be opted in via the ENABLE_AUTOGEN_TOOLS option.

To facilitate this some refactoring had to be done, like moving some
common setup for all autogen tools into a separate call, and making
sure that extend_target understands the autogen options, because some
ui files are only added conditionally.

Also the conversion script has been adapted to output the
ENABLE_AUTOGEN_TOOLS option whenever a .pro file contains at least
one FORMS += foo assignment.

Note that we don't really use AUTORCC while building Qt, so nothing
opts into that at the moment.

Task-number: QTBUG-75875
Change-Id: I889c4980e9fb1b74ba361abed4044737f8842ea4
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-05-22 14:58:08 +00:00
Alexandru Croitor
76f5b784ce Fix the fix to correctly handle comments in multi-line assignments
The previous fix where the Grammar comment style was changed to
remove the newlines was incorrect, because if you have

 foo=1#comment
 bar=2

after the Grammar comment ignoring, it would transform into

 foo=1bar=2

which will clearly fail to parse, so the new line has to stay.

But we would still have the following case which would fail:
 foo=a \
 # comment
 b

Apparently qmake things that's the equivalent of
foo=a b

but the grammar parses it as
 foo=a \
 \n (newline)
 b

Thus the parsing fails because there's a newline and then some
weird 'b' token which the grammar does not expect.

The best fix I found is to preprocess the source, to remove
completely commented out lines.

So:
 foo=a \
 # comment
 b

gets transformed into

 foo=a \
 b

Change-Id: I2487a0dbf94a6ad4d917d0a0ce05247341e9b7da
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-20 13:18:11 +00:00
Alexandru Croitor
846adfe079 Handle the new cmdline config feature in pro2cmake
cmdline implies console, so use that also to determine if a binary
should get a GUI flag.

Change-Id: I084e0a45785df96a7dc2c101af5305fbb39efbc3
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-20 09:23:14 +00:00
Alexandru Croitor
826821658d Workaround fix in pro2cmake Operation __init__ method
There are still call sites that call Operation.__init__ with a string
instead of a list. Restore the handling of such a case.

Amends 5fe8a38af3

Change-Id: I2a4d5c5cb5b460bf02b6da02d42d8cc8d5eb4192
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-20 09:18:56 +00:00
Alexandru Croitor
c454e622a1 Don't exclude QtCore from dependencies when generating modules
and plugins.

src/network/network.pro for instance depends on core-private, but
because we ignore adding QtCore as a public dependency, the exported
Config file for Network doesn't depend on QtCore anymore, so if a user
only links against Network, they won't automatically link against
Core.

Change-Id: I4a60ffae7e071927360b8ccf6b1b7479ab391060
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-20 09:15:31 +00:00
Alexandru Croitor
e0a6e9f3aa Fix parsing qmake assignments that have comments in between
For some reason the python comment regex that we used does not ignore
the line break at the end of a comment line.

This caused issues when parsing multi line assignments with comments
in between.

Use our own regex for comments to circumvent the issue. It was found
while trying to port the qtimageformats repo.

Added a pytest as well.

Change-Id: Ie4bbdac2d1e1c133bc787a995224d0bbd8238204
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-17 13:31:03 +00:00
Tobias Hunger
5fe8a38af3 CMake: Fix test_operations
Fix test_operations and do some small mypy cleanups along the way

Change-Id: I6586b5d3491e5dcf44252c098516f0922fa60420
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-17 13:26:26 +00:00
Tobias Hunger
e9085f4162 CMake: pro2cmake.py: Add target_link_libraries to examples
Change-Id: Ic7054f0c88e228496b7f4855bffa5620ed717f9b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-09 13:14:45 +00:00
Tobias Hunger
35ed41e547 CMake: pro2cmake.py: Generate find_package information into examples
Change-Id: I6dab13ebea4386019f14be5f29a143d194268aac
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-09 13:14:37 +00:00
Alexandru Croitor
bc4687f5bb Keep "special case" modifications when regenerating project files
The pro2cmake.py file is now smarter, and can reapply "special case"
modifications after regenerating a CMakeLists.txt file.

This substantially lowers the maintenance burden when regenerating
many files at once.

See the special_case_helper.py file for details on how it works.

Make sure to commit the generated .prev_CMakeLists.txt file alongside
your CMakeLists.txt changes.

To disable the preservation behavior, you can pass -s or
--skip-special-case-preservation to the script.

To keep around temporary files that are created during this process,
you can pass -k or --keep-temporary-files.

To get more debug output, pass --debug-special-case-preservation.

Fixes: QTBUG-75619
Change-Id: I6d8ba52ac5feb5020f31d47841203104c2a061d8
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-09 11:29:54 +00:00
Tobias Hunger
80e0c615a9 CMake: pro2cmake.py: Separate library extraction logic from writing out data
Separate the logic to find all used libraries from the code that writes out
the link_library information into the CMakeLists(.gen)?.txt files.

This patch will remove some "PUBLIC_LIBRARIES Qt::Core" from generated files.
This is due to us handling some Qt libraries in special ways in some of our
add_qt_* helpers. These special libraries were added to the LIBRARIES section,
but actually they should be added to the PUBLIC_LIBRARIES section instead. Do
so now, so that the newly generated files do not break things again.

Change-Id: I588781087a8aecc4d879e949735671d8085f0698
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-09 11:29:25 +00:00
Tobias Hunger
c4dd1a8fe3 CMake: pro2cmake.py: Extract writing lists into cmake files
Extract the actual functionality to write a list of "things" below
headers and/or cmake parameter and followed by a footer.

Reuse this functionality everywhere we write a list of things.

Change-Id: Ia7647be465b4788a2b1e6a5dbede1ca868f24ae2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-09 07:38:17 +00:00
Tobias Hunger
ee3d9a8b67 CMake: scripts: Fix mypy issues
Change-Id: I706740be79eccd6bf08213fdaf747dde08cd053a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-09 07:37:58 +00:00
Tobias Hunger
5608bf3cba CMake: pro2cmake.py: Add basic support for examples
Examples need to be built stand-alone and as part of Qt, so they
need a special CMakeLists.txt file that supports both use-cases.

Add an --is-example switch to pro2cmake to make it generate these special
CMakeLists.txt files.

This is basic support only and is currently still missing the necessary
find_package calls.

Change-Id: Ie770287350fb8a41e872cb0ea607923caa33073d
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-09 07:37:42 +00:00
Tobias Hunger
0c498ef4ff CMake: pro2cmake.py: Do not fail when run from .pro-file directory
Change-Id: I285b05986e3a58efc060ca0b5732f6e3f5121476
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-08 11:25:58 +00:00
Alexandru Croitor
aed2c1f5ae Force pro2cmake.py to always change the dir to the converted pro file
Otherwise if you call the script from a different directory, path
handling becomes broken and certain files are not found.

Change-Id: Ia2f60abbd312a771330b3d5e928e1ccd0b4a845b
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-05-07 17:08:28 +00:00
Tobias Hunger
059f4ade7f CMake: pro2cmake.py: Fix library mapping
Fix library substitution again which broke when I merged all the
library related pieces of information.

Keep Qt and 3rdparty libraries separate so that dbus does not get
mapped into Qt::DBus (or the other way around).

Make names in helper.py more consistent while at it.

Change-Id: I5e5bf02bdabf8bafe991c5701deca76bde4df2c3
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-05-06 14:20:50 +00:00
Tobias Hunger
7874ce7801 CMake: Improve mapping of libraries
Merge all data related to mapping libraries into one data structure
in helper.py.

Use that data for everything related to library mapping.

This change enables way more features now like e.g. adding find_package
calls into generated files.

Change-Id: Ibbd2a1063cbeb65277582d434a6a672d62fc170b
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-05-03 13:42:25 +00:00
Tobias Hunger
6630937e63 Merge commit 'dev' into 'wip/cmake-merge'
Change-Id: I176c40d031be26a1dd1cf08843e448a660598783
2019-04-16 16:32:08 +02:00
Tobias Hunger
ce809cab29 CMake: pro2cmake.py: Fix handling of libraries in QT
Change-Id: I5737a285ca0575a454e60fad231435d96b2f1be7
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-04-12 11:35:18 +00:00
Tobias Hunger
95cdb0d1ae CMake: pro2cmake.py: Inherrit VPATH from parent scopes
Change-Id: I95b62fdf3a4cba674bef5a58f0d414464daa3b0c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-04-12 11:35:09 +00:00
Tobias Hunger
2ec3f492a7 CMake: pro2cmake.py: Handle setting a key with $$key in the value
Change-Id: I86552ed2a30f07f8c6060b2bad04fd5489b1d482
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-04-12 11:35:01 +00:00
Tobias Hunger
92b0d7b911 CMake: Support /nolink libraries
Add a helper function to QtBuild that generates Foo_nolink versions
of Foo library targets.

Map 'Foo/nolink' libs found in qmake to Foo_nolink.

Automatically run helper function to create _nolink targets as
part of extend_target.

Change-Id: I4c23ea68b3037d23c9a31d4ac272a6bd0565f7c0
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-04-10 09:36:40 +00:00
Tobias Hunger
511124bf1d CMake: pro2cmake.py: Fix plugin type only prints first letter
Change-Id: Ib8989d2c13199d804d0c069903ca1c5eb61763e6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2019-04-09 09:54:23 +00:00
Tobias Hunger
3bc9586e61 CMake: pro2cmake.py: Move defines from QMAKE_CXX_FLAGS into DEFINES
Change-Id: If3bfe715032b21dc046e63a79b0318a161d7a371
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:29:08 +00:00
Tobias Hunger
3cf92ab183 CMake: pro2cmake.py: Better separation between public and private libraries
Change-Id: I95a941ae92a77e049437d3cdd7d06eece11588f4
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:28:58 +00:00
Tobias Hunger
8c28c85465 CMake: pro2cmake.py: Strip '-D' from compile options
Change-Id: I8239d36dc6af1b5eeded7f293dd6c151c10e289f
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:28:44 +00:00
Tobias Hunger
9d83b54015 CMake: pro2cmake.py: Fix handling of file paths
This patch makes sure to store unchanged filenames as taken from qmake
into the scopes it creates.

The scopes are then kept longer: Merging/including scopes is handled by
adding the scope to a private _included_children member of the parent scope.

The methods to access data from scopes are then changed to take the
_included_children into account and a new "get_files" method is added,
that does all the necessary mapping to handle scope-dependent things like
$$PWD, etc.

This makes sure src/network is converted correctly incl. all the .pri-files it
includes as well as src/platformsupport/themes.pro. Both have been troublesome
before.

Change-Id: I28e92b7bcee1b91b248c17201c2729a54a3ce4a1
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:28:30 +00:00
Tobias Hunger
52f676bf66 CMake: pro2cmake.py: Better representation of scopes as str
Include the current directory in the scope __repr__ output to make
it easier to understand where things wrt. include or file names go
wrong.

Change-Id: I09a6c17c6d8d547f1f64801bcde3c2e10c925ee1
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:28:18 +00:00
Tobias Hunger
77a465ea2d CMake: pro2cmake.py: Better handling of debug features
Improve way that debug features are detected.

Change-Id: Ic0f0e74885e6d1e7f236c9efe4c77482ac212dbc
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:28:08 +00:00
Tobias Hunger
9365a01531 CMake: pro2cmake.py: Fix typing information
Change-Id: Iaa6ea69c3c72eb1b626a976fcdb16243f15c609e
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2019-04-09 07:27:59 +00:00
Alexandru Croitor
547c7f8ca5 Fix architecture conditions
The actual variable that contains the architecture is
TEST_architecture_arch. TEST_architecture only contains the value
if the test was performed or not.

Fix the conversion script and all the generated files.

Change-Id: Icb3480832cab894948f4fef03b8bc8187cab6152
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-04-03 09:00:08 +00:00
Alexandru Croitor
e2e4525f78 Fix simplification of conditions involving apple
Previously a condition like APPLE AND (NOT APPLE_OSX) got simplified
to APPLE, which is wrong.

This happened by accident due to some sub-family simplifications
involving BSD, which APPLE was part of.

Technically APPLE is BSD derived, but for the purposes of the
conversion script consider APPLE not to be a BSD (hopefully there
should be no cases of using the bsd scope for apple machines in
qmake files.

Also regenerate the fontdatabase project, where the issue was found.

Change-Id: I18dcf4f29ffbff48c183ba95ca2a2e5cd5325d86
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-04-02 13:32:21 +00:00
Tobias Hunger
03a64ab80d CMake: pro2cmake.py: Handle QDBUSXML2CPP_*_HEADER_FLAGS
Change-Id: Ib5d34a6bf550a11154109064e4e718d0c79c722b
Reviewed-by: Albert Astals Cid <albert.astals.cid@kdab.com>
2019-03-29 14:29:47 +00:00
Tobias Hunger
75658794e4 CMake: pro2cmake.py: Improve handling of dbus interfaces/adaptor sources
Change-Id: I3b642e1ae31996a81618e312183f4cc168bbbe1b
Reviewed-by: Albert Astals Cid <albert.astals.cid@kdab.com>
2019-03-29 14:29:29 +00:00
Tobias Hunger
b9a3217c41 CMake: pro2cmake.py: Handle $$PWD better
Change-Id: I2e28b652c60d3490138ae0548b32d010faccc5a4
Reviewed-by: Albert Astals Cid <albert.astals.cid@kdab.com>
2019-03-29 14:28:51 +00:00
Tobias Hunger
828f2965e7 CMake: pro2cmake.py: deduplicate and sort libraries
Change-Id: I924cfac365a0b4ba18c2579820bc37729f1ea8d9
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-03-29 14:06:41 +00:00
Jędrzej Nowacki
a697df786d Fix exception when parsing tests.pro
The ParseResults may be a nested list of list. Now the code doesn't
raise exceptions, but it fails in do_include as includes that doesn't
provide resolved path will fail. Anyway step in the right direction.

Change-Id: Ice44e4c10d221293cc6c1facca30abd5495791be
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-03-28 13:18:25 +00:00
Jędrzej Nowacki
601c840973 Do not validate file existence if path contains variables
We do not need to validate everything, while converting project
files. Some checks can be left to building step.

It fixes some false positive NOTFOUND errors.

Change-Id: I81ff2421fdea13add0bfc03086152a47bce39908
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-03-28 09:29:06 +00:00
Jędrzej Nowacki
89f34ee42a Fix turned logic in RemoveOperation
It is regression caused by a0a94576fa
("Fix RemoveOperation").

Add unit test for all operation types to make sure this code actually
works:-)

Change-Id: I97c94cb3411f05de89422e3fa2222f2217a09e49
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-03-28 06:30:32 +00:00
Jędrzej Nowacki
a0a94576fa Fix RemoveOperation
The operation was using an empty set as a base, so it was not really
functional.

Change-Id: I98fd80c1ede31994857aa1f0c8947ca7b9f76649
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-03-26 11:14:32 +00:00