qt5base-lts/cmake
Alexandru Croitor ec90f9013b CMake: Fix handling of CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH
While trying to implement the 'host artifact reuse' Coin instructions
change, a bug surfaced where the qemu configurations didn't find
the host tools and instead tried to use the cross-compiled tools
while building qtbase, which failed due to not finding the
runtime linker (another unsolved issue).
Before the host artifact reuse change, the host tools were found
successfully.

The difference that caused the issue is that the target install prefix
was a direct subfolder of the host prefix.
host - /home/qt/work/qt/install
target - /home/qt/work/qt/install/target

Before the host reuse change the install prefixes were as follows
host - /home/qt/work/qt/install/host
target - /home/qt/work/qt/install/target

While looking for the Qt6CoreTools package, we temporarily set
CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH to contain first
'/home/qt/work/qt/install' and then '/home/qt/work/qt/install/target'.
CMake then reroots the CMAKE_PREFIX_PATH values onto values in
CMAKE_FIND_ROOT_PATH, making an MxN list of prefixes to search.
Rerooting essentially means concatenating 2 paths, unless the
considered prefix is a subfolder of the root path.

What happened was that the first considered value was
'/home/qt/work/qt/install/home/qt/work/qt/install', a non-existent
location that gets discarded.

The second considered value was '/home/qt/work/qt/install/target.
The second value is the result of seeing that
'/home/qt/work/qt/install/target' is a subfolder of
'/home/qt/work/qt/install' and thus the root path is stripped.

All of this is done in cmFindPackageCommand::FindConfig() ->
cmFindCommon::RerootPaths.

The behavior above caused the target tools be found instead of the
host ones.

Before the host reuse change, both of the initial constructed prefixes
were discared due to them not existing, e.g.
'/home/qt/work/qt/install/target/home/qt/work/qt/install/target'
and '/home/qt/work/qt/install/host/home/qt/work/qt/install/host'

One of the later prefixes combined CMAKE_FIND_ROOT_PATH ==
'/home/qt/work/qt/install/host' + CMAKE_PREFIX_PATH == '/' resulting
in '/home/qt/work/qt/install/host/' and this accidentally found the
host tools package.

We actually stumbled upon this issue a while ago when implementing Qt
5.14 Android CMake support in 52c799ed44
That commit message mentions the fix is to add a "lib/cmake"
suffix to the PATHS option of find_package().
This would cause the subfolder => strip root behavior mentioned
above.

So finally the fix.

First, make sure not to append QT_HOST_PATH in the toolchain file,
there shouldn't be any need to do that, give that we temporarily set
it when looking for Tools packages.

Second, recreate the subdir scenario in the Qt toolchain file by
setting CMAKE_FIND_ROOT_PATH to the current (relocated) install
prefix as usual, but also setting CMAKE_PREFIX_PATH to a new value
poining to the CMake directory.
Aka '/home/alex/qt' and '/home/alex/qt/lib/cmake'.

Third, when a QT_HOST_PATH is given, save 2 paths in the generated Qt
toolchain: QT_HOST_PATH and QT_HOST_PATH_CMAKE_DIR. There are the host
equivalents of the target ones above. Use these values when looking
for host tools in Qt6CoreModuleDependencies.cmake, again facilitaing
the subdir behavior.
Note these are currently absolute paths and are not relocatable.
We'll have to figure out if it's even possible to make the host path
relocatable.

Finally as a cleanup, look for the Qt6HostInfo package in QtSetup
strictly in the given QT_HOST_PATH, so CMake doesn't accidentally find
a system Qt package.

Change-Id: Iefbcfbbcedd35f1c33417ab7e9f44eaf35ff6337
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-07-24 17:33:21 +02:00
..
3rdparty CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
macos cmake: Add default Info.plist for macOS with some important keys 2020-03-13 15:23:56 +01:00
QtBuildInternals CMake: Fix configuration of CMake tests in-build-tree 2020-07-20 18:12:23 +02:00
tests
configure-cmake-mapping.md CMake: Document the mapping of configure options to CMake arguments 2020-07-02 09:00:08 +02:00
FindATSPI2.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindCups.cmake
FindDB2.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindDirectFB.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
Finddouble-conversion.cmake
FindGLESv2.cmake CMake: Use correct framework link flags in scripts and .pri files 2020-07-01 10:03:38 +02:00
FindGSSAPI.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindGTK3.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindLibproxy.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindLibsystemd.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindLibudev.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindMtdev.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindMySQL.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindOracle.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindPPS.cmake
FindSlog2.cmake CMake: Namespace all our IMPORTED targets 2020-04-29 23:08:32 +02:00
FindTslib.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindWrapAtomic.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapDBus1.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapDoubleConversion.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapFreetype.cmake CMake: Handle conditions in third party find modules correctly 2020-03-05 16:08:14 +01:00
FindWrapHarfbuzz.cmake CMake: Handle conditions in third party find modules correctly 2020-03-05 16:08:14 +01:00
FindWrapOpenGL.cmake CMake: Use correct framework link flags in scripts and .pri files 2020-07-01 10:03:38 +02:00
FindWrapOpenSSL.cmake CMake: Handle finding of OpenSSL headers correctly 2020-04-08 22:03:24 +02:00
FindWrapOpenSSLHeaders.cmake CMake: Handle finding of OpenSSL headers correctly 2020-04-08 22:03:24 +02:00
FindWrapPCRE2.cmake CMake: Handle conditions in third party find modules correctly 2020-03-05 16:08:14 +01:00
FindWrapPNG.cmake CMake: Handle conditions in third party find modules correctly 2020-03-05 16:08:14 +01:00
FindWrapRt.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapSystemFreetype.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapSystemHarfbuzz.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapSystemPCRE2.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindWrapSystemPNG.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
FindXKB_COMMON_X11.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindXRender.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
FindZSTD.cmake CMake: Mark all find_package(PkgConfig) to be quiet 2020-06-29 12:23:10 +02:00
ModuleDescription.json.in Write out a module description file 2019-10-21 11:28:55 +00:00
Qt3rdPartyLibraryConfig.cmake.in Add support for qt_helper_lib() 2020-02-12 08:43:29 +00:00
qt.toolchain.cmake.in CMake: Fix handling of CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH 2020-07-24 17:33:21 +02:00
QtAutoDetect.cmake CMake: Print various CMake and Qt CMake information variables 2020-07-20 18:12:29 +02:00
QtBaseCMakeTesting.cmake
QtBaseConfigureTests.cmake CMake: Print various CMake and Qt CMake information variables 2020-07-20 18:12:29 +02:00
QtBaseGlobalTargets.cmake CMake: Fix handling of CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH 2020-07-24 17:33:21 +02:00
QtBuild.cmake Avoid enabling of "warnings_are_errors" flags for tests 2020-07-23 15:37:48 +03:00
QtBuildInformation.cmake CMake: Fix path to qt-cmake in configure summary 2020-07-09 08:55:38 +02:00
QtBuildInternalsExtra.cmake.in CMake: Fix CMAKE_INSTALL_PREFIX assignment in QtBuildInternalsExtra 2020-07-13 14:56:44 +02:00
QtCompilerFlags.cmake CMake: Allow disabling warnings similar to CONFIG += warn_off 2020-02-25 15:43:15 +01:00
QtCompilerOptimization.cmake CMake: Fix incorrect SIMD arch_haswell and avx profile conditions 2020-07-09 23:02:39 +02:00
QtConfig.cmake.in cmake: Pick up custom Info.plist in user projects 2020-06-26 14:58:42 +02:00
QtFeature.cmake CMake: Pass CMAKE_OSX_ARCHITECTURES to try_compile on macOS 2020-07-20 18:12:26 +02:00
QtFileConfigure.txt.in CMake: Introduce qt_configure_file 2020-03-23 09:54:56 +01:00
QtFindWrapConfigExtra.cmake.in CMake: Handle conditions in third party find modules correctly 2020-03-05 16:08:14 +01:00
QtFindWrapHelper.cmake CMake: Display found information in some of the Find modules 2020-06-26 22:14:48 +02:00
QtFinishPrlFile.cmake CMake: Write object libs of Qt resources to .prl files 2020-06-16 17:56:35 +02:00
QtGenerateExtPri.cmake CMake: Generate information about 3rdparty libs in module .pri files 2020-05-19 13:26:53 +02:00
QtGenerateLibPri.cmake CMake: Use correct framework link flags in scripts and .pri files 2020-07-01 10:03:38 +02:00
QtHostInfoConfig.cmake.in CMake: Write QT_CPU_FEATURES to qmodule.pri 2020-07-02 15:35:17 +02:00
QtInternalTargets.cmake Add -Wsuggest-override for gcc >= 9.2 2020-07-24 13:10:51 +03:00
QtModuleConfig.cmake.in CMake: Provide old style CMake variables for target info 2020-07-17 09:43:00 +02:00
QtModuleDependencies.cmake.in CMake: Fix handling of CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH 2020-07-24 17:33:21 +02:00
QtModuleToolsConfig.cmake.in CMake: Fix creation of versionless targets for tools 2019-12-06 11:11:16 +00:00
QtModuleToolsDependencies.cmake.in Provide a way to specify extra tool package dependencies 2019-11-26 10:58:56 +00:00
QtModuleToolsVersionlessTargets.cmake.in CMake: Fix creation of versionless targets for tools 2019-12-06 11:11:16 +00:00
QtPlatformAndroid.cmake Raise the Android SDK build requirement to lvl 28 - compile fix 2020-06-23 16:39:04 +02:00
QtPlatformSupport.cmake CMake: Add additional info which was missing in qconfig.pri 2020-06-17 15:13:43 +02:00
QtPluginConfig.cmake.in Implement qtbase fixes for superbuilds 2020-01-08 10:42:21 +00:00
QtPluginDependencies.cmake.in Always try to find dependencies for plugins 2019-10-14 15:02:52 +00:00
QtPlugins.cmake.in Fix qt_import_plugins compatibility with Qt 5 2020-01-27 11:59:51 +00:00
QtPostProcess.cmake CMake: Provide old style CMake variables for target info 2020-07-17 09:43:00 +02:00
QtProcessConfigureArgs.cmake CMake: Fix -pch configure option 2020-07-08 07:09:18 +02:00
QtProperties.cmake Rename / prefix all our private API functions with qt_ 2019-11-14 09:05:33 +00:00
QtSeparateDebugInfo.cmake CMake: Make use of CMAKE_CURRENT_FUNCTION_LIST_DIR 2020-04-17 23:10:08 +02:00
QtSeparateDebugInfo.Info.plist.in CMake: Port the 'separate_debug_info' feature 2020-03-06 13:38:37 +01:00
QtSetup.cmake CMake: Fix handling of CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH 2020-07-24 17:33:21 +02:00
QtStandaloneTestsConfig.cmake.in Make standalone tests build via top level repo project 2019-11-08 15:42:32 +00:00
README.md CMake: Document the mapping of configure options to CMake arguments 2020-07-02 09:00:08 +02:00

Status

Port is still on-going. Most of qtbase and qtsvg is ported. Other repositories are ported, but not under CI control yet. Some libraries, tests and examples are still missing.

Note: You need CMake 3.16.0 or later for most platforms (due to new AUTOMOC json feature). You need CMake 3.17.0 to build Qt for iOS with the simulator_and_device feature. You need CMake 3.17.0 + Ninja to build Qt in debug_and_release mode on Windows / Linux. You need CMake 3.18.0 + Ninja to build Qt on macOS in debug_and_release mode when using frameworks.

Intro

The CMake update offers an opportunity to revisit some topics that came up during the last few years.

  • The Qt build system does not support building host tools during a cross-compilation run. You need to build a Qt for your host machine first and then use the platform tools from that version. The decision to do this was reached independent of cmake: This does save resources on build machines as the host tools will only get built once.

  • For now Qt still ships and builds bundled 3rd party code, due to time constraints on getting all the necessary pieces together in order to remove the bundled code (changes are necessary not only in the build system but in other parts of the SDK like the Qt Installer).

  • There is less need for bootstrapping. Only moc and rcc (plus the lesser known tracegen and qfloat16-tables) are linking against the bootstrap Qt library. Everything else can link against the full QtCore. This will include qmake. Qmake is supported as a build system for applications using Qt going forward and will not go away anytime soon.

  • We keep the qmake-based Qt build system working so that we do not interfere too much with ongoing development.

Building against homebrew on macOS

You may use brew to install dependencies needed to build QtBase.

  • Install homebrew: /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • Build Qt dependencies: brew install pcre2 harfbuzz freetype
  • Install cmake: brew install cmake
  • When running cmake in qtbase, pass -DCMAKE_PREFIX_PATH=/usr/local

Building

The basic way of building with cmake is as follows:

    cd {build directory}
    cmake -DCMAKE_INSTALL_PREFIX=/path/where/to/install {path to source directory}
    cmake --build .
    cmake --install .

The mapping of configure options to CMake arguments is described here.

You need one build directory per Qt module. The build directory can be a sub-directory inside the module qtbase/build or an independent directory qtbase_build. The installation prefix is chosen when running cmake by passing -DCMAKE_INSTALL_PREFIX. To build more than one Qt module, make sure to pass the same install prefix.

cmake --build and cmake --install are simple wrappers around the basic build tool that CMake generated a build system for. It works with any supported build backend supported by cmake, but you can also use the backend build tool directly, e.g. by running make.

CMake has a ninja backend that works quite well and is noticeably faster (and more featureful) than make, so you may want to use that:

    cd {build directory}
    cmake -GNinja -DCMAKE_INSTALL_PREFIX=/path/where/to/install {path to source directory}
    cmake --build .
    cmake --install .

You can look into the generated build.ninja file if you're curious and you can also build targets directly, such as ninja lib/libQt6Core.so.

Make sure to remove CMakeCache.txt if you forgot to set the CMAKE_INSTALL_PREFIX on the first configuration, otherwise a second re-configuration will not pick up the new install prefix.

You can use cmake-gui {path to build directory} or ccmake {path to build directory} to configure the values of individual cmake variables or Qt features. After changing a value, you need to choose the configure step (usually several times:-/), followed by the generate step (to generate makefiles/ninja files).

Developer Build

When working on Qt itself, it can be tedious to wait for the install step. In that case you want to use the developer build option, to get as many auto tests enabled and no longer be required to make install:

    cd {build directory}
    cmake -GNinja -DFEATURE_developer_build=ON {path to source directory}
    cmake --build .
    # do NOT make install

Specifying configure.json features on the command line

QMake defines most features in configure.json files, like -developer-build or -no-opengl.

In CMake land, we currently generate configure.cmake files from the configure.json files into the source directory next to them using the helper script path_to_qtbase_source/util/cmake/configurejson2cmake.py. They are checked into the repository. If the feature in configure.json has the name "dlopen", you can specify whether to enable or disable that feature in CMake with a -D flag on the CMake command line. So for example -DFEATURE_dlopen=ON or -DFEATURE_sql_mysql=OFF. At the moment, if you change a FEATURE flag's value, you have to remove the CMakeCache.txt file and reconfigure with CMake. And even then you might stumble on some issues when reusing an existing build, because of an automoc bug in upstream CMake.

Building with CCache

You can pass -DQT_USE_CCACHE=ON to make the build system look for ccache in your PATH and prepend it to all C/C++/Objective-C compiler calls. At the moment this is only supported for the Ninja and the Makefile generators.

Cross Compiling

Compiling for a target architecture that's different than the host requires one build of Qt for the host. This "host build" is needed because the process of building Qt involves the compilation of intermediate code generator tools, that in turn are called to produce source code that needs to be compiled into the final libraries. These tools are built using Qt itself and they need to run on the machine you're building on, regardless of the architecure you are targeting.

Build Qt regularly for your host system and install it into a directory of your choice using the CMAKE_INSTALL_PREFIX variable. You are free to disable the build of tests and examples by passing -DBUILD_EXAMPLES=OFF and -DBUILD_TESTING=OFF.

With this installation of Qt in place, which contains all tools needed, we can proceed to create a new build of Qt that is cross-compiled to the target architecture of choice. You may proceed by setting up your environment. The CMake wiki has further information how to do that at

https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/CrossCompiling

Yocto based device SDKs come with an environment setup script that needs to be sourced in your shell and takes care of setting up environment variables and a cmake alias with a toolchain file, so that you can call cmake as you always do.

In order to make sure that Qt picks up the code generator tools from the host build, you need to pass an extra parameter to cmake:

    -DQT_HOST_PATH=/path/to/your/host_build

The specified path needs to point to a directory that contains an installed host build of Qt.

Cross Compiling for Android

In order to cross-compile Qt to Android, you need a host build (see instructions above) and an Android build. In addition, it is necessary to install the Android NDK.

The environment for Android can be set up using the following steps:

  • Set the ANDROID_NDK_HOME environment variable to the path where you have installed the Android NDK.
  • Set the ANDROID_SDK_HOME environment variable to the path where you have installed the Android SDK.

When running cmake in qtbase, pass -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK_HOME/build/cmake/android.toolchain.cmake -DQT_HOST_PATH=/path/to/your/host/build -DANDROID_SDK_ROOT=$ANDROID_SDK_HOME -DCMAKE_INSTALL_PREFIX=$INSTALL_PATH

If you don't supply the configuration argument -DANDROID_ABI=..., it will default to armeabi-v7a. To target other architectures, use one of the following values:

  • arm64: -DANDROID_ABI=arm64-v8
  • x86: -DANDROID_ABI=x86
  • x86_64: -DANDROID_ABI=x86_64

By default we set the android API level to 21. Should you need to change this supply the following configuration argument to the above CMake call: -DANDROID_NATIVE_API_LEVEL=${API_LEVEL}

Cross compiling for iOS

In order to cross-compile Qt to iOS, you need a host macOS build.

When running cmake in qtbase, pass -DCMAKE_SYSTEM_NAME=iOS -DQT_HOST_PATH=/path/to/your/host/build -DCMAKE_INSTALL_PREFIX=$INSTALL_PATH

If you don't supply the configuration argument -DQT_UIKIT_SDK=..., it will default to iphonesimulator. To target another SDK / device type, use one of the following values:

  • iphonesimulator: -DQT_UIKIT_SDK=iphonesimulator
  • iphoneos: -DQT_UIKIT_SDK=iphoneos
  • simulator_and_device: -DQT_FORCE_SIMULATOR_AND_DEVICE=ON -DQT_UIKIT_SDK=

Depending on what value you pass to -DQT_UIKIT_SDK= a list of target architectures is chosen by default:

  • iphonesimulator: x86_64
  • iphoneos: arm64
  • simulator_and_device: arm64;x86_64

You can try choosing a different list of architectures by passing -DCMAKE_OSX_ARCHITECTURES=x86_64;i386. Note that if you choose different architectures compared to the default ones, the build might fail. Only do it if you know what you are doing.

simulator_and_device special considerations

To do a simulator_and_device build, an unreleased version of CMake is required (3.17.0).

Debugging CMake files

CMake allows specifying the --trace and --trace-expand options, which work like qmake -d -d: As the cmake code is evaluated, the values of parameters and variables is shown. This can be a lot of output, so you may want to redirect it to a file using the --trace-redirect=log.txt option.

Porting Help

We have some python scripts to help with the conversion from qmake to cmake. These scripts can be found in utils/cmake.

configurejson2cmake.py

This script converts all configure.json in the Qt repository to configure.cmake files for use with CMake. We want to generate configure.cmake files for the foreseeable future, so if you need to tweak the generated configure.cmake files, please tweak the generation script instead.

configurejson2cmake.py is run like this: util/cmake/configurejson2cmake.py . in the top-level source directory of a Qt repository.

pro2cmake.py

pro2cmake.py generates a skeleton CMakeLists.txt file from a .pro-file. You will need to polish the resulting CMakeLists.txt file, but e.g. the list of files, etc. should be extracted for you.

pro2cmake.py is run like this: path_to_qtbase_source/util/cmake/pro2cmake.py some.pro.

run_pro2cmake.py

`` A small helper script to run pro2cmake.py on all .pro-files in a directory. Very useful to e.g. convert all the unit tests for a Qt module over to cmake;-)

run_pro2cmake.py is run like this: path_to_qtbase_source/util/cmake/run_pro2cmake.py some_dir.

vcpkg support

The initial port used vcpkg to provide 3rd party packages that Qt requires.

At the moment the Qt CI does not use vcpkg anymore, and instead builds bundled 3rd party sources if no relevant system package is found.

While the supporting code for building with vcpkg is still there, it is not tested at this time.

How to convert certain constructs

qmake CMake
qtHaveModule(foo) if(TARGET Qt::foo)
qtConfig(foo) if (QT_FEATURE_foo)