There are two possible interpretations of "expected failure": either
the test *must* fail (exactly the inverse of an ordinary test, with
success becoming failure and failure becoming success), or the test
*may* fail (with success intended, but failure possible in some
environments). Autotools had the second interpretation, which seems
more useful in practice, but Meson has the first.
Instead of using should_fail, we can put the tests in one of two new
suites: "flaky" is intended for tests that succeed or fail unpredictably
according to the test environment or chance, while "failing" is for
tests that ought to succeed but currently never do as a result of a
bug or missing functionality. With a sufficiently new version of Meson,
the flaky and failing tests are not run by default, but can be requested
by running a setup that does not exclude them, with a command like:
meson test --setup=x11_unstable --suite=flaky --suite=failing
As a bonus, now that we're setting up setups and their excluded suites
programmatically, the gsk-compare-broadway tests are also excluded by
default when running the test setup for a non-broadway backend.
When running the tests in CI, --suite=gtk overrides the default
exclude_suites, so we have to specify --no-suite=flaky and
--no-suite=failing explicitly.
This arrangement is inspired by GNOME/glib!2987, which was contributed
by Marco Trevisan.
Signed-off-by: Simon McVittie <smcv@debian.org>
This is supposed to test the most fallback GL stuff, so we might want to
set even more env vars here.
Also enable the run for the Fedora builder in CI.
Unify the many copies of diff_with_file in one source
file, and patch it to detect diff at runtime and fall
back to a simple strcmp if we don't have it. Make all
tests use this new testutils.c, and stop requiring
diff for building the tests.
This should let us allow to build on Windows with the
default value for -Dbuild-tests.
Check whether we really have x11 and wayland enabled before we try to setup the
tests to use these respective GDK backends, and only attempt to setup tests
running with the Broadway backend if it has been enabled.
Also, add a setup for running tests with the GDK-Win32 backend on Windows, for
builds that target Windows.
To build a better world sometimes means having to tear the old one down.
-- Alexander Pierce, "Captain America: The Winter Soldier"
ATK served us well for nearly 20 years, but the world has changed, and
GTK has changed with it. Now ATK is mostly a hindrance towards improving
the accessibility stack:
- it maps to a very specific implementation, AT-SPI, which is Linux and
Unix specific
- it requires implementing the same functionality in three different
layers of the stack: AT-SPI, ATK, and GTK
- only GTK uses it; every other Linux and Unix toolkit and application
talks to AT-SPI directly, including assistive technologies
Sadly, we cannot incrementally port GTK to a new accessibility stack;
since ATK insulates us entirely from the underlying implementation, we
cannot replace it piecemeal. Instead, we're going to remove everything
and then incrementally build on a clean slate:
- add an "accessible" interface, implemented by GTK objects directly,
which describe the accessible role and state changes for every UI
element
- add an "assistive technology context" to proxy a native accessibility
API, and assign it to every widget
- implement the AT context depending on the platform
For more information, see: https://gitlab.gnome.org/GNOME/gtk/-/issues/2833
Add test setups that set the GDK_BACKEND and
TEST_OUTPUT_SUBDIR environment variables.
This lets use run
meson test --setup x11 --suite reftest
meson test --setup wayland --suite reftest
and the output will be nicely separated.
We still need to do compositor / display server
setup from the outside.
