Commit Graph

40 Commits

Author SHA1 Message Date
Matthias Clasen
b6e79a7234 testsuite: Use catch as testwrapper
If we find a binary named catch, use it as test wrapper. This
will be used to capture stacktraces for crashes in ci.
2024-07-01 22:29:36 -04:00
Alice Mikhaylenko
1765e1b7c2 testsuite: Add GTK_DEBUG=css
Make named color tests work, since they expect deprecation warnings atm.
2024-06-03 21:53:52 +04:00
Matthias Clasen
b916c4dac1 ci: Reshuffle test setups
Add a wayland_gl setup that explicitly uses desktop GL, and rename
wayland_gles to wayland_gles2 (since that is what it does).

In ci, make the fedora-x86_64 runner run tests with wayland_gl
and wayland_gles2, and make the fedora-release runner run test
with wayland and x11.
2023-11-19 08:24:38 -05:00
Matthias Clasen
5dd0d39a6b Drop performance tests
These were an experient, and not actively used,
and libsysprof has been changing api so they broke.

Fixes: #6112
2023-09-21 13:14:48 -04:00
Patrick Griffis
59941930f7 build: Fix building with introspection disabled and testsuite enabled
The headless tests are written in Python and need g-i.
2023-07-23 19:30:30 -05:00
Benjamin Otte
53bebd2ed1 testsuite: Set GIO_USE_VFS=local everywhere
And do so centrally, not randomly in individual tests.

(Hopefully) fixes spurious test failures in CI.

Related: #5867
2023-06-09 02:29:24 +02:00
Matthias Clasen
9a9257d7be tests: Don't disable vertex arrays
It doesn't work, and that makes tests fail.
2023-05-09 23:02:48 -04:00
Matthias Clasen
f81aa14d93 Merge wayland_smalltexture and wayland_gles
We don't really need to have two full separate
test runs for these conditions, so combine them
into one test setup.
2023-05-04 15:13:09 -04:00
Matthias Clasen
be8e203edc Make the headless input tests work
These needed some updates and workarounds.

They are now part of the testsuite and can
be run like this:

    meson test -Cbuild --suite headless
2023-05-03 23:27:09 -04:00
Matthias Clasen
a235b59089 Skip some tests with small textures
When textures are too large, we will scale them down
to fit the max texture size, which will cause comparisons
to fail.
2023-05-01 16:29:45 -04:00
Benjamin Otte
b64f7050ba testsuite: Make GLES testsuite run use GLES 2
We want to support GLES 2, so make sure we test that support.

Also force-disable common extensions we don't explicitly check for and
don't want to accidentally use.
2023-04-15 17:55:20 +02:00
Matthias Clasen
e7bb7785fa testsuite: Improve setup name consistency
Rename the 'waylandgles' test setup to 'wayland_gles'.
2023-03-19 10:19:00 -04:00
Matthias Clasen
ace2122d45 testsuite: Add a test setup for slicing
Add a 'wayland_smalltexture' test setup that forces
the max texture size to be 1024, which lets us test
the texture slicing codepaths.
2023-03-19 10:15:22 -04:00
Simon McVittie
957dd49ef7 testsuite: Use separate setups for unstable tests instead of should_fail
There are two possible interpretations of "expected failure": either
the test *must* fail (exactly the inverse of an ordinary test, with
success becoming failure and failure becoming success), or the test
*may* fail (with success intended, but failure possible in some
environments). Autotools had the second interpretation, which seems
more useful in practice, but Meson has the first.

Instead of using should_fail, we can put the tests in one of two new
suites: "flaky" is intended for tests that succeed or fail unpredictably
according to the test environment or chance, while "failing" is for
tests that ought to succeed but currently never do as a result of a
bug or missing functionality. With a sufficiently new version of Meson,
the flaky and failing tests are not run by default, but can be requested
by running a setup that does not exclude them, with a command like:

    meson test --setup=x11_unstable --suite=flaky --suite=failing

As a bonus, now that we're setting up setups and their excluded suites
programmatically, the gsk-compare-broadway tests are also excluded by
default when running the test setup for a non-broadway backend.

When running the tests in CI, --suite=gtk overrides the default
exclude_suites, so we have to specify --no-suite=flaky and
--no-suite=failing explicitly.

This arrangement is inspired by GNOME/glib!2987, which was contributed
by Marco Trevisan.

Signed-off-by: Simon McVittie <smcv@debian.org>
2022-11-24 13:46:33 +00:00
Matthias Clasen
df12a41cbb testsuite: Set GDK_DEBUG unconditionally
The default-settings key works in non-debug builds
too, nowadays.
2021-11-03 20:39:33 -04:00
Benjamin Otte
e581c148d9 testsuite: Add run for GLES
This is supposed to test the most fallback GL stuff, so we might want to
set even more env vars here.

Also enable the run for the Fedora builder in CI.
2021-10-09 06:27:21 +02:00
Matthias Clasen
6a24e0c7c3 Add an introspection api test
Every beginning is small... just check one constant for now.
2021-05-05 19:18:32 -04:00
Matthias Clasen
e808fb424a testsuite: Stop requiring diff
Unify the many copies of diff_with_file in one source
file, and patch it to detect diff at runtime and fall
back to a simple strcmp if we don't have it. Make all
tests use this new testutils.c, and stop requiring
diff for building the tests.

This should let us allow to build on Windows with the
default value for -Dbuild-tests.
2021-01-21 23:57:24 -05:00
Chun-wei Fan
cf5a6a003a testsuite: Setup tests env by platform
Check whether we really have x11 and wayland enabled before we try to setup the
tests to use these respective GDK backends, and only attempt to setup tests
running with the Broadway backend if it has been enabled.

Also, add a setup for running tests with the GDK-Win32 backend on Windows, for
builds that target Windows.
2020-08-05 16:23:14 +08:00
Matthias Clasen
73477b5abf testsuite: Use Wayland by default
I'm tired of xwayland crashing and killing my shell.
2020-07-30 22:16:31 -04:00
Emmanuele Bassi
83eac3f33a a11y: Restart the accessibility test suite 2020-07-26 20:49:08 +01:00
Emmanuele Bassi
c63087a563 Remove ATK
To build a better world sometimes means having to tear the old one down.
        -- Alexander Pierce, "Captain America: The Winter Soldier"

ATK served us well for nearly 20 years, but the world has changed, and
GTK has changed with it. Now ATK is mostly a hindrance towards improving
the accessibility stack:

 - it maps to a very specific implementation, AT-SPI, which is Linux and
   Unix specific
 - it requires implementing the same functionality in three different
   layers of the stack: AT-SPI, ATK, and GTK
 - only GTK uses it; every other Linux and Unix toolkit and application
   talks to AT-SPI directly, including assistive technologies

Sadly, we cannot incrementally port GTK to a new accessibility stack;
since ATK insulates us entirely from the underlying implementation, we
cannot replace it piecemeal. Instead, we're going to remove everything
and then incrementally build on a clean slate:

 - add an "accessible" interface, implemented by GTK objects directly,
   which describe the accessible role and state changes for every UI
   element
 - add an "assistive technology context" to proxy a native accessibility
   API, and assign it to every widget
 - implement the AT context depending on the platform

For more information, see: https://gitlab.gnome.org/GNOME/gtk/-/issues/2833
2020-07-26 20:31:14 +01:00
Matthias Clasen
d31bb8b503 testsuite: Don't pass GDK_DEBUG for release builds
The debug env vars are ignored in release builds,
and may spew warnings about that fact that break
tests.
2020-06-28 13:42:02 -04:00
Matthias Clasen
a73457237d Skip broken a11y tests 2020-06-06 14:18:44 -04:00
Matthias Clasen
3dafdcbab0 testsuite: Remove GDK_DEBUG=misc
This snuck in by accident in recent ci setup
changes, and breaks the test runs by adding
stray output.
2020-05-18 16:38:11 -04:00
Matthias Clasen
509db01319 tests: Centralize our test setup a bit
Put all of the constant environment into the
toplevel meson.build file, to reduce repetition
and copy-paste errors.
2020-05-15 14:11:53 -04:00
Matthias Clasen
ff4552c842 testsuite: Add test setups for backends
Add test setups that set the GDK_BACKEND and
TEST_OUTPUT_SUBDIR environment variables.

This lets use run
meson test --setup x11 --suite reftest
meson test --setup wayland --suite reftest
and the output will be nicely separated.

We still need to do compositor / display server
setup from the outside.
2020-05-15 14:11:53 -04:00
Matthias Clasen
878f35515d Make the performance testcase more general
Reuse the performance test for layout and snapshot timings.
2020-01-22 00:59:04 -05:00
Emmanuele Bassi
48cc26246f Check for diff's availability
The tests suite calls `diff` in various places, which means we need to
check if it's installed.
2019-11-01 15:20:47 +00:00
Benjamin Otte
c564a349e5 testsuite: Make reftests work again
Well, the testrunner works again, the tests don't all succeed.

So exclude them from CI.
2019-03-29 09:56:52 +01:00
Matthias Clasen
4a22704d82 Fix test environments 2017-10-18 17:30:20 +02:00
Matthias Clasen
eb71cd195a testsuite: cosmetics
Put all tests in a suite, and order them in a nicer way.
This makes the test output look better.
2017-10-11 19:33:43 -05:00
Matthias Clasen
a4deb7e7b4 Reshuffle the css tests a bit 2017-10-10 00:18:21 -04:00
Matthias Clasen
8beb78149d Start reftests for gsk
Add a framework similar to what we have for gtk reftests
that lets us drop in a function to generate render nodes,
and compare reference output.
2017-10-02 00:21:56 -04:00
Emmanuele Bassi
0226163269 meson: Build the reftests suite
It does not pass, but at least we can check it.
2017-05-10 11:37:40 +01:00
Emmanuele Bassi
a54b6be606 build: Add common settings for tests
We are going to reuse them in the various parts of the test suite.
2017-05-03 15:10:58 +01:00
Tim-Philipp Müller
7cbf92505c meson: testsuites: tools: add meson build 2017-05-03 15:10:55 +01:00
Tim-Philipp Müller
b95a5678f4 meson: testsuite: reftests: add build defs, but needs more work
Not sure how these are supposed to work yet, we can specify
arguments for running the tests uninstalled as well with test(),
if that helps in any way.
2017-05-03 15:10:55 +01:00
Timm Bäder
409ed21594 meson: Sort input files by name 2017-05-03 15:10:52 +01:00
Timm Bäder
1e8a606403 meson: Add unit test files 2017-05-03 15:10:50 +01:00