meson seems somewhat weak when it comes to handling
test output. We need to get the output from different
test runs into different locations, and the only
way to communicate from a test setup with the actual
test code seems the environment, so use that.
Make all tests that produce output in files respect
a TEST_OUTPUT_SUBDIR environment variable which specifies
the name of a subdirectory to use. This is combined
with the existing --output argument, which specifies
a per-test location.
Affected tests are reftests, css performance tests
and gsk compare tests.
In the hope of making ci-only failures less of a black hole,
add a backtrace to the messsage for criticals.
This could eventually go into GLib (pass backtrace symbols along
as a log field for criticals), but for now this will do.
The nice thing about that is that we can then log messages about the
errors to the log.
And then we can read the logs of the CI machinery and actually know
what's going on.
With autotools the schemas were compiled into each test suite directory
and the tests set GSETTINGS_SCHEMA_DIR to the test build directory.
With meson's gnome.compile_schemas() we can not define a target directory
so just make sure it is built in the gtk directory and set GSETTINGS_SCHEMA_DIR
to the gtk build directory when running the tests.
This makes the gtk+:gtk suite pass when no gtk is installed on the system.
gtk-reftest already had an --output=DIR option to tell it where
to save all the resulting images. Now you can combine this with
the --compare-with=DIR option in a second run to make gtk-reftest
compare the .out.png files from the first run with the .out.png
files of the current run, instead of producing .ref.png files.
The intended use for this is to verify that changes do not affect
the generated output.
Make gtk-reftest consult the REFTEST_MODULE_DIR environment
variable to find out where to look for modules, and fix the
libtool hack to construct the .libs subdirectory correctly.
This adds an inhibit api that code from the reftest module
can use to delay the taking of the snapshot. Also refactor
the code in gtk-reftest to use the inhibit mechanism for
its own delaying of the snapshot until after the first
expose.
When connecting signal names, gtk-reftest now allows you to use a colon
in the signal handler name like so:
module:function_name
where module is a module loaded from the same directory (or the .libs
subdirectory for compatibility with uninstalled libtool) as the running
test and the function is resolved in that module. Of course, normal
function names work as before.
