The leak sanitizer causes on average 3-4 tests
to segfault during a testsuite run. Disable it
for now to see if we can get a successful
testsuite run with asan alone.
Run our testsuite under asan. We do this in a
separate build, since we need to turn off
introspection to make the build succeed.
As Michael Catanzaro pointed out, this requires a
privileged runner in order to use kernel apis.
When running the testsuite with the address sanitizer,
many of our dependencies cause it to report cause it
to report memory leaks, causing tests to fail.
Therefore, point the leak sanitizer at a list of
suppressions. The list is kept in the lsan.supp
file in git.
Add libasan and libubsan to fedora-base:v20,
and build fedora:20 on top of it.
This is so we can build and run the tests with
the address and undefined behavior sanitizers.
Split the fedora image into fedora-base (containing just
the packages) and fedora (adding the user setup), and add
a fedora-docs image that is adding pandoc on top of the
fedora-base image.
The test setup mechanism in meson is not flexible
enough to let us run different suites depending
on setup, so just pass in explicitly which suites
we want to skip, depending on the backend.
We want the test names in the junit xml to be
unique across all the tests in a job, so we need
to include the backend in the test name.
And we also want to see the used backend in
the html report.
They use powershell instead of cmd.exe, use the tag win32-ps instead of win32,
and run Windows 2016 instead of 2012r2.
The old runners will be switched off in the comming weeks.
We are going to need to hide the report generation into the test runner
script, as we want the job to produce the reports even in case of
failure, instead of bailing out immediately.
We should have a single job for building both static and shared
versions of GTK. On the other hand, having a separate job for the
release build would be a plus.
Additionally, we shouldn't use an opaque script to build GTK; the only
step in the process that requires hand-holding is running the tests
suites under Xvfb, and having the build options visible from the YAML
file gives us a better idea of what kind of build we're running.
We should show all the possible result sections:
- passed
- skipped
- expected failures
- failures
- timed out
Even if we consider the first three to be successes.
My web design skills are terrible, but it's better than nothing; I'm
sure this will lead to somebody filing a merge request to make the
test report look a lot better.
While we're at it, let's include the reftest images inside the report
itself, so we don't have to hunt them down.
The JUnit cover report is useful, but only up to a point; for instance,
it's not used unless it's part of a merge request. This means you don't
get a report if you're pushing to a branch that does not have an MR open.
With a simple Python script and some minimal templating, we can generate
an HTML report from the "I Can't Believe it's not JSON™" log that Meson
produces, and keep it as a CI artifact.
Since we're embedding text coming from the tests into the report, we
should specify an encoding for both the source JSON file and the target
XML file when opening them.
GitLab's CI will bail out at the first failure, which means the
JSON-to-JUnit conversion script won't run unless it's part of the same
script that we run for building an testing.
A recent dependency change in MSYS2 made it pull in vulkan, which made
meson think it's available but it somehow links against the system vulkan dll
instead.
Disable vulkan for now.
There where some problems (??) with ccache not detecting changes during meson
checks. Setting CCACHE_DISABLE during the meson execution makes ccache not use
the cache and pass things directly to the compiler.
The Fedora base image we use for our CI doesn't always keep Meson up to
date with our requirements, so it's better if we just install Meson via
Python's pip.
Turns out that GCC errors out when building the GLib test suite, as it
now checks for overflows in allocator functions, and we're testing for
those.
This would not be an issue for GTK, but since we're building GLib as a
subproject, we get failures for those as well.
Until we can find out how to disable errors for subprojects, or fix the
GLib test suite not to trip up warnings in GCC, we're going to live
without compiler warnings treated as errors for a while.
xvfb doesn't like C.utf8 and returns XLocaleNotSupported.
While (afaik) C.utf8 and C.UTF.8 should be the same thing, and the former
is returned by locale -a on Fedora, switch to C.UTF-8 to make xvfb happy.
This makes gtk+:gdk tests pass.
The GitLab cache is kept across jobs, whether they succeeded or not:
this means that if a compiler check fails during the Meson
configuration, the small compiler program gets cached and restored the
next time the job is run, thus failing again.
