AuroraMiddleware/gtk2
1
0
Fork 0
forked from AuroraMiddleware/gtk
Code Issues Pull Requests Projects Releases Wiki Activity
fc47b913b1
gtk2/.gitlab-ci/run-tests.sh

108 lines
2.8 KiB
Bash
Raw Normal View History

ci: Refactor the CI pipeline We should have a single job for building both static and shared versions of GTK. On the other hand, having a separate job for the release build would be a plus. Additionally, we shouldn't use an opaque script to build GTK; the only step in the process that requires hand-holding is running the tests suites under Xvfb, and having the build options visible from the YAML file gives us a better idea of what kind of build we're running.
2020-01-29 09:35:10 +00:00
#!/bin/bash
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
set +x
set +e
ci: Refactor the CI pipeline We should have a single job for building both static and shared versions of GTK. On the other hand, having a separate job for the release build would be a plus. Additionally, we shouldn't use an opaque script to build GTK; the only step in the process that requires hand-holding is running the tests suites under Xvfb, and having the build options visible from the YAML file gives us a better idea of what kind of build we're running.
2020-01-29 09:35:10 +00:00
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
srcdir=$( pwd )
ci: Refactor the CI pipeline We should have a single job for building both static and shared versions of GTK. On the other hand, having a separate job for the release build would be a plus. Additionally, we shouldn't use an opaque script to build GTK; the only step in the process that requires hand-holding is running the tests suites under Xvfb, and having the build options visible from the YAML file gives us a better idea of what kind of build we're running.
2020-01-29 09:35:10 +00:00
builddir=$1
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
backend=$2
ci: Refactor the CI pipeline We should have a single job for building both static and shared versions of GTK. On the other hand, having a separate job for the release build would be a plus. Additionally, we shouldn't use an opaque script to build GTK; the only step in the process that requires hand-holding is running the tests suites under Xvfb, and having the build options visible from the YAML file gives us a better idea of what kind of build we're running.
2020-01-29 09:35:10 +00:00
ci: Set LSAN_OPTIONS for tests When running the testsuite with the address sanitizer, many of our dependencies cause it to report cause it to report memory leaks, causing tests to fail. Therefore, point the leak sanitizer at a list of suppressions. The list is kept in the lsan.supp file in git.
2020-07-09 02:19:32 +00:00
# Ignore memory leaks lower in dependencies
ci: Don't print lsan suppressions Printing these out breaks TAP output, and we really aren't interested in them.
2021-01-22 18:01:09 +00:00
export LSAN_OPTIONS=suppressions=$srcdir/lsan.supp:print_suppressions=0
Force malloc of g_slices in asan build Otherwise leaks could be missed.
2021-01-22 05:13:13 +00:00
export G_SLICE=always-malloc
ci: Set LSAN_OPTIONS for tests When running the testsuite with the address sanitizer, many of our dependencies cause it to report cause it to report memory leaks, causing tests to fail. Therefore, point the leak sanitizer at a list of suppressions. The list is kept in the lsan.supp file in git.
2020-07-09 02:19:32 +00:00
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
case "${backend}" in
x11)
ci: Pass -norest to Xvfb This may help with the sporadic "No display" failures we are seeing when running tests under X11 in ci.
2020-08-04 15:59:32 +00:00
xvfb-run -a -s "-screen 0 1024x768x24 -noreset" \
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
meson test -C ${builddir} \
ci: Use the timeout multiplier from the environment When running our test suite on our CI infrastructure.
2020-07-10 15:08:14 +00:00
--timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
--print-errorlogs \
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
--setup=${backend} \
ci: Add missing newline escape Otherwise we're going to run the accessibility test suite, and it's going to fail.
2020-01-29 11:31:00 +00:00
--suite=gtk \
ci: Don't run opengl renderer tests on Broadway The test setup mechanism in meson is not flexible enough to let us run different suites depending on setup, so just pass in explicitly which suites we want to skip, depending on the backend.
2020-05-16 14:12:47 +00:00
--no-suite=gsk-compare-broadway
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
# Store the exit code for the CI run, but always
# generate the reports
exit_code=$?
;;
wayland)
export XDG_RUNTIME_DIR="$(mktemp -p $(pwd) -d xdg-runtime-XXXXXX)"
tests: Disable idle locking in headless weston In my local testing i was burned by the default 300sec value here. I'm not sure if we actually hit this in the CI, but lets not take any chances.
2020-05-28 13:15:04 +00:00
weston --backend=headless-backend.so --socket=wayland-5 --idle-time=0 &
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
compositor=$!
export WAYLAND_DISPLAY=wayland-5
meson test -C ${builddir} \
ci: Use the timeout multiplier from the environment When running our test suite on our CI infrastructure.
2020-07-10 15:08:14 +00:00
--timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
--print-errorlogs \
--setup=${backend} \
--suite=gtk \
ci: Don't run opengl renderer tests on Broadway The test setup mechanism in meson is not flexible enough to let us run different suites depending on setup, so just pass in explicitly which suites we want to skip, depending on the backend.
2020-05-16 14:12:47 +00:00
--no-suite=gsk-compare-broadway
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
exit_code=$?
kill ${compositor}
;;
testsuite: Add run for GLES This is supposed to test the most fallback GL stuff, so we might want to set even more env vars here. Also enable the run for the Fedora builder in CI.
2021-10-08 16:38:51 +00:00
waylandgles)
export XDG_RUNTIME_DIR="$(mktemp -p $(pwd) -d xdg-runtime-XXXXXX)"
weston --backend=headless-backend.so --socket=wayland-6 --idle-time=0 &
compositor=$!
export WAYLAND_DISPLAY=wayland-6
meson test -C ${builddir} \
--timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \
--print-errorlogs \
--setup=${backend} \
--suite=gtk \
--no-suite=gsk-compare-broadway
exit_code=$?
kill ${compositor}
;;
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
broadway)
export XDG_RUNTIME_DIR="$(mktemp -p $(pwd) -d xdg-runtime-XXXXXX)"
${builddir}/gdk/broadway/gtk4-broadwayd :5 &
server=$!
export BROADWAY_DISPLAY=:5
meson test -C ${builddir} \
ci: Use the timeout multiplier from the environment When running our test suite on our CI infrastructure.
2020-07-10 15:08:14 +00:00
--timeout-multiplier "${MESON_TEST_TIMEOUT_MULTIPLIER}" \
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
--print-errorlogs \
--setup=${backend} \
--suite=gtk \
ci: Don't run opengl renderer tests on Broadway The test setup mechanism in meson is not flexible enough to let us run different suites depending on setup, so just pass in explicitly which suites we want to skip, depending on the backend.
2020-05-16 14:12:47 +00:00
--no-suite=gsk-compare-opengl
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
ci: Ignore Broadway failures These need some more work, so ignore them for now, to get the initial multi-backend testsuite into production.
2020-05-15 19:23:22 +00:00
# don't let Broadway failures fail the run, for now
exit_code=0
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
kill ${server}
;;
testsuite: Add run for GLES This is supposed to test the most fallback GL stuff, so we might want to set even more env vars here. Also enable the run for the Fedora builder in CI.
2021-10-08 16:38:51 +00:00
*)
echo "Failed to add ${backend} to .gitlab-ci/run-tests.sh"
exit 1
;;
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
esac
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
ci: Fix linking of the CI assets in the report The report is relative to the build directory, and so are the generated assets.
2020-02-12 18:03:48 +00:00
cd ${builddir}
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
$srcdir/.gitlab-ci/meson-junit-report.py \
--project-name=gtk \
ci: Add the backend to the reports We want the test names in the junit xml to be unique across all the tests in a job, so we need to include the backend in the test name. And we also want to see the used backend in the html report.
2020-05-15 23:49:18 +00:00
--backend=${backend} \
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
--job-id="${CI_JOB_NAME}" \
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
--output=report-${backend}.xml \
meson-logs/testlog-${backend}.json
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
$srcdir/.gitlab-ci/meson-html-report.py \
--project-name=gtk \
ci: Add the backend to the reports We want the test names in the junit xml to be unique across all the tests in a job, so we need to include the backend in the test name. And we also want to see the used backend in the html report.
2020-05-15 23:49:18 +00:00
--backend=${backend} \
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
--job-id="${CI_JOB_NAME}" \
ci: Run tests with different backends Run the testsuite under x11, wayland and broadway in the fedora-x86_64 job. This requires us to use the v17 image which includes weston.
2020-05-15 13:44:24 +00:00
--reftest-output-dir="testsuite/reftests/output/${backend}" \
--output=report-${backend}.html \
meson-logs/testlog-${backend}.json
ci: Generate the report for the release builds We are going to need to hide the report generation into the test runner script, as we want the job to produce the reports even in case of failure, instead of bailing out immediately.
2020-01-29 13:36:23 +00:00
exit $exit_code
Reference in New Issue Copy Permalink