skia2/modules/pathkit/tests/testReporter.js

129 lines
4.5 KiB
JavaScript
Raw Normal View History

[PathKit] Adding test infrastructure to support Gold output To get the gold images out of the browser tests, this adds testReporter.js and pathkit_aggregator.go. testReporter bundles up the output as a base64 encoded PNG and sends it over the local network to pathkit_aggregator. pathkit_aggregator will keep a list of test results reported in this way and write the PNGs to /OUT of the container (which is the swarming output directory). Finally, after all the tests are run, the helper script "test_pathkit.sh" makes a POST request that creates the JSON file that gold expects (following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md) pathkit_aggregator takes many command line arguments which control the keys that Gold needs in order to ingest and handle the data. Of note, this creates a new set (i.e. source_type) of gold images called "pathkit", which will distinguish it from "gm", "image", etc. There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg", (representing the 2 output types of PathKit). This CL doesn't quite handle SVG yet, as it needs a way to convert SVG to PNG in the browser and will be addressed in a follow up CL. A "standard" gm is sized at 600x600. This was arbitrarily picked. Note that the functions in testReporter.js return Promises based on the fetch requests to post the data. This eliminates the race condition between the /report_gold_data and /dump_json since running the karma tests won't return until all reports are done. Other changes of note: - Adds go to karma-chrome-tests container. - renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with the name of test_pathkit.py and make for easier grepping. - Increases the JS test timeout to 10s (up from 5) to hopefully avoid the flakes seen in the Debug Test. Bug: skia:8216 Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2 Reviewed-on: https://skia-review.googlesource.com/147042 Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-08-15 17:45:28 +00:00
const REPORT_URL = 'http://localhost:8081/report_gold_data'
// Set this to enforce that the gold server must be up.
// Typically used for debugging.
const fail_on_no_gold = false;
function reportCanvas(canvas, testname, outputType='canvas') {
[PathKit] Adding test infrastructure to support Gold output To get the gold images out of the browser tests, this adds testReporter.js and pathkit_aggregator.go. testReporter bundles up the output as a base64 encoded PNG and sends it over the local network to pathkit_aggregator. pathkit_aggregator will keep a list of test results reported in this way and write the PNGs to /OUT of the container (which is the swarming output directory). Finally, after all the tests are run, the helper script "test_pathkit.sh" makes a POST request that creates the JSON file that gold expects (following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md) pathkit_aggregator takes many command line arguments which control the keys that Gold needs in order to ingest and handle the data. Of note, this creates a new set (i.e. source_type) of gold images called "pathkit", which will distinguish it from "gm", "image", etc. There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg", (representing the 2 output types of PathKit). This CL doesn't quite handle SVG yet, as it needs a way to convert SVG to PNG in the browser and will be addressed in a follow up CL. A "standard" gm is sized at 600x600. This was arbitrarily picked. Note that the functions in testReporter.js return Promises based on the fetch requests to post the data. This eliminates the race condition between the /report_gold_data and /dump_json since running the karma tests won't return until all reports are done. Other changes of note: - Adds go to karma-chrome-tests container. - renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with the name of test_pathkit.py and make for easier grepping. - Increases the JS test timeout to 10s (up from 5) to hopefully avoid the flakes seen in the Debug Test. Bug: skia:8216 Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2 Reviewed-on: https://skia-review.googlesource.com/147042 Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-08-15 17:45:28 +00:00
let b64 = canvas.toDataURL('image/png');
return _report(b64, outputType, testname);
[PathKit] Adding test infrastructure to support Gold output To get the gold images out of the browser tests, this adds testReporter.js and pathkit_aggregator.go. testReporter bundles up the output as a base64 encoded PNG and sends it over the local network to pathkit_aggregator. pathkit_aggregator will keep a list of test results reported in this way and write the PNGs to /OUT of the container (which is the swarming output directory). Finally, after all the tests are run, the helper script "test_pathkit.sh" makes a POST request that creates the JSON file that gold expects (following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md) pathkit_aggregator takes many command line arguments which control the keys that Gold needs in order to ingest and handle the data. Of note, this creates a new set (i.e. source_type) of gold images called "pathkit", which will distinguish it from "gm", "image", etc. There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg", (representing the 2 output types of PathKit). This CL doesn't quite handle SVG yet, as it needs a way to convert SVG to PNG in the browser and will be addressed in a follow up CL. A "standard" gm is sized at 600x600. This was arbitrarily picked. Note that the functions in testReporter.js return Promises based on the fetch requests to post the data. This eliminates the race condition between the /report_gold_data and /dump_json since running the karma tests won't return until all reports are done. Other changes of note: - Adds go to karma-chrome-tests container. - renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with the name of test_pathkit.py and make for easier grepping. - Increases the JS test timeout to 10s (up from 5) to hopefully avoid the flakes seen in the Debug Test. Bug: skia:8216 Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2 Reviewed-on: https://skia-review.googlesource.com/147042 Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-08-15 17:45:28 +00:00
}
function reportSVG(svg, testname) {
// This converts an SVG to a base64 encoded PNG. It basically creates an
// <img> element that takes the inlined SVG and draws it on a canvas.
// The trick is we have to wait until the image is loaded, thus the Promise
// wrapping below.
let svgStr = svg.outerHTML;
let tempImg = document.createElement('img');
let tempCanvas = document.createElement('canvas');
let canvasCtx = tempCanvas.getContext('2d');
setCanvasSize(canvasCtx, svg.getAttribute('width'), svg.getAttribute('height'));
return new Promise(function(resolve, reject) {
tempImg.onload = () => {
canvasCtx.drawImage(tempImg, 0, 0);
let b64 = tempCanvas.toDataURL('image/png');
_report(b64, 'svg', testname).then(() => {
resolve();
[PathKit] Rework API to avoid extra copies unless explicitly called for. Breaking Changes: - All method calls that mutate a path now return the same JS path object to allow chaining (moveTo, lineTo, trim, op, simplify, etc). Pre-existing code likely will need to have some delete() methods removed because the path will be deleted multiple times. See chaining.js for this code (basically, we wrote our own binding code since the default code wasn't quite flexible enough) - GetCanvasFillType -> GetFillTypeString (Was in https://skia-review.googlesource.com/c/skia/+/147209) Since Canvas and SVG use the same strings, it seemed logical to make them share. - stroke() now takes a single object instead of 3 params. This object currently can have up to 4 params, cap, join, width, miter_limit. This object can be expanded on in future versions as more configuration options are added. As per custom with v0 software, we bump the minor version to 0.2.X to indicate breaking changes in a pre-release software package. Other changes of note: - Simple tests added for effects (see effects.specs.js) A follow up CL will handle the Gold (correctness tests) - Simple tests added for equals and copy constructors (from https://skia-review.googlesource.com/c/skia/+/147209) - Added transform() to allow for arbitrary matrix transforms - Added SimpleMatrix as a value_array, which means users can provide a 9 element array which will be converted to SimpleMatrix and then SkMatrix on the C++ side. - Renamed helpers_externs.js to externs.js and expanded it greatly. This was necessitated by the code written in chaining.js - Fixed a few bugs in previous tests (svg gold test race condition, uncaught exception in svg reporting) See also https://skia-review.googlesource.com/c/skia/+/147209 which allows .moveTo .lineTo, etc to chain on the C++ SkPath. Bug: skia:8216 Change-Id: I7450cd8b7b5377cf15c962b02d161677b62d7e15 Reviewed-on: https://skia-review.googlesource.com/147115 Reviewed-by: Mike Reed <reed@google.com>
2018-08-17 17:52:56 +00:00
}).catch((e) => reject(e));
[PathKit] Adding test infrastructure to support Gold output To get the gold images out of the browser tests, this adds testReporter.js and pathkit_aggregator.go. testReporter bundles up the output as a base64 encoded PNG and sends it over the local network to pathkit_aggregator. pathkit_aggregator will keep a list of test results reported in this way and write the PNGs to /OUT of the container (which is the swarming output directory). Finally, after all the tests are run, the helper script "test_pathkit.sh" makes a POST request that creates the JSON file that gold expects (following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md) pathkit_aggregator takes many command line arguments which control the keys that Gold needs in order to ingest and handle the data. Of note, this creates a new set (i.e. source_type) of gold images called "pathkit", which will distinguish it from "gm", "image", etc. There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg", (representing the 2 output types of PathKit). This CL doesn't quite handle SVG yet, as it needs a way to convert SVG to PNG in the browser and will be addressed in a follow up CL. A "standard" gm is sized at 600x600. This was arbitrarily picked. Note that the functions in testReporter.js return Promises based on the fetch requests to post the data. This eliminates the race condition between the /report_gold_data and /dump_json since running the karma tests won't return until all reports are done. Other changes of note: - Adds go to karma-chrome-tests container. - renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with the name of test_pathkit.py and make for easier grepping. - Increases the JS test timeout to 10s (up from 5) to hopefully avoid the flakes seen in the Debug Test. Bug: skia:8216 Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2 Reviewed-on: https://skia-review.googlesource.com/147042 Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-08-15 17:45:28 +00:00
};
tempImg.setAttribute('src', 'data:image/svg+xml;,' + svgStr);
});
}
// For tests that just do a simple path and return it as a string, wrap it in
// a proper svg and send it off. Supports fill (nofill means just stroke it).
// This uses the "standard" size of 600x600.
function reportSVGString(svgstr, testname, fillRule='nofill') {
let newPath = document.createElementNS('http://www.w3.org/2000/svg', 'path');
newPath.setAttribute('stroke', 'black');
if (fillRule !== 'nofill') {
newPath.setAttribute('fill', 'orange');
newPath.setAttribute('fill-rule', fillRule);
} else {
newPath.setAttribute('fill', 'rgba(255,255,255,0.0)');
}
newPath.setAttribute('d', svgstr);
let newSVG = document.createElementNS('http://www.w3.org/2000/svg', 'svg');
newSVG.appendChild(newPath);
// helps with the conversion to PNG.
newSVG.setAttribute('xmlns', 'http://www.w3.org/2000/svg');
newSVG.setAttribute('width', 600);
newSVG.setAttribute('height', 600);
return reportSVG(newSVG, testname);
}
// Reports a canvas and then an SVG of this path. Puts it on a standard size canvas.
function reportPath(path, testname, done) {
let canvas = document.createElement('canvas');
let canvasCtx = canvas.getContext('2d');
// Set canvas size and make it a bit bigger to zoom in on the lines
standardizedCanvasSize(canvasCtx);
canvasCtx.stroke(path.toPath2D());
let svgStr = path.toSVGString();
return reportCanvas(canvas, testname).then(() => {
reportSVGString(svgStr, testname).then(() => {
done();
}).catch(reportError(done));
}).catch(reportError(done));
}
// data is a base64 encoded png, outputType is the value that goes with the
// key 'config' when reporting.
[PathKit] Adding test infrastructure to support Gold output To get the gold images out of the browser tests, this adds testReporter.js and pathkit_aggregator.go. testReporter bundles up the output as a base64 encoded PNG and sends it over the local network to pathkit_aggregator. pathkit_aggregator will keep a list of test results reported in this way and write the PNGs to /OUT of the container (which is the swarming output directory). Finally, after all the tests are run, the helper script "test_pathkit.sh" makes a POST request that creates the JSON file that gold expects (following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md) pathkit_aggregator takes many command line arguments which control the keys that Gold needs in order to ingest and handle the data. Of note, this creates a new set (i.e. source_type) of gold images called "pathkit", which will distinguish it from "gm", "image", etc. There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg", (representing the 2 output types of PathKit). This CL doesn't quite handle SVG yet, as it needs a way to convert SVG to PNG in the browser and will be addressed in a follow up CL. A "standard" gm is sized at 600x600. This was arbitrarily picked. Note that the functions in testReporter.js return Promises based on the fetch requests to post the data. This eliminates the race condition between the /report_gold_data and /dump_json since running the karma tests won't return until all reports are done. Other changes of note: - Adds go to karma-chrome-tests container. - renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with the name of test_pathkit.py and make for easier grepping. - Increases the JS test timeout to 10s (up from 5) to hopefully avoid the flakes seen in the Debug Test. Bug: skia:8216 Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2 Reviewed-on: https://skia-review.googlesource.com/147042 Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-08-15 17:45:28 +00:00
function _report(data, outputType, testname) {
return fetch(REPORT_URL, {
method: 'POST',
mode: 'no-cors',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
'output_type': outputType,
'data': data,
'test_name': testname,
})
}).then(() => console.log(`Successfully reported ${testname} to gold aggregator`));
}
function reportError(done) {
return (e) => {
console.log("Error with fetching. Likely could not connect to aggegator server", e.message);
if (fail_on_no_gold) {
expect(e).toBeUndefined();
}
done();
};
}
function setCanvasSize(ctx, width, height) {
ctx.canvas.width = width;
ctx.canvas.height = height;
}
function standardizedCanvasSize(ctx) {
setCanvasSize(ctx, 600, 600);
}
// A wrapper to catch and print a stacktrace to the logs.
// Exceptions normally shows up in the browser console,
// but not in the logs that appear on the bots AND a thrown
// exception will normally cause a test to time out.
// This wrapper mitigates both those pain points.
function catchException(done, fn) {
return () => {
try {
fn()
} catch (e) {
console.log('Failed with the following error', e);
expect(e).toBeFalsy();
debugger;
done();
}
// We don't call done with finally because
// that would make the break the asynchronous nature
// of fn().
}
[PathKit] Adding test infrastructure to support Gold output To get the gold images out of the browser tests, this adds testReporter.js and pathkit_aggregator.go. testReporter bundles up the output as a base64 encoded PNG and sends it over the local network to pathkit_aggregator. pathkit_aggregator will keep a list of test results reported in this way and write the PNGs to /OUT of the container (which is the swarming output directory). Finally, after all the tests are run, the helper script "test_pathkit.sh" makes a POST request that creates the JSON file that gold expects (following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md) pathkit_aggregator takes many command line arguments which control the keys that Gold needs in order to ingest and handle the data. Of note, this creates a new set (i.e. source_type) of gold images called "pathkit", which will distinguish it from "gm", "image", etc. There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg", (representing the 2 output types of PathKit). This CL doesn't quite handle SVG yet, as it needs a way to convert SVG to PNG in the browser and will be addressed in a follow up CL. A "standard" gm is sized at 600x600. This was arbitrarily picked. Note that the functions in testReporter.js return Promises based on the fetch requests to post the data. This eliminates the race condition between the /report_gold_data and /dump_json since running the karma tests won't return until all reports are done. Other changes of note: - Adds go to karma-chrome-tests container. - renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with the name of test_pathkit.py and make for easier grepping. - Increases the JS test timeout to 10s (up from 5) to hopefully avoid the flakes seen in the Debug Test. Bug: skia:8216 Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2 Reviewed-on: https://skia-review.googlesource.com/147042 Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-08-15 17:45:28 +00:00
}