0ee2627026
Add skiatest::Failure to keep track of data about a test failure.
Reporter::reportFailed and ::onReportFailed now take Failure as a
parameter. This allows the implementation to treat the failure as it
wishes. Provide a helper to format the failure the same as prior to
the change.
Update the macros for calling reportFailed (REPORTER_ASSERT etc) to
create a Failure object.
Convert a direct call to reportFailed to the macro ERRORF.
Write Failures to Json.
Sample output when running dm on the dummy test crrev.com/705723004:
{
"test_results" : {
"failures" : [
{
"condition" : "0 > 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 10,
"message" : ""
},
{
"condition" : "false",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 4,
"message" : ""
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 5,
"message" : "I can too count!"
},
{
"condition" : "",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 6,
"message" : "seven is 7"
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 14,
"message" : "I can too count!"
}
]
}
}
Report all of the failures from one test.
Previously, if one test had multiple failures, only one was reportered.
e.g:
Failures:
test Dummy: ../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
Now, we get all the messages:
Failures:
test Dummy: ../../tests/DummyTest.cpp:4 false
../../tests/DummyTest.cpp:5 I can too count!: 1 == 3
../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
(Note that we still state "3 failures" because 3 DM::Tasks failed.)
BUG=skia:3082
BUG=skia:2454
Review URL: https://codereview.chromium.org/694703005
|
||
|---|---|---|
| .. | ||
| DM.cpp | ||
| DMCpuGMTask.cpp | ||
| DMCpuGMTask.h | ||
| DMGpuGMTask.cpp | ||
| DMGpuGMTask.h | ||
| DMGpuSupport.h | ||
| DMJsonWriter.cpp | ||
| DMJsonWriter.h | ||
| DMPDFRasterizeTask.cpp | ||
| DMPDFRasterizeTask.h | ||
| DMPDFTask.cpp | ||
| DMPDFTask.h | ||
| DMPipeTask.cpp | ||
| DMPipeTask.h | ||
| DMQuiltTask.cpp | ||
| DMQuiltTask.h | ||
| DMReporter.cpp | ||
| DMReporter.h | ||
| DMSerializeTask.cpp | ||
| DMSerializeTask.h | ||
| DMSKPTask.cpp | ||
| DMSKPTask.h | ||
| DMTask.cpp | ||
| DMTask.h | ||
| DMTaskRunner.cpp | ||
| DMTaskRunner.h | ||
| DMTestTask.cpp | ||
| DMTestTask.h | ||
| DMUtil.cpp | ||
| DMUtil.h | ||
| DMWriteTask.cpp | ||
| DMWriteTask.h | ||
| README | ||
DM (Diamond Master, a.k.a Dungeon master, a.k.a GM 2).
DM is like GM, but multithreaded. It doesn't do everything GM does.
DM's design is based around Tasks and a TaskRunner.
A Task represents an independent unit of work that might fail. We make a task
for each GM/configuration pair we want to run. Tasks can kick off new tasks
themselves. For example, a CpuTask can kick off a ReplayTask to make sure
recording and playing back an SkPicture gives the same result as direct
rendering.
The TaskRunner runs all tasks on one of two threadpools, whose sizes are
configurable by --cpuThreads and --gpuThreads. Ideally we'd run these on a
single threadpool but it can swamp the GPU if we shove too much work into it at
once. --cpuThreads defaults to the number of cores on the machine.
--gpuThreads defaults to 1, but you may find 2 or 4 runs a little faster.
So the main flow of DM is:
for each GM:
for each configuration:
kick off a new task
< tasks run, maybe fail, and maybe kick off new tasks >
wait for all tasks to finish
report failures