Most of this is (obviously) not necessary to do, but once
I started, I figured I'd just get it all. Tools (nanobench,
DM, skiaserve), all GMs, benches, and unit tests, plus support
code (command line parsing and config stuff).
This is almost entirely mechanical.
Bug: skia:
Change-Id: I209500f8df8c5bd43f8298ff26440d1c4d7425fb
Reviewed-on: https://skia-review.googlesource.com/131153
Reviewed-by: Mike Klein <mtklein@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Remove REPORTER_ASSERT_MESSAGE.
Change-Id: I6d00715901159c93e22d182fe24aac92b5fdbcf4
Reviewed-on: https://skia-review.googlesource.com/100361
Commit-Queue: Brian Salomon <bsalomon@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
This is to alleviate problems due to the command buffer getting bent out of shape when the current
OpenGL context is switched out from under it (because we ran a test with a native GL context). This,
however is not a full solution. More changes will be required to ensure that after running each
command buffer or native test we bind the null context. This does allow us to take a step in that
direction without breaking anything too badly. Moreover, there is no real benefit to reusing a
GrContextFactory.
Modifies DEF_GPUTEST to take GrContextOptions rather than a factory to use. Tests were already using
their own factories anyway.
In tests that use GrContextFactory the factory instance is moved to the inner loop.
Modifies gpucts and skia_test to not use persistent GrContextFactories.
Change-Id: Ie7a36793545c775f2f30653ead6fec93a3d22717
Reviewed-on: https://skia-review.googlesource.com/71861
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Salomon <bsalomon@google.com>
Change-Id: I562d438bd65e9fd900cfc6831f971b4af25c8ae6
Reviewed-on: https://skia-review.googlesource.com/26361
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@chromium.org>
BUG=skia:
Change-Id: Ie7d4fac3024b361a281f456fec2b3a837e2bfe43
Reviewed-on: https://skia-review.googlesource.com/6881
Commit-Queue: Mike Reed <reed@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
Currently, just inject the Ganesh context type when running unit tests.
Obviously, we can use this to supply other contextual information around
tests that do many variations of configs, formats, etc...
BUG=skia:
Change-Id: Iab96632a92ec632e4d132bbcc17a91a8dd251e78
Reviewed-on: https://skia-review.googlesource.com/5565
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Update the ANGLE test GL context, GrContextFactory, and config parsing to allow explicit control of ANGLE front/backend.
This will allow us to explicitly test ES2 vs ES3 interfaces to ANGLE as well as D3D9, D3D11, and OpenGL backends.
Also makes the angle api types valid in all builds (but will just fail when SK_ANGLE=1 or not on windows for the d3d backends).
BUG=skia:5804
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2381033002
CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Win-MSVC-ShuttleC-GPU-GTX960-x86_64-Debug-ANGLE-Trybot
Review-Url: https://codereview.chromium.org/2381033002
The following are currently unused in Android, Google3, Chromium, and Mozilla:
- SkEvent
- SkTime::GetMSecs
- SK_TIME_FACTOR (also unused in Skia)
- SkAutoTime
I left uses of SkMSec more-or-less intact for SkEvent, SkAnimator, and SkInterpolator. SkInterpolator is used in Chromium, so I did not want to change the API. The views/ and animator/ code is crufty, so it didn't seem worthwhile to refactor it. Instead, I added SkEvent::GetMSecsSinceStartup, which is likely to be adequate for use in SampleApp.
I also left SkMSec where it is used to measure a duration rather than a timestamp. With the exception of SkMovie, which is used in Android, all of the uses appear to measure the execution time of a piece of code, which I would hope does not exceed 2^31 milliseconds.
Added skiatest::Timer to support a common idiom in tests where we want to measure the wallclock time in integer milliseconds. (Not used in tests/PathOpsSkpClipTest.cpp because it redefines things in Test.h.)
Removed tabs in tests/StrokerTest.cpp.
BUG=skia:4632
GOLD_TRYBOT_URL= https://gold.skia.org/search2?unt=true&query=source_type%3Dgm&master=false&issue=1811613004
Review URL: https://codereview.chromium.org/1811613004
Add support for feeding the tests with contexts directly to the unit
test framework.
This fixes the problem where tests are more complex than needed just in
order to run the test code with multiple backends.
Also makes it possible to change the logic how contexts are
created. Instead of direct numbering, the different testable contexts
may be generated from filtered cross-product of context options. For
example: currently NVPR is a type of context. However, it could be also
an on/off feature of any context. In order to test this kind of context,
the enumeration can not be just of context type. It's simpler
to move the enumeration out of the tests.
A test targeting both normal and GPU backends would look like:
static void test_obj_behavior(skiatest::Reporter* reporter,
SkObj* obj, [other params] ) {
... test with obj and param ..
}
DEF_TEST(ObjBehavior, reporter) {
for (auto& object : generate_object) {
for (auto& other_param : generate_other_variant) {
test_obj_behavior(reporter, object, other_param);
}
}
}
#if SK_SUPPORT_GPU
DEF_GPUTEST_FOR_ALL_CONTEXTS(ObjBehavior_Gpu, reporter, context) {
for (auto& object : generate_gpu_object) {
for (auto& other_param : generate_other_variant) {
test_obj_behavior(reporter, object, other_param);
}
}
}
#endif
Uses the feature in SurfaceTests as an example.
Moves SkSurface -related tests from ImageTest to SurfaceTest.
BUG=skia:2992
Review URL: https://codereview.chromium.org/1446453003
When possible use run-time checks (via SkDocument::CreatePDF)
When PDF is disabled, do not compile tests/PDF*.cpp
Review URL: https://codereview.chromium.org/1278403006
skiatest::Test class is now a simple struct. Some
functionalty, such as counting errors or timing is now
handled elsewhere.
skiatest:Reporter is now a simpler abstract class. The two
implementations handle test errors.
DM and pathops_unittest updated.
Review URL: https://codereview.chromium.org/830513004
This fixes every case where virtual and SK_OVERRIDE were on the same line,
which should be the bulk of cases. We'll have to manually clean up the rest
over time unless I level up in regexes.
for f in (find . -type f); perl -p -i -e 's/virtual (.*)SK_OVERRIDE/\1SK_OVERRIDE/g' $f; end
BUG=skia:
Review URL: https://codereview.chromium.org/806653007
Add skiatest::Failure to keep track of data about a test failure.
Reporter::reportFailed and ::onReportFailed now take Failure as a
parameter. This allows the implementation to treat the failure as it
wishes. Provide a helper to format the failure the same as prior to
the change.
Update the macros for calling reportFailed (REPORTER_ASSERT etc) to
create a Failure object.
Convert a direct call to reportFailed to the macro ERRORF.
Write Failures to Json.
Sample output when running dm on the dummy test crrev.com/705723004:
{
"test_results" : {
"failures" : [
{
"condition" : "0 > 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 10,
"message" : ""
},
{
"condition" : "false",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 4,
"message" : ""
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 5,
"message" : "I can too count!"
},
{
"condition" : "",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 6,
"message" : "seven is 7"
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 14,
"message" : "I can too count!"
}
]
}
}
Report all of the failures from one test.
Previously, if one test had multiple failures, only one was reportered.
e.g:
Failures:
test Dummy: ../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
Now, we get all the messages:
Failures:
test Dummy: ../../tests/DummyTest.cpp:4 false
../../tests/DummyTest.cpp:5 I can too count!: 1 == 3
../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
(Note that we still state "3 failures" because 3 DM::Tasks failed.)
BUG=skia:3082
BUG=skia:2454
Review URL: https://codereview.chromium.org/694703005
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
Reason for revert:
Leaks, leaks, leaks.
Original issue's description:
> SkThreadPool ~~> SkTaskGroup
>
> SkTaskGroup is like SkThreadPool except the threads stay in
> one global pool. Each SkTaskGroup itself is tiny (4 bytes)
> and its wait() method applies only to tasks add()ed to that
> instance, not the whole thread pool.
>
> This means we don't need to bring up new thread pools when
> tests themselves want to use multithreading (e.g. pathops,
> quilt). We just create a new SkTaskGroup and wait for that
> to complete. This should be more efficient, and allow us
> to expand where we use threads to really latency sensitive
> places. E.g. we can probably now use these in nanobench
> for CPU .skp rendering.
>
> Now that all threads are sharing the same pool, I think we
> can remove most of the custom mechanism pathops tests use
> to control threading. They'll just ride on the global pool
> with all other tests now.
>
> This (temporarily?) removes the GPU multithreading feature
> from DM, which we don't use.
>
> On my desktop, DM runs a little faster (57s -> 55s) in
> Debug, and a lot faster in Release (36s -> 24s). The bots
> show speedups of similar proportions, cutting more than a
> minute off the N4/Release and Win7/Debug runtimes.
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, reed@google.com, mtklein@chromium.orgTBR=bsalomon@google.com, bungeman@google.com, caryclark@google.com, mtklein@chromium.org, reed@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/533393002
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
That way when declaring a test with DEF_TEST() macro, you don't have to
uniquify the test name because it might colide with the class it is
being testing.
For example, if you are testing SkBase64 and do:
DEF_TEST(SkBase64, reporter) {
}
That will generate an error because the macro will declare a function
named SkBase64 which colides with the type SkBase64.
By adding Test to the function name we avoid this problem.
Fixed the entries found with the following command line:
$ git grep "Test, r" | grep DEF
BUG=None
TEST=make tests && out/Debug/tests
R=mtklein@google.com
Author: tfarina@chromium.org
Review URL: https://codereview.chromium.org/345753007
Reason for revert:
Some benchmarks are written in a way that makes this change unsafe (e.g. const char* resPath = GetResourcePath().c_str(); in SkipZeroesBench) and Valgrind and ASAN caught that. We can try again after a more careful cleanup of GetResourcePath().
Original issue's description:
> Clean up resourcePath code.
>
> 1) Make the implementation of SetResourcePath/GetResourcePath of GM and SkBenchmark match with the one in Test.
> 2) Make gResourcePath a static pointer to const char and move it inside the classes.
>
> BUG=None
> TEST=make tests && out/Debug/tests
> make gm && out/Debug/gm
> make bench && out/Debug/bench
> R=mtklein@google.com
>
> Committed: https://skia.googlesource.com/skia/+/52e4f413ffe2d281f9e90ff2147db08083ffcba7R=tfarina@chromium.orgTBR=tfarina@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=None
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/320733002
1) Make the implementation of SetResourcePath/GetResourcePath of GM and SkBenchmark match with the one in Test.
2) Make gResourcePath a static pointer to const char and move it inside the classes.
BUG=None
TEST=make tests && out/Debug/tests
make gm && out/Debug/gm
make bench && out/Debug/bench
R=mtklein@google.com
Author: tfarina@chromium.org
Review URL: https://codereview.chromium.org/319473003
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
Committed: http://code.google.com/p/skia/source/detail?r=13586R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13592 2bbb7eff-a529-9590-31e7-b0007b416f81
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13586 2bbb7eff-a529-9590-31e7-b0007b416f81
