Add support for feeding the tests with contexts directly to the unit
test framework.
This fixes the problem where tests are more complex than needed just in
order to run the test code with multiple backends.
Also makes it possible to change the logic how contexts are
created. Instead of direct numbering, the different testable contexts
may be generated from filtered cross-product of context options. For
example: currently NVPR is a type of context. However, it could be also
an on/off feature of any context. In order to test this kind of context,
the enumeration can not be just of context type. It's simpler
to move the enumeration out of the tests.
A test targeting both normal and GPU backends would look like:
static void test_obj_behavior(skiatest::Reporter* reporter,
SkObj* obj, [other params] ) {
... test with obj and param ..
}
DEF_TEST(ObjBehavior, reporter) {
for (auto& object : generate_object) {
for (auto& other_param : generate_other_variant) {
test_obj_behavior(reporter, object, other_param);
}
}
}
#if SK_SUPPORT_GPU
DEF_GPUTEST_FOR_ALL_CONTEXTS(ObjBehavior_Gpu, reporter, context) {
for (auto& object : generate_gpu_object) {
for (auto& other_param : generate_other_variant) {
test_obj_behavior(reporter, object, other_param);
}
}
}
#endif
Uses the feature in SurfaceTests as an example.
Moves SkSurface -related tests from ImageTest to SurfaceTest.
BUG=skia:2992
Review URL: https://codereview.chromium.org/1446453003
When possible use run-time checks (via SkDocument::CreatePDF)
When PDF is disabled, do not compile tests/PDF*.cpp
Review URL: https://codereview.chromium.org/1278403006
skiatest::Test class is now a simple struct. Some
functionalty, such as counting errors or timing is now
handled elsewhere.
skiatest:Reporter is now a simpler abstract class. The two
implementations handle test errors.
DM and pathops_unittest updated.
Review URL: https://codereview.chromium.org/830513004
This fixes every case where virtual and SK_OVERRIDE were on the same line,
which should be the bulk of cases. We'll have to manually clean up the rest
over time unless I level up in regexes.
for f in (find . -type f); perl -p -i -e 's/virtual (.*)SK_OVERRIDE/\1SK_OVERRIDE/g' $f; end
BUG=skia:
Review URL: https://codereview.chromium.org/806653007
Add skiatest::Failure to keep track of data about a test failure.
Reporter::reportFailed and ::onReportFailed now take Failure as a
parameter. This allows the implementation to treat the failure as it
wishes. Provide a helper to format the failure the same as prior to
the change.
Update the macros for calling reportFailed (REPORTER_ASSERT etc) to
create a Failure object.
Convert a direct call to reportFailed to the macro ERRORF.
Write Failures to Json.
Sample output when running dm on the dummy test crrev.com/705723004:
{
"test_results" : {
"failures" : [
{
"condition" : "0 > 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 10,
"message" : ""
},
{
"condition" : "false",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 4,
"message" : ""
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 5,
"message" : "I can too count!"
},
{
"condition" : "",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 6,
"message" : "seven is 7"
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 14,
"message" : "I can too count!"
}
]
}
}
Report all of the failures from one test.
Previously, if one test had multiple failures, only one was reportered.
e.g:
Failures:
test Dummy: ../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
Now, we get all the messages:
Failures:
test Dummy: ../../tests/DummyTest.cpp:4 false
../../tests/DummyTest.cpp:5 I can too count!: 1 == 3
../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
(Note that we still state "3 failures" because 3 DM::Tasks failed.)
BUG=skia:3082
BUG=skia:2454
Review URL: https://codereview.chromium.org/694703005
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
Reason for revert:
Leaks, leaks, leaks.
Original issue's description:
> SkThreadPool ~~> SkTaskGroup
>
> SkTaskGroup is like SkThreadPool except the threads stay in
> one global pool. Each SkTaskGroup itself is tiny (4 bytes)
> and its wait() method applies only to tasks add()ed to that
> instance, not the whole thread pool.
>
> This means we don't need to bring up new thread pools when
> tests themselves want to use multithreading (e.g. pathops,
> quilt). We just create a new SkTaskGroup and wait for that
> to complete. This should be more efficient, and allow us
> to expand where we use threads to really latency sensitive
> places. E.g. we can probably now use these in nanobench
> for CPU .skp rendering.
>
> Now that all threads are sharing the same pool, I think we
> can remove most of the custom mechanism pathops tests use
> to control threading. They'll just ride on the global pool
> with all other tests now.
>
> This (temporarily?) removes the GPU multithreading feature
> from DM, which we don't use.
>
> On my desktop, DM runs a little faster (57s -> 55s) in
> Debug, and a lot faster in Release (36s -> 24s). The bots
> show speedups of similar proportions, cutting more than a
> minute off the N4/Release and Win7/Debug runtimes.
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, reed@google.com, mtklein@chromium.orgTBR=bsalomon@google.com, bungeman@google.com, caryclark@google.com, mtklein@chromium.org, reed@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/533393002
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
That way when declaring a test with DEF_TEST() macro, you don't have to
uniquify the test name because it might colide with the class it is
being testing.
For example, if you are testing SkBase64 and do:
DEF_TEST(SkBase64, reporter) {
}
That will generate an error because the macro will declare a function
named SkBase64 which colides with the type SkBase64.
By adding Test to the function name we avoid this problem.
Fixed the entries found with the following command line:
$ git grep "Test, r" | grep DEF
BUG=None
TEST=make tests && out/Debug/tests
R=mtklein@google.com
Author: tfarina@chromium.org
Review URL: https://codereview.chromium.org/345753007
Reason for revert:
Some benchmarks are written in a way that makes this change unsafe (e.g. const char* resPath = GetResourcePath().c_str(); in SkipZeroesBench) and Valgrind and ASAN caught that. We can try again after a more careful cleanup of GetResourcePath().
Original issue's description:
> Clean up resourcePath code.
>
> 1) Make the implementation of SetResourcePath/GetResourcePath of GM and SkBenchmark match with the one in Test.
> 2) Make gResourcePath a static pointer to const char and move it inside the classes.
>
> BUG=None
> TEST=make tests && out/Debug/tests
> make gm && out/Debug/gm
> make bench && out/Debug/bench
> R=mtklein@google.com
>
> Committed: https://skia.googlesource.com/skia/+/52e4f413ffe2d281f9e90ff2147db08083ffcba7R=tfarina@chromium.orgTBR=tfarina@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=None
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/320733002
1) Make the implementation of SetResourcePath/GetResourcePath of GM and SkBenchmark match with the one in Test.
2) Make gResourcePath a static pointer to const char and move it inside the classes.
BUG=None
TEST=make tests && out/Debug/tests
make gm && out/Debug/gm
make bench && out/Debug/bench
R=mtklein@google.com
Author: tfarina@chromium.org
Review URL: https://codereview.chromium.org/319473003
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
Committed: http://code.google.com/p/skia/source/detail?r=13586R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13592 2bbb7eff-a529-9590-31e7-b0007b416f81
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13586 2bbb7eff-a529-9590-31e7-b0007b416f81
This macro replaces:
SkString str;
str.printf("Foo test Expected %d got %d", x, y);
reporter->reportFailed(str);
with the shorter code:
REPORTF(reporter, ("Foo test Expected %d got %d", x, y));
The new form also appends __FILE__:__LINE__ to the message before calling reportFailed().
BUG=
R=mtklein@google.com
Review URL: https://codereview.chromium.org/132843002
git-svn-id: http://skia.googlecode.com/svn/trunk@13016 2bbb7eff-a529-9590-31e7-b0007b416f81
Obviously these are all currently function pointers of type T(*)(P) for various
T and P. In bench refactoring, I'm trying to register a function pointer of
type T(*)(), which can't be done as is (passing P=void doesn't work). This
also lets us register things like primitives, which is conceivable useful.
BUG=
R=reed@google.com, scroggo@google.com
Review URL: https://codereview.chromium.org/23453031
git-svn-id: http://skia.googlecode.com/svn/trunk@11082 2bbb7eff-a529-9590-31e7-b0007b416f81
make_canonical_dir_path only worked if the provided directory
did not end with a slash. Remove this function, and call
SkPathJoin instead. Update the documentation to acknowledge
that this is an acceptable use of SkPathJoin, and update its
test.
R=epoger@google.com
Review URL: https://codereview.chromium.org/16098011
git-svn-id: http://skia.googlecode.com/svn/trunk@9458 2bbb7eff-a529-9590-31e7-b0007b416f81
modify threaded path ops tests to check
Background: this CL came out of a conversation with Eric where I learned that 10s of machines host 100s of bots. Since the bot hosting tests may be shared with many other tasks, it seems unwise for path ops to launch multiple test threads.
The change here is to make launching multiple threads "opt-in" and by default, bots can run path ops in a single thread.
Review URL: https://codereview.chromium.org/14002007
git-svn-id: http://skia.googlecode.com/svn/trunk@8750 2bbb7eff-a529-9590-31e7-b0007b416f81
- fix rand for Android
- build unit test on linux
- use atomic inc in test count
- add casting for Android
git-svn-id: http://skia.googlecode.com/svn/trunk@8610 2bbb7eff-a529-9590-31e7-b0007b416f81
The problem arises on devices like the Nexus 10 where we allow the
destruction of resources using the destructor of a static variable.
However, we have no guarentee that the GPU driver has not already
cleaned up it's resources prior to our static destructor.
Review URL: https://codereview.appspot.com/6851124
git-svn-id: http://skia.googlecode.com/svn/trunk@6599 2bbb7eff-a529-9590-31e7-b0007b416f81
I have manually examined all of these diffs and restored a few files that
seem to require manual adjustment.
The following files still need to be modified manually, in a separate CL:
android_sample/SampleApp/AndroidManifest.xml
android_sample/SampleApp/res/layout/layout.xml
android_sample/SampleApp/res/menu/sample.xml
android_sample/SampleApp/res/values/strings.xml
android_sample/SampleApp/src/com/skia/sampleapp/SampleApp.java
android_sample/SampleApp/src/com/skia/sampleapp/SampleView.java
experimental/CiCarbonSampleMain.c
experimental/CocoaDebugger/main.m
experimental/FileReaderApp/main.m
experimental/SimpleCocoaApp/main.m
experimental/iOSSampleApp/Shared/SkAlertPrompt.h
experimental/iOSSampleApp/Shared/SkAlertPrompt.m
experimental/iOSSampleApp/SkiOSSampleApp-Base.xcconfig
experimental/iOSSampleApp/SkiOSSampleApp-Debug.xcconfig
experimental/iOSSampleApp/SkiOSSampleApp-Release.xcconfig
gpu/src/android/GrGLDefaultInterface_android.cpp
gyp/common.gypi
gyp_skia
include/ports/SkHarfBuzzFont.h
include/views/SkOSWindow_wxwidgets.h
make.bat
make.py
src/opts/memset.arm.S
src/opts/memset16_neon.S
src/opts/memset32_neon.S
src/opts/opts_check_arm.cpp
src/ports/SkDebug_brew.cpp
src/ports/SkMemory_brew.cpp
src/ports/SkOSFile_brew.cpp
src/ports/SkXMLParser_empty.cpp
src/utils/ios/SkImageDecoder_iOS.mm
src/utils/ios/SkOSFile_iOS.mm
src/utils/ios/SkStream_NSData.mm
tests/FillPathTest.cpp
Review URL: http://codereview.appspot.com/4816058
git-svn-id: http://skia.googlecode.com/svn/trunk@1982 2bbb7eff-a529-9590-31e7-b0007b416f81
