Add skiatest::Failure to keep track of data about a test failure.
Reporter::reportFailed and ::onReportFailed now take Failure as a
parameter. This allows the implementation to treat the failure as it
wishes. Provide a helper to format the failure the same as prior to
the change.
Update the macros for calling reportFailed (REPORTER_ASSERT etc) to
create a Failure object.
Convert a direct call to reportFailed to the macro ERRORF.
Write Failures to Json.
Sample output when running dm on the dummy test crrev.com/705723004:
{
"test_results" : {
"failures" : [
{
"condition" : "0 > 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 10,
"message" : ""
},
{
"condition" : "false",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 4,
"message" : ""
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 5,
"message" : "I can too count!"
},
{
"condition" : "",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 6,
"message" : "seven is 7"
},
{
"condition" : "1 == 3",
"file_name" : "../../tests/DummyTest.cpp",
"line_no" : 14,
"message" : "I can too count!"
}
]
}
}
Report all of the failures from one test.
Previously, if one test had multiple failures, only one was reportered.
e.g:
Failures:
test Dummy: ../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
Now, we get all the messages:
Failures:
test Dummy: ../../tests/DummyTest.cpp:4 false
../../tests/DummyTest.cpp:5 I can too count!: 1 == 3
../../tests/DummyTest.cpp:6 seven is 7
test Dummy2: ../../tests/DummyTest.cpp:10 0 > 3
test Dummy3: ../../tests/DummyTest.cpp:14 I can too count!: 1 == 3
3 failures.
(Note that we still state "3 failures" because 3 DM::Tasks failed.)
BUG=skia:3082
BUG=skia:2454
Review URL: https://codereview.chromium.org/694703005
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
Reason for revert:
Leaks, leaks, leaks.
Original issue's description:
> SkThreadPool ~~> SkTaskGroup
>
> SkTaskGroup is like SkThreadPool except the threads stay in
> one global pool. Each SkTaskGroup itself is tiny (4 bytes)
> and its wait() method applies only to tasks add()ed to that
> instance, not the whole thread pool.
>
> This means we don't need to bring up new thread pools when
> tests themselves want to use multithreading (e.g. pathops,
> quilt). We just create a new SkTaskGroup and wait for that
> to complete. This should be more efficient, and allow us
> to expand where we use threads to really latency sensitive
> places. E.g. we can probably now use these in nanobench
> for CPU .skp rendering.
>
> Now that all threads are sharing the same pool, I think we
> can remove most of the custom mechanism pathops tests use
> to control threading. They'll just ride on the global pool
> with all other tests now.
>
> This (temporarily?) removes the GPU multithreading feature
> from DM, which we don't use.
>
> On my desktop, DM runs a little faster (57s -> 55s) in
> Debug, and a lot faster in Release (36s -> 24s). The bots
> show speedups of similar proportions, cutting more than a
> minute off the N4/Release and Win7/Debug runtimes.
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, reed@google.com, mtklein@chromium.orgTBR=bsalomon@google.com, bungeman@google.com, caryclark@google.com, mtklein@chromium.org, reed@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/533393002
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
Share command flags between dm and unit tests.
Also, allow dm's core to be included by itself and iOSShell.
Command line flags that are the same (or nearly the same) in DM
and in skia_tests have been moved to common_flags. Authors,
please check to see that the shared common flag is correct for
the tool.
For iOS, the 'tool_main' entry point has a wrapper to allow multiple
tools to be statically linked in the iOSShell.
Since SkCommandLineFlags::Parse can only be called once, these calls
are disabled in the IOS build.
Since the iOS app directory is dynamically assigned a name, use '@' to
select it. (This is the same convention chosen by the Mobile Harness
iOS file system utilities.)
Move the heart of dm.gyp into dm.gypi so that it can be included by
itself and iOSShell.gyp.
Add tools/flags/SkCommonFlags.* to define and declare common
command line flags.
Add support for dm to iOSShell.
BUG=skia:
R=scroggo@google.com, mtklein@google.com, jvanverth@google.com, bsalomon@google.com
Author: caryclark@google.com
Review URL: https://codereview.chromium.org/389653004
Mike K: please sanity check Test.cpp and skia_test.cpp
Feel free to look at the rest, but I don't expect any in depth review of path ops innards.
Path Ops first iteration used QuickSort to order segments radiating from an intersection to compute the winding rule.
This revision uses a circular sort instead. Breaking out the circular sort into its own long-lived structure (SkOpAngle) allows doing less work and provides a home for caching additional sorting data.
The circle sort is more stable than the former sort, has a robust ordering and fewer exceptions. It finds unsortable ordering less often. It is less reliant on the initial curve tangent, using convex hulls instead whenever it can.
Additional debug validation makes sure that the computed structures are self-consistent. A new visualization tool helps verify that the angle ordering is correct.
The 70+M tests pass with this change on Windows, Mac, Linux 32 and Linux 64 in debug and release.
R=mtklein@google.com, reed@google.com
Author: caryclark@google.com
Review URL: https://codereview.chromium.org/131103009
git-svn-id: http://skia.googlecode.com/svn/trunk@14183 2bbb7eff-a529-9590-31e7-b0007b416f81
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
Committed: http://code.google.com/p/skia/source/detail?r=13586R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13592 2bbb7eff-a529-9590-31e7-b0007b416f81
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13586 2bbb7eff-a529-9590-31e7-b0007b416f81
These guys are not heavily contended nor speed critical. No need for atomics,
plus this makes tsan stop complaining (correctly) about reading fNextIndex
unsafely in onEnd.
I took a look at failCount/fFailCount, which I think is safely atomic and quite
conveniently so: It's never read until all the threads which could possibly
increment it have terminated (except for the one where it was created,
obviously). We could guard it with a mutex too, but maybe we can let this one
slide.
BUG=
R=bungeman@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/25357002
git-svn-id: http://skia.googlecode.com/svn/trunk@11561 2bbb7eff-a529-9590-31e7-b0007b416f81
PathOps tests internal routines direcctly. Check to make sure that
test points, lines, quads, curves, triangles, and bounds read from
arrays are valid (i.e., don't contain NaN) before calling the
test function.
Repurpose the test flags.
- make 'v' verbose test region output against path output
- make 'z' single threaded (before it made it multithreaded)
The latter change speeds up tests run by the buildbot by 2x to 3x.
BUG=
Review URL: https://codereview.chromium.org/19374003
git-svn-id: http://skia.googlecode.com/svn/trunk@10107 2bbb7eff-a529-9590-31e7-b0007b416f81
make_canonical_dir_path only worked if the provided directory
did not end with a slash. Remove this function, and call
SkPathJoin instead. Update the documentation to acknowledge
that this is an acceptable use of SkPathJoin, and update its
test.
R=epoger@google.com
Review URL: https://codereview.chromium.org/16098011
git-svn-id: http://skia.googlecode.com/svn/trunk@9458 2bbb7eff-a529-9590-31e7-b0007b416f81
This modifies the command line to take test
matches of the form:
--match [or -m] [~][^]match[$] [~][^]match[$] ...
~ causes a matching test to always be skipped
^ requires the start of the test to match
$ requires the end of the test to match
^ and $ requires an exact match
If a test does not match any list entry,
it is skipped unless some list entry starts with ~
Review URL: https://codereview.chromium.org/14650009
git-svn-id: http://skia.googlecode.com/svn/trunk@8955 2bbb7eff-a529-9590-31e7-b0007b416f81
modify threaded path ops tests to check
Background: this CL came out of a conversation with Eric where I learned that 10s of machines host 100s of bots. Since the bot hosting tests may be shared with many other tasks, it seems unwise for path ops to launch multiple test threads.
The change here is to make launching multiple threads "opt-in" and by default, bots can run path ops in a single thread.
Review URL: https://codereview.chromium.org/14002007
git-svn-id: http://skia.googlecode.com/svn/trunk@8750 2bbb7eff-a529-9590-31e7-b0007b416f81
I don't understand why this change is necessary. On Android,
SkCommandLineFlags.h is not found, but only in this project.
Other projects depend on flags and include the file without
using the full path. Likewise, this works on other platforms.
Removing for now until I figure out the correct fix.
Review URL: https://codereview.chromium.org/13910008
git-svn-id: http://skia.googlecode.com/svn/trunk@8621 2bbb7eff-a529-9590-31e7-b0007b416f81
I removed unused parameters in the tests wherever it was trivial to do so. I'm trying to get the easy ones out of the way before we get into more involved discussions around this.
Review URL: https://codereview.appspot.com/7394055
git-svn-id: http://skia.googlecode.com/svn/trunk@7891 2bbb7eff-a529-9590-31e7-b0007b416f81
