Commit Graph

97 Commits

Author SHA1 Message Date
mtklein
d603b22903 Suggested version with 'undo'.
BUG=skia:

Review URL: https://codereview.chromium.org/931483002
2015-02-17 11:13:33 -08:00
msarett
95f192d199 Adding new benchmark to test image decoding performance.
BUG=skia:

Review URL: https://codereview.chromium.org/918673002
2015-02-13 09:05:42 -08:00
mtklein
6dee2ad388 Try again to use a C++11 feature in DM as a canary.
CQ_EXTRA_TRYBOTS=client.skia.compile:Build-Ubuntu13.10-GCC4.8-NaCl-Release-Trybot,Build-Mac10.7-Clang-Arm7-Debug-iOS-Trybot

BUG=skia:

Review URL: https://codereview.chromium.org/898863003
2015-02-05 09:53:44 -08:00
mtklein
b9eb4ac0f1 patch from issue 886233004 at patchset 40001 (http://crrev.com/886233004#ps40001)
... with changes proposed in the review.

BUG=skia:

Review URL: https://codereview.chromium.org/894013002
2015-02-02 18:26:03 -08:00
mtklein
e67164d9b3 DM: wire up --leaks / -l again
No public API changes.
TBR=reed@google.com

BUG=skia:

Review URL: https://codereview.chromium.org/893043002
2015-02-02 13:24:37 -08:00
mtklein
8a4527e98a SVG backend in DM
Not enabled by default, but this should get you SKPs, GMs etc for free to play with.

$ out/Debug/dm -w svgs --src gm skp --config svg

BUG=skia:

Review URL: https://codereview.chromium.org/892693002
2015-01-31 20:00:58 -08:00
mtklein
edc93bc740 Suggestions and merge in the other CL.
I had some suggestions on the subset CL, and took the opportunity to rebase it
against head and merge in the other color type CL.

BUG=skia:

Review URL: https://codereview.chromium.org/893703002
2015-01-30 13:22:23 -08:00
mtklein
8d17a13a71 DM: paths as implict strings too.
BUG=skia:

Review URL: https://codereview.chromium.org/891823002
2015-01-30 11:42:31 -08:00
halcanary
022afb8384 DM::NullSink
Motivation:  The null sink can act as a control for experiments.

Review URL: https://codereview.chromium.org/873723007
2015-01-30 11:00:12 -08:00
halcanary
23b03c3c5a dm: allow multiple --images flags, allow single files
Review URL: https://codereview.chromium.org/872993005
2015-01-30 09:58:58 -08:00
halcanary
fc37ad1680 dm: allow multiple --skp flags, allow single files
Review URL: https://codereview.chromium.org/885353002
2015-01-30 07:31:19 -08:00
mtklein
9c3f17d6e8 Fold gmtoskp into DM, as --src gm --config skp.
BUG=skia:

Review URL: https://codereview.chromium.org/885733002
2015-01-28 11:35:18 -08:00
mtklein
62bd1a69ea add -r to DM
$ out/Debug/dm -w good
$ out/Debug/dm -r good -w bad && echo "hooray no diffs!"

BUG=skia:

Review URL: https://codereview.chromium.org/863093003
2015-01-27 14:46:26 -08:00
mtklein
a17241bd0a Write dm.json periodically instead of only once at the end.
This way if a bot crashes, we might get some partial results in gold rather
than none.  We do the same sort of thing in nanobench for perf.

BUG=skia:3255

Review URL: https://codereview.chromium.org/872443003
2015-01-23 05:48:00 -08:00
mtklein
55e88b226c More natural way to serialize GPU tasks and tests.
This basically takes out the Windows-only hacks and promotes them to
cross-platform behavior driven by --gpu_threading.
    - When --gpu_threading is false (the default), this puts GPU tasks and tests
      together in the same GPU enclave.  They all run serially.
    - When --gpu_threading is true, both the tests and the tasks run totally
      independently, just like the thread-safe CPU-bound work.

BUG=skia:3255

Review URL: https://codereview.chromium.org/847273005
2015-01-21 15:50:13 -08:00
mtklein
7edca21226 Don't test pipe modes nobody uses.
SkDeferredCanvas uses a simple pipe: no cross-process, no shared-address, etc.
(see src/utils/SkDeferredCanvas.cpp:306).

We could just remove these modes from the bot configs, but I'd like to take the
opportunity to simplify the DM code too.  I'll happily volunteer to put things
back should we decide we want to test these modes.

BUG=skia:

Review URL: https://codereview.chromium.org/861303003
2015-01-21 13:18:51 -08:00
mtklein
6c5fed2d5c DM: --tests -> --src tests
BUG=skia:3255

Review URL: https://codereview.chromium.org/856173002
2015-01-20 13:47:23 -08:00
mtklein
19f306013e Allow pdf as a --config.
We just can't run this on the bots yet.  It's fine locally.

BUG=skia:3255

Review URL: https://codereview.chromium.org/857373002
2015-01-20 13:34:39 -08:00
mtklein
9264a95608 Image subsets as part of the name instead of a separate source type.
This will hold us closer to the principle that the test name (and only the test
name) should correspond to expected output.  By the same reasoning, mix in the
number of subsets: if that changes, the expected output also changes.

BUG=skia:3255

Review URL: https://codereview.chromium.org/863723002
2015-01-20 10:11:53 -08:00
halcanary
87f3ba4847 Simplify skiatest framework.
skiatest::Test class is now a simple struct.  Some
functionalty, such as counting errors or timing is now
handled elsewhere.

skiatest:Reporter is now a simpler abstract class.  The two
implementations handle test errors.

DM and pathops_unittest updated.

Review URL: https://codereview.chromium.org/830513004
2015-01-20 09:30:20 -08:00
mtklein
05982367f5 Revert of Add temporary blacklist debugging. (patchset #1 id:1 of https://codereview.chromium.org/805643004/)
Reason for revert:
fixed!

Original issue's description:
> Add temporary blacklist debugging.
>
> Not sure why blacklists don't seem to work on bots.  They look right.
>
> NOTREECHECKS=true
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/f77156e1925cd5d5504070fe5533a4c456732c1b

TBR=mtklein@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=skia:

Review URL: https://codereview.chromium.org/851903003
2015-01-15 19:11:55 -08:00
mtklein
f77156e192 Add temporary blacklist debugging.
Not sure why blacklists don't seem to work on bots.  They look right.

NOTREECHECKS=true

BUG=skia:

Review URL: https://codereview.chromium.org/805643004
2015-01-15 16:21:24 -08:00
mtklein
9dc0910c2c newlines after failures
NOTREECHECKS=true

BUG=skia:3255

Review URL: https://codereview.chromium.org/853123003
2015-01-15 15:47:33 -08:00
mtklein
2f64eec5df Totally serialize gpu tests on Windows.
NOTREECHECKS=true

BUG=skia:3255

Review URL: https://codereview.chromium.org/845193006
2015-01-15 14:20:42 -08:00
mtklein
e1315526cd Turn back on image and subset sources now that they're blacklisted on Android.
To land after https://codereview.chromium.org/850403002/

NOTREECHECKS=true

BUG=skia:3255

Review URL: https://codereview.chromium.org/857483003
2015-01-15 13:58:56 -08:00
mtklein
a2ef642b00 DM: add basic --blacklist <config> <srcType> <name> functionality.
NOTREECHECKS=true

BUG=skia:3255

Review URL: https://codereview.chromium.org/817573005
2015-01-15 13:44:22 -08:00
mtklein
82d2843cc5 turn back on gpu tests
NOTREECHECKS=true

BUG=skia:3255

Review URL: https://codereview.chromium.org/817573004
2015-01-15 12:46:02 -08:00
mtklein
748ca3bf2d Sketch DM refactor.
BUG=skia:3255

I think this supports everything DM used to, but has completely refactored how
it works to fit the design in the bug.

Configs like "tiles-gpu" are automatically wired up.

I wouldn't suggest looking at this as a diff.  There's just a bunch of deleted
files, a few new files, and one new file that shares a name with a deleted file
(DM.cpp).

NOTREECHECKS=true

Committed: https://skia.googlesource.com/skia/+/709d2c3e5062c5b57f91273bfc11a751f5b2bb88

Review URL: https://codereview.chromium.org/788243008
2015-01-15 10:56:12 -08:00
mtklein
114c3cd054 Revert of Sketch DM refactor. (patchset #45 id:850001 of https://codereview.chromium.org/788243008/)
Reason for revert:
plenty of data

Original issue's description:
> Sketch DM refactor.
>
> BUG=skia:3255
>
>
> I think this supports everything DM used to, but has completely refactored how
> it works to fit the design in the bug.
>
> Configs like "tiles-gpu" are automatically wired up.
>
> I wouldn't suggest looking at this as a diff.  There's just a bunch of deleted
> files, a few new files, and one new file that shares a name with a deleted file
> (DM.cpp).
>
> NOTREECHECKS=true
>
> Committed: https://skia.googlesource.com/skia/+/709d2c3e5062c5b57f91273bfc11a751f5b2bb88

TBR=bsalomon@google.com,mtklein@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=skia:3255

Review URL: https://codereview.chromium.org/853883004
2015-01-15 10:15:02 -08:00
mtklein
709d2c3e50 Sketch DM refactor.
BUG=skia:3255

I think this supports everything DM used to, but has completely refactored how
it works to fit the design in the bug.

Configs like "tiles-gpu" are automatically wired up.

I wouldn't suggest looking at this as a diff.  There's just a bunch of deleted
files, a few new files, and one new file that shares a name with a deleted file
(DM.cpp).

NOTREECHECKS=true

Review URL: https://codereview.chromium.org/788243008
2015-01-15 08:30:25 -08:00
halcanary
9e398f79c9 add --rasterPDF flag to DM
If no rasterizer is compiled in, this flag does nothing.  Default
value (true) gives the same behavior as before.

Review URL: https://codereview.chromium.org/830333005
2015-01-10 11:18:04 -08:00
scroggo
1d09ee7c04 Add more filetypes to DM's image tests.
ico, wbmp, plus the alternate suffix jpeg.

Also check for capitalized versions, since files sometimes use
capitalized suffixes.

BUG=skia:3235

Review URL: https://codereview.chromium.org/798383003
2014-12-15 08:53:35 -08:00
mtklein
f6139f7c38 Add image decoding mode to DM.
This is meant to supplant skimage.

BUG=skia:3235

Review URL: https://codereview.chromium.org/802793002
2014-12-12 16:41:12 -08:00
jcgregorio
3b27adef0a Revert of Make nanobench and dm be usable from Chromium build (patchset #5 id:80001 of https://codereview.chromium.org/657373002/)
Reason for revert:
Causing breakages on Mac build.

Original issue's description:
> Make nanobench and dm be usable from Chromium build
>
> Move the app logic for each app as follows:
>
> <app>.cpp -- the file which contains main(). Embedders that compile
> their own apps, such as ios shell, upcoming Chromium dm etc, do not use this.
>
> <app>_main.cpp -- the main logic of the Skia test application. This will be
> used by Skia -compiled apps as well as embedder -compiled apps.
>
> <app>_main.h -- the API for the main logic. This will be
> used by Skia -compiled apps as well as embedder -compiled apps.
>
> This way (the upcoming) Chromium dm can setup its Chromium-specific setup
> in custom main(), and then call dm_main(), without the need of any
> SK_BUILD_FOR_XXXX defines controlling whether the tool defines main or not.
>
> BUG=skia:2992
>
> Committed: https://skia.googlesource.com/skia/+/c092d3bdab5f723576cc0346cea3ee282a9cb444

TBR=mtklein@chromium.org,mtklein@google.com,borenet@google.com,kkinnunen@nvidia.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:2992

Review URL: https://codereview.chromium.org/724073002
2014-11-13 08:06:40 -08:00
kkinnunen
c092d3bdab Make nanobench and dm be usable from Chromium build
Move the app logic for each app as follows:

<app>.cpp -- the file which contains main(). Embedders that compile
their own apps, such as ios shell, upcoming Chromium dm etc, do not use this.

<app>_main.cpp -- the main logic of the Skia test application. This will be
used by Skia -compiled apps as well as embedder -compiled apps.

<app>_main.h -- the API for the main logic. This will be
used by Skia -compiled apps as well as embedder -compiled apps.

This way (the upcoming) Chromium dm can setup its Chromium-specific setup
in custom main(), and then call dm_main(), without the need of any
SK_BUILD_FOR_XXXX defines controlling whether the tool defines main or not.

BUG=skia:2992

Review URL: https://codereview.chromium.org/657373002
2014-11-13 05:00:57 -08:00
jvanverth
4736e1434a Get gpudft support working in dm, gm, nanobench and bench_pictures
Adds a new config to test distance field text.
Clean up some flags and #defines to read "distance field text",
not "distance field fonts" to be consistent with Chromium

NOTREECHECKS=true

Committed: https://skia.googlesource.com/skia/+/06ba179838ba4fe187cf290750aeeb4a02a2960b

Review URL: https://codereview.chromium.org/699453005
2014-11-07 07:12:46 -08:00
jvanverth
aa30ab3079 Revert of Get gpudft support working in dm, gm, nanobench and bench_pictures (patchset #2 id:20001 of https://codereview.chromium.org/699453005/)
Reason for revert:
Not compiling in ANGLE build

Original issue's description:
> Get gpudft support working in dm, gm, nanobench and bench_pictures
>
> Adds a new config to test distance field text.
> Clean up some flags and #defines to read "distance field text",
> not "distance field fonts" to be consistent with Chromium
>
> NOTREECHECKS=true
>
> Committed: https://skia.googlesource.com/skia/+/06ba179838ba4fe187cf290750aeeb4a02a2960b

TBR=bsalomon@google.com,mtklein@google.com,reed@google.com
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/707723005
2014-11-06 13:52:45 -08:00
jvanverth
06ba179838 Get gpudft support working in dm, gm, nanobench and bench_pictures
Adds a new config to test distance field text.
Clean up some flags and #defines to read "distance field text",
not "distance field fonts" to be consistent with Chromium

NOTREECHECKS=true

Review URL: https://codereview.chromium.org/699453005
2014-11-06 13:38:52 -08:00
scroggo
7a10fb6bea Separate JSON functions from DMWriteTask.
Add JsonWriter, which handles Json output from DM, in preparation for
adding json output for tests. This change should not affect behavior.

BUG=skia:2454

Review URL: https://codereview.chromium.org/702513003
2014-11-04 07:21:10 -08:00
halcanary
3dc5d70459 Use MacOS's CoreGraphics PDF Rasterizer in DM
BUG=3061

Review URL: https://codereview.chromium.org/650323004
2014-10-27 06:24:11 -07:00
bsalomon
06cddec857 Print GPU cache stats in nanobench/dm with veryVerbose
Review URL: https://codereview.chromium.org/680553002
2014-10-24 10:40:50 -07:00
mtklein
197ceda929 Remove expectations / -r from DM entirely.
It's getting tricky to coordinate changes to output for bots with -r,
and -r is not widely used.  The suggested alternative is to run skdiff.

BUG=skia:
R=jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/553583004
2014-09-09 07:36:57 -07:00
mtklein
23c94f05ca Let .skps have Expectations (i.e. work with -r) too.
BUG=skia:
R=jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/549183002
2014-09-08 09:12:28 -07:00
mtklein
1d0f1642e8 Start to rework DM JSON handling.
DM's striking off into its own JSON world.  This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.

For what it's worth, I basically want to change _all_ these pieces,
  - MD5 is slow and we can replace it with something faster,
  - JSON schema needs room to grow more data,
  - it'd be nice to hash once instead of twice when reading and writing,
  - this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.

E.x. file for now:

mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
   "3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
   "3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
   "3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
   "aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
   "aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
   "aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...

BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/546873002
2014-09-08 08:05:18 -07:00
mtklein
406654be7a SkThreadPool ~~> SkTaskGroup
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool.  Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.

This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt).  We just create a new SkTaskGroup and wait for that
to complete.  This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places.  E.g. we can probably now use these in nanobench
for CPU .skp rendering.

Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading.  They'll just ride on the global pool
with all other tests now.

This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.

On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s).  The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.

BUG=skia:

Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6f

R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/531653002
2014-09-03 15:34:37 -07:00
mtklein
2460bbdfbb Revert of SkThreadPool ~~> SkTaskGroup (patchset #4 id:60001 of https://codereview.chromium.org/531653002/)
Reason for revert:
Leaks, leaks, leaks.

Original issue's description:
> SkThreadPool ~~> SkTaskGroup
>
> SkTaskGroup is like SkThreadPool except the threads stay in
> one global pool.  Each SkTaskGroup itself is tiny (4 bytes)
> and its wait() method applies only to tasks add()ed to that
> instance, not the whole thread pool.
>
> This means we don't need to bring up new thread pools when
> tests themselves want to use multithreading (e.g. pathops,
> quilt).  We just create a new SkTaskGroup and wait for that
> to complete.  This should be more efficient, and allow us
> to expand where we use threads to really latency sensitive
> places.  E.g. we can probably now use these in nanobench
> for CPU .skp rendering.
>
> Now that all threads are sharing the same pool, I think we
> can remove most of the custom mechanism pathops tests use
> to control threading.  They'll just ride on the global pool
> with all other tests now.
>
> This (temporarily?) removes the GPU multithreading feature
> from DM, which we don't use.
>
> On my desktop, DM runs a little faster (57s -> 55s) in
> Debug, and a lot faster in Release (36s -> 24s).  The bots
> show speedups of similar proportions, cutting more than a
> minute off the N4/Release and Win7/Debug runtimes.
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6f

R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, reed@google.com, mtklein@chromium.org
TBR=bsalomon@google.com, bungeman@google.com, caryclark@google.com, mtklein@chromium.org, reed@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:

Author: mtklein@google.com

Review URL: https://codereview.chromium.org/533393002
2014-09-03 14:17:48 -07:00
mtklein
9c7207b5dc SkThreadPool ~~> SkTaskGroup
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool.  Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.

This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt).  We just create a new SkTaskGroup and wait for that
to complete.  This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places.  E.g. we can probably now use these in nanobench
for CPU .skp rendering.

Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading.  They'll just ride on the global pool
with all other tests now.

This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.

On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s).  The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.

BUG=skia:
R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/531653002
2014-09-03 14:06:48 -07:00
mtklein
60c7707958 Guard ANGLE and MESA instead of defining them as native when unsupported.
Since we've added "angle" to the default list of configs we've been running
kNative_GLContextType tests twice in DM, once for the real native, once for
angle.  This pointlessly doubles the GPU critical path in DM.  Whoops.

NOTREECHECKS=true

BUG=skia:
R=bsalomon@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/471063002
2014-08-14 10:36:55 -07:00
mtklein
72ebb9f1dc DM: conserve memory when using --skps
- limit to maximum 1000x1000 viewport like in bench_pictures and nanobench
 - use lazy bitmap decoding when reading the pictures from disk

BUG=skia:
R=reed@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/448263002
2014-08-07 14:27:03 -07:00
mtklein
36ea6c81c2 For some reason, starting tests before gms makes DM run much faster.
Must be getting the path ops tests going early helps?

On my desktop,
Release: 50s -> 24s
Debug: 62s -> 40s

BUG=skia:
R=bsalomon@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/435423003
2014-08-05 07:19:46 -07:00