This will hold us closer to the principle that the test name (and only the test
name) should correspond to expected output. By the same reasoning, mix in the
number of subsets: if that changes, the expected output also changes.
BUG=skia:3255
Review URL: https://codereview.chromium.org/863723002
skiatest::Test class is now a simple struct. Some
functionalty, such as counting errors or timing is now
handled elsewhere.
skiatest:Reporter is now a simpler abstract class. The two
implementations handle test errors.
DM and pathops_unittest updated.
Review URL: https://codereview.chromium.org/830513004
BUG=skia:3255
I think this supports everything DM used to, but has completely refactored how
it works to fit the design in the bug.
Configs like "tiles-gpu" are automatically wired up.
I wouldn't suggest looking at this as a diff. There's just a bunch of deleted
files, a few new files, and one new file that shares a name with a deleted file
(DM.cpp).
NOTREECHECKS=true
Committed: https://skia.googlesource.com/skia/+/709d2c3e5062c5b57f91273bfc11a751f5b2bb88
Review URL: https://codereview.chromium.org/788243008
Reason for revert:
plenty of data
Original issue's description:
> Sketch DM refactor.
>
> BUG=skia:3255
>
>
> I think this supports everything DM used to, but has completely refactored how
> it works to fit the design in the bug.
>
> Configs like "tiles-gpu" are automatically wired up.
>
> I wouldn't suggest looking at this as a diff. There's just a bunch of deleted
> files, a few new files, and one new file that shares a name with a deleted file
> (DM.cpp).
>
> NOTREECHECKS=true
>
> Committed: https://skia.googlesource.com/skia/+/709d2c3e5062c5b57f91273bfc11a751f5b2bb88TBR=bsalomon@google.com,mtklein@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=skia:3255
Review URL: https://codereview.chromium.org/853883004
BUG=skia:3255
I think this supports everything DM used to, but has completely refactored how
it works to fit the design in the bug.
Configs like "tiles-gpu" are automatically wired up.
I wouldn't suggest looking at this as a diff. There's just a bunch of deleted
files, a few new files, and one new file that shares a name with a deleted file
(DM.cpp).
NOTREECHECKS=true
Review URL: https://codereview.chromium.org/788243008
If no rasterizer is compiled in, this flag does nothing. Default
value (true) gives the same behavior as before.
Review URL: https://codereview.chromium.org/830333005
ico, wbmp, plus the alternate suffix jpeg.
Also check for capitalized versions, since files sometimes use
capitalized suffixes.
BUG=skia:3235
Review URL: https://codereview.chromium.org/798383003
Reason for revert:
Causing breakages on Mac build.
Original issue's description:
> Make nanobench and dm be usable from Chromium build
>
> Move the app logic for each app as follows:
>
> <app>.cpp -- the file which contains main(). Embedders that compile
> their own apps, such as ios shell, upcoming Chromium dm etc, do not use this.
>
> <app>_main.cpp -- the main logic of the Skia test application. This will be
> used by Skia -compiled apps as well as embedder -compiled apps.
>
> <app>_main.h -- the API for the main logic. This will be
> used by Skia -compiled apps as well as embedder -compiled apps.
>
> This way (the upcoming) Chromium dm can setup its Chromium-specific setup
> in custom main(), and then call dm_main(), without the need of any
> SK_BUILD_FOR_XXXX defines controlling whether the tool defines main or not.
>
> BUG=skia:2992
>
> Committed: https://skia.googlesource.com/skia/+/c092d3bdab5f723576cc0346cea3ee282a9cb444TBR=mtklein@chromium.org,mtklein@google.com,borenet@google.com,kkinnunen@nvidia.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:2992
Review URL: https://codereview.chromium.org/724073002
Move the app logic for each app as follows:
<app>.cpp -- the file which contains main(). Embedders that compile
their own apps, such as ios shell, upcoming Chromium dm etc, do not use this.
<app>_main.cpp -- the main logic of the Skia test application. This will be
used by Skia -compiled apps as well as embedder -compiled apps.
<app>_main.h -- the API for the main logic. This will be
used by Skia -compiled apps as well as embedder -compiled apps.
This way (the upcoming) Chromium dm can setup its Chromium-specific setup
in custom main(), and then call dm_main(), without the need of any
SK_BUILD_FOR_XXXX defines controlling whether the tool defines main or not.
BUG=skia:2992
Review URL: https://codereview.chromium.org/657373002
Reason for revert:
Not compiling in ANGLE build
Original issue's description:
> Get gpudft support working in dm, gm, nanobench and bench_pictures
>
> Adds a new config to test distance field text.
> Clean up some flags and #defines to read "distance field text",
> not "distance field fonts" to be consistent with Chromium
>
> NOTREECHECKS=true
>
> Committed: https://skia.googlesource.com/skia/+/06ba179838ba4fe187cf290750aeeb4a02a2960bTBR=bsalomon@google.com,mtklein@google.com,reed@google.com
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/707723005
Adds a new config to test distance field text.
Clean up some flags and #defines to read "distance field text",
not "distance field fonts" to be consistent with Chromium
NOTREECHECKS=true
Review URL: https://codereview.chromium.org/699453005
Add JsonWriter, which handles Json output from DM, in preparation for
adding json output for tests. This change should not affect behavior.
BUG=skia:2454
Review URL: https://codereview.chromium.org/702513003
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
Reason for revert:
Leaks, leaks, leaks.
Original issue's description:
> SkThreadPool ~~> SkTaskGroup
>
> SkTaskGroup is like SkThreadPool except the threads stay in
> one global pool. Each SkTaskGroup itself is tiny (4 bytes)
> and its wait() method applies only to tasks add()ed to that
> instance, not the whole thread pool.
>
> This means we don't need to bring up new thread pools when
> tests themselves want to use multithreading (e.g. pathops,
> quilt). We just create a new SkTaskGroup and wait for that
> to complete. This should be more efficient, and allow us
> to expand where we use threads to really latency sensitive
> places. E.g. we can probably now use these in nanobench
> for CPU .skp rendering.
>
> Now that all threads are sharing the same pool, I think we
> can remove most of the custom mechanism pathops tests use
> to control threading. They'll just ride on the global pool
> with all other tests now.
>
> This (temporarily?) removes the GPU multithreading feature
> from DM, which we don't use.
>
> On my desktop, DM runs a little faster (57s -> 55s) in
> Debug, and a lot faster in Release (36s -> 24s). The bots
> show speedups of similar proportions, cutting more than a
> minute off the N4/Release and Win7/Debug runtimes.
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/9c7207b5dc71dc5a96a2eb107d401133333d5b6fR=caryclark@google.com, bsalomon@google.com, bungeman@google.com, reed@google.com, mtklein@chromium.orgTBR=bsalomon@google.com, bungeman@google.com, caryclark@google.com, mtklein@chromium.org, reed@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/533393002
SkTaskGroup is like SkThreadPool except the threads stay in
one global pool. Each SkTaskGroup itself is tiny (4 bytes)
and its wait() method applies only to tasks add()ed to that
instance, not the whole thread pool.
This means we don't need to bring up new thread pools when
tests themselves want to use multithreading (e.g. pathops,
quilt). We just create a new SkTaskGroup and wait for that
to complete. This should be more efficient, and allow us
to expand where we use threads to really latency sensitive
places. E.g. we can probably now use these in nanobench
for CPU .skp rendering.
Now that all threads are sharing the same pool, I think we
can remove most of the custom mechanism pathops tests use
to control threading. They'll just ride on the global pool
with all other tests now.
This (temporarily?) removes the GPU multithreading feature
from DM, which we don't use.
On my desktop, DM runs a little faster (57s -> 55s) in
Debug, and a lot faster in Release (36s -> 24s). The bots
show speedups of similar proportions, cutting more than a
minute off the N4/Release and Win7/Debug runtimes.
BUG=skia:
R=caryclark@google.com, bsalomon@google.com, bungeman@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/531653002
Allow GM results to be compared across machines and platforms by
standardizing the fonts used by all tests.
This adds runtime flags to DM to use either the system font context (the
default), the fonts in the resources directory ( --resourceFonts ) or a set
of canonical paths generated from the fonts ( --portableFonts ).
This CL should leave the current DM results unchanged by default.
If the portable font data or resource font is missing when DM is run, it
falls back to using the system font context.
The create_test_font tool generates the paths and metrics read by DM
with the --portableFonts flag set, and generates the font substitution
tables read by DM with the --resourceFonts flag set.
If DM is run in SkDebug mode with the --reportUsedChars flag set, it
generates the corresponding data compiled into the create_test_font tool.
All GM tests set their typeface information by calling either
sk_tool_utils::set_portable_typeface or
sk_tool_utils::portable_typeface .
(The former takes the paint, the latter returns a SkTypeface.) These calls
can be removed in the future when the Font Manager can be superceded.
BUG=skia:2687
R=mtklein@google.com
Review URL: https://codereview.chromium.org/407183003
Share command flags between dm and unit tests.
Also, allow dm's core to be included by itself and iOSShell.
Command line flags that are the same (or nearly the same) in DM
and in skia_tests have been moved to common_flags. Authors,
please check to see that the shared common flag is correct for
the tool.
For iOS, the 'tool_main' entry point has a wrapper to allow multiple
tools to be statically linked in the iOSShell.
Since SkCommandLineFlags::Parse can only be called once, these calls
are disabled in the IOS build.
Since the iOS app directory is dynamically assigned a name, use '@' to
select it. (This is the same convention chosen by the Mobile Harness
iOS file system utilities.)
Move the heart of dm.gyp into dm.gypi so that it can be included by
itself and iOSShell.gyp.
Add tools/flags/SkCommonFlags.* to define and declare common
command line flags.
Add support for dm to iOSShell.
BUG=skia:
R=scroggo@google.com, mtklein@google.com, jvanverth@google.com, bsalomon@google.com
Author: caryclark@google.com
Review URL: https://codereview.chromium.org/389653004
Always build the tools with JSON, but either build our own
or use the system's.
Rename skia_build_json_writer to skia_use_system_jsoncpp,
since we now always build with JSON.
Remove SK_BUILD_JSON_WRITER, which was only there so
we could build without JSON it in the framework.
BUG=skia:2448
R=djsollen@google.com, reed@google.com
Author: scroggo@google.com
Review URL: https://codereview.chromium.org/303913002
The main meat of things is in SkThreadPool. We can now give SkThreadPool a
type for each thread to create and destroy on its local stack. It's TLS
without going through SkTLS.
I've split the DM tasks into CpuTasks that run on threads with no TLS, and
GpuTasks that run on threads with a thread local GrContextFactory.
The old CpuTask and GpuTask have been renamed to CpuGMTask and GpuGMTask.
Upshot: default run of out/Debug/dm goes from ~45 seconds to ~20 seconds.
BUG=skia:
R=bsalomon@google.com, mtklein@google.com, reed@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/179233005
git-svn-id: http://skia.googlecode.com/svn/trunk@13632 2bbb7eff-a529-9590-31e7-b0007b416f81
Also:
- make GrMemoryPoolBenches threadsafe
- some tweaks to various DM code
- rename GM::shortName() to getName() to match benches and tests
On my desktop, (289 GMs, 617 benches) x 4 configs, 227 tests takes 46s in Debug, 14s in Release. (Still minutes faster than running tests && bench && gm.) GPU singlethreading is definitely the limiting factor again; going to reexamine whether that's helpful to thread it again.
BUG=skia:
R=reed@google.com, bsalomon@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178473006
git-svn-id: http://skia.googlecode.com/svn/trunk@13603 2bbb7eff-a529-9590-31e7-b0007b416f81
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
Committed: http://code.google.com/p/skia/source/detail?r=13586R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13592 2bbb7eff-a529-9590-31e7-b0007b416f81
- refactor GYPs and a few flags
- make GPU tests grab a thread-local GrContextFactory when needed as we do in DM for GMs
- add a few more UI features to make DM more like tests
I believe this makes the program 'tests' obsolete.
It should be somewhat faster to run the two sets together than running the old binaries serially:
- serial: tests 20s (3m18s CPU), dm 21s (3m01s CPU)
- together: 27s (6m21s CPU)
Next up is to incorporate benches. I'm only planning there on a single-pass sanity check, so that won't obsolete the program 'bench' just yet.
Tested: out/Debug/tests && out/Debug/dm && echo ok
BUG=skia:
R=reed@google.com, bsalomon@google.com, mtklein@google.com, tfarina@chromium.org
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/178273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13586 2bbb7eff-a529-9590-31e7-b0007b416f81
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
This is sort of the near-minimal proof-of-concept skeleton.
- It can run existing GMs.
- It supports most configs (just not PDF).
- --replay is the only "fancy" feature it currently supports
Hopefully you will be disturbed by its speed.
BUG=
R=epoger@google.com
Review URL: https://codereview.chromium.org/22839016
git-svn-id: http://skia.googlecode.com/svn/trunk@11802 2bbb7eff-a529-9590-31e7-b0007b416f81
