Commit Graph

244 Commits

Author SHA1 Message Date
mtklein
d664c21a38 nanobench: lazily decode bitmaps from SKPs
This makes it considerably cheaper to run SKP recording benchmarks, without
affecting their measurements and without really affecting SKP playback
benchmarks at all.

On my machine, running out/Release/nanobench --match skp --config nondrendering
drops in run time from 6.7s to 2.5s, and the peak RAM usage drops from 129M to 50M.

I'm strongly considering making this lazy decoding the default.

BUG=skia:
R=robertphillips@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/572933006
2014-09-16 13:36:12 -07:00
mtklein
fd731ce804 Measure picture recording speed in nanobench.
Today we measure SkPicture playback speed, but not the time it takes to record
the SkPicture.  This fixes that by reading SKPs from disk and re-recording them.

On the console, recording shows up first as the nonrendering skp benches,
followed later by the usual playback benches:

maxrss  loops   min median  mean    max stddev  samples     config  bench
51M  2   165µs   168µs   169µs   178µs   3%  ▆▄▃█▂▄▁▂▁▁  nonrendering    tabl_slashdot.skp
57M  1   9.72ms  9.77ms  9.79ms  9.97ms  1%  █▂▂▅▃▂▁▄▂▁  nonrendering    desk_pokemonwiki.skp
57M  32  2.92µs  2.96µs  3.03µs  3.46µs  6%  ▅▁▁▁▁▁▁█▂▁  nonrendering    desk_yahoosports.skp
...
147M 1   3.86ms  3.87ms  3.97ms  4.81ms  7%  █▁▁▁▁▁▁▁▁▁  8888    tabl_slashdot.skp_1
147M 1   4.54ms  4.56ms  4.55ms  4.56ms  0%  █▅▇▅█▅▂▁▅▁  565     tabl_slashdot.skp_1
147M 2   3.08ms  3.24ms  4.17ms  8.18ms  50% █▁▁█▁▁▁▁▁▁  gpu     tabl_slashdot.skp_1
147M 1   1.61ms  1.62ms  1.69ms  2.33ms  13% █▁▁▁▁▁▁▁▁▁  8888    desk_pokemonwiki.skp_1
147M 1   1.44ms  1.44ms  1.45ms  1.47ms  1%  ▅▂█▂▂▅▁▁▂▁  565     desk_pokemonwiki.skp_1
...

On skiaperf.com, they'll also be separated out from playback benches by bench_type.

BUG=skia:
R=reed@google.com, mtklein@google.com, jcgregorio@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/559153002
2014-09-10 12:19:30 -07:00
mtklein
962890568d Distinguish common and unique names for skiaperf.com.
Turns out we tack on the size post-facto in ResultsWriter::bench(), so the only
place we need getUniqueName() to differ from getName() is SKPBench.

BUG=skia:
R=jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/552303004
2014-09-10 12:05:59 -07:00
qiankun.miao
8247ec313d Fix format of nanobench result
Column of samples is too wide. This makes config is not align with the
'config' title. Pad 'samples' tilte with some whitespaces to fix this
issue.

BUG=skia:
R=mtklein@google.com

Author: qiankun.miao@intel.com

Review URL: https://codereview.chromium.org/549153005
2014-09-09 19:24:36 -07:00
mtklein
ea65bfa8de Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->

{
   "build_number" : "20",
   "git_hash" : "abcd",
   "key" : {
      "arch" : "x86",
      "gpu" : "nvidia",
      "model" : "z620"
   },
   "results" : [
      {
         "key" : {
            "config" : "565",
            "name" : "ninepatch-stretch"
         },
         "md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
         "options" : {
            "source_type" : "GM"
         }
      },
      {
         "key" : {
            "config" : "8888",
            "name" : "ninepatch-stretch"
         },
         "md5" : "3e8a42f35a1e76f00caa191e6310d789",
         "options" : {
            "source_type" : "GM"
         }
      },
...

This breaks -r, but that's okay.  Going to follow up this CL with one that removes that entirely.

BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/551873003
2014-09-09 07:59:46 -07:00
bsalomon
49f085dddf "NULL !=" = NULL
R=reed@google.com

Author: bsalomon@google.com

Review URL: https://codereview.chromium.org/544233002
2014-09-05 13:34:00 -07:00
robertphillips
c5ba71d2e5 Change SkPicture::draw to playback
R=reed@google.com

Author: robertphillips@google.com

Review URL: https://codereview.chromium.org/540963002
2014-09-04 08:42:50 -07:00
reed
e5ea500d47 Hide fields in SkImageInfo
R=rmistry@google.com
TBR=bsalomon

Author: reed@google.com

Review URL: https://codereview.chromium.org/536003002
2014-09-03 11:54:58 -07:00
robertphillips
a8d7f0b13c Try out scalar picture sizes
This paves the way for removing the 'fTile' parameter from SkPictureShader (although that should be a different CL). If we like this we could also move to providing an entire cull SkRect.

R=reed@google.com, mtklein@google.com, fmalita@google.com, fmalita@chromium.org

Author: robertphillips@google.com

Review URL: https://codereview.chromium.org/513983002
2014-08-29 08:03:56 -07:00
reed
233c295980 fix numerical overflows in 565 blends
BUG=skia:2797
R=mtklein@google.com

Author: reed@google.com

Review URL: https://codereview.chromium.org/474983007
2014-08-22 07:27:14 -07:00
mtklein
2084050a33 Add --bbh (default true) to nanobench.
Chrome's using a bounding box, so it's a good idea for our
bots to do so too.

When set, we'll create an SkTileGrid to match the
parameters of --clip, and so should always hit its fast
path.

This will impose a small overhead (querying the BBH) on all
SKPs, but make large SKPs render more quickly.  E.g. on
GPU desk_pokemonwiki should show about a 30% improvement,
tabl_mozilla about 40%, and one very long page from my
personal suite, askmefast.com, gets 5x faster.

(The performance changes are not the point of the CL, but
something we should be aware of.)

BUG=
R=bsalomon@google.com, mtklein@google.com, robertphillips@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/497493003
2014-08-21 15:51:22 -07:00
mtklein
1915b62637 Add --properties for things like gitHash that describe the current nanobench run.
--key describes the type of run (describes the line on the chart), --properties
describes the run itself (describes the dot on the chart).

We'll pass --properties gitHash <git hash> build_number <build number>  --key
... to nanobench from the bots.

And... delete a whole lot of dead code.

Example: nanobench --properties gitHash foo build_number 1234 --key bar baz

{
   "build_number" : "1234",
   "gitHash" : "foo",
   "key" : {
      "bar" : "baz"
   },
   "results" : {
....

Friends with https://codereview.chromium.org/491943002
BUG=skia:
R=jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/488213002
2014-08-20 11:45:00 -07:00
mtklein
753b870c62 Define NDEBUG instead of SK_DEBUG/SK_RELEASE.
This makes our builds more like Chrome's, and will make our builds fail
if we accidentally use if SK_DEBUG instead of ifdef SK_DEBUG.

BUG=skia:
R=mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/481513004
2014-08-20 07:38:46 -07:00
mtklein
afb4379dbc Print max RSS in GM and nanobench too.
Everyone used MB, so update the API to just return that.

BUG=skia:
R=halcanary@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/483323002
2014-08-19 15:55:55 -07:00
mtklein
94e51567dd Add --options to nanobench, similar to --key but for non-identifying options.
Friends with https://codereview.chromium.org/487233003/

Example of out/Release/nanobench --options build_number 12374 --match patch_grid_texs_small

{
   "gitHash":"unknown-revision",
   "options":{
      "build_number":"12374",
      "system":"UNIX"
   },
   "results":{
      "patch_grid_texs_small_640_480":{
         "565":{
            "max_ms":0.268116,
            "mean_ms":0.2318529,
            "median_ms":0.235337,
            "min_ms":0.219158,
            "options":{
               "source_type":"bench"
            },
            "stddev_ms":0.01491263917658814
         },
         "8888":{
            "max_ms":0.231881,
            "mean_ms":0.2214668,
            "median_ms":0.219356,
            "min_ms":0.218887,
            "options":{
               "source_type":"bench"
            },
            "stddev_ms":0.004595541070791701
         },
         "gpu":{
            "max_ms":0.1398304782608696,
            "mean_ms":0.128833402173913,
            "median_ms":0.1316798695652174,
            "min_ms":0.1111915434782609,
            "options":{
               "GL_RENDERER":"Quadro 600/PCIe/SSE2",
               "GL_SHADING_LANGUAGE_VERSION":"4.40 NVIDIA via Cg compiler",
               "GL_VENDOR":"NVIDIA Corporation",
               "GL_VERSION":"4.4.0 NVIDIA 331.79",
               "source_type":"bench"
            },
            "stddev_ms":0.008923738937837156
         }
      }
   }
}

BUG=skia:
R=jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/490683002
2014-08-19 12:41:56 -07:00
krajcevski
12b3544028 Add runtime flag to turn compressed alpha masks on
R=bsalomon@google.com, robertphillips@google.com, mtklein@google.com

Author: krajcevski@google.com

Review URL: https://codereview.chromium.org/468293002
2014-08-13 12:06:26 -07:00
krajcevski
69a5560ea5 Add entry point for passing options to the GrContextFactory
R=bsalomon@google.com, mtklein@google.com, robertphillips@google.com

Author: krajcevski@google.com

Review URL: https://codereview.chromium.org/465073002
2014-08-13 10:46:31 -07:00
mtklein
b511042bb0 Fix iOS build by centralizing --writePath.
CQ_EXTRA_TRYBOTS=tryserver.skia:Build-Mac10.7-Clang-Arm7-Debug-iOS-Trybot

BUG=skia:
R=mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/452633002
2014-08-07 15:20:02 -07:00
bsalomon
6eb03cc06d Add option to dump images from nanobench.
Add option to set the repeat count to any number, replacs the --runOnce flag.

R=mtklein@google.com

Author: bsalomon@google.com

Review URL: https://codereview.chromium.org/450743002
2014-08-07 14:28:50 -07:00
bsalomon
3b4d077fba Add angle config to nanobench and make angle a default config for dm and nanobench.
NOTREECHECKS=true
R=mtklein@google.com

Author: bsalomon@google.com

Review URL: https://codereview.chromium.org/441333003
2014-08-06 10:52:34 -07:00
mtklein
2069e22003 Fix calibration loop failure condition.
With the old logic, if the last attempt succeeded, we'd say we failed.

We also print two lines for loop calibration failures.  Quiet that down.

BUG=skia:
R=djsollen@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/431503004
2014-08-04 13:57:39 -07:00
mtklein
6e33e232df Skip all .skps in 565.
More SKPs are failing with a 565 target on other platforms:

http://108.170.220.120:10117/builders/Test-Win7-ShuttleA-HD2000-x86-Debug/builds/2938/steps/RunNanobench/logs/stdio

BUG=skia:2797
R=mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/431983006
2014-08-01 08:23:39 -07:00
mtklein
92007583e4 SKPs-as-benches in nanobench
This is meant to replace bench_pictures.

CQ_EXTRA_TRYBOTS=tryserver.skia:Build-Mac10.7-Clang-Arm7-Release-iOS-Trybot

BUG=skia:
R=bsalomon@google.com, jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/425393004
2014-08-01 07:46:53 -07:00
mtklein
e714e75c72 nanobench: support GMs-as-benches
BUG=skia:
R=jcgregorio@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/438683002
2014-07-31 12:13:48 -07:00
bsalomon
2354f8432a Test abandoning GL context in dm/nanobench.
Rename GrContext::contextDestroyed to GrContext::abandonContext.

Remove GrContext::resetContext.

R=robertphillips@google.com

Author: bsalomon@google.com

Review URL: https://codereview.chromium.org/422903002
2014-07-28 13:48:36 -07:00
bsalomon
c2553373ee Make nanobench setup configs outside of loop over benchmarks
R=mtklein@google.com

Author: bsalomon@google.com

Review URL: https://codereview.chromium.org/410683005
2014-07-22 13:09:05 -07:00
caryclark
17f0b6df72 share dm and command flags
Share command flags between dm and unit tests.
Also, allow dm's core to be included by itself and iOSShell.

Command line flags that are the same (or nearly the same) in DM
and in skia_tests have been moved to common_flags. Authors,
please check to see that the shared common flag is correct for
the tool.

For iOS, the 'tool_main' entry point has a wrapper to allow multiple
tools to be statically linked in the iOSShell.
Since SkCommandLineFlags::Parse can only be called once, these calls
are disabled in the IOS build.

Since the iOS app directory is dynamically assigned a name, use '@' to
select it. (This is the same convention chosen by the Mobile Harness
iOS file system utilities.)

Move the heart of dm.gyp into dm.gypi so that it can be included by
itself and iOSShell.gyp.

Add tools/flags/SkCommonFlags.* to define and declare common
command line flags.

Add support for dm to iOSShell.

BUG=skia:
R=scroggo@google.com, mtklein@google.com, jvanverth@google.com, bsalomon@google.com

Author: caryclark@google.com

Review URL: https://codereview.chromium.org/389653004
2014-07-22 10:15:35 -07:00
jcgregorio
05c4560ab3 Fix GLubyte to GrGLubyte.
BUG=skia:
R=mtklein@google.com

Author: jcgregorio@google.com

Review URL: https://codereview.chromium.org/405433008
2014-07-17 13:54:36 -07:00
jcgregorio
bf5e5237b8 Change JSON output of nanobench.
We're moving away from BigQuery for storing results so the output doens't have to conform to BQ requirements, which allows simplifying the format. Also stop parsing the filename for information and pass in buildbot parameters explicitly.

Adds the following flags to nanobench:

  --key
  --gitHash

BUG=skia:
R=mtklein@google.com, bsalomon@google.com

Author: jcgregorio@google.com

Review URL: https://codereview.chromium.org/392393002
2014-07-17 13:14:17 -07:00
mtklein
04d53a5cd5 Revert of nanobench: --veryVerbose for more Win7 debugging (https://codereview.chromium.org/401663002/)
Reason for revert:
Shouldn't be needed anymore.

Original issue's description:
> nanobench: --veryVerbose for more Win7 debugging
>
> BUG=skia:
>
> Committed: https://skia.googlesource.com/skia/+/e57452d

R=mtklein@chromium.org
TBR=mtklein@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=skia:

Author: mtklein@google.com

Review URL: https://codereview.chromium.org/404543004
2014-07-17 11:23:07 -07:00
Mike Klein
e57452debd nanobench: --veryVerbose for more Win7 debugging
BUG=skia:

Review URL: https://codereview.chromium.org/401663002
2014-07-17 13:19:22 -04:00
mtklein
55b0ffc486 nanobench: Protect more against infinite loops.
BUG=skia:
R=mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/404543002
2014-07-17 08:38:23 -07:00
Mike Klein
912947737a Use __rdtsc on Windows.
This seems to be ~100x higher resolution than QueryPerformanceCounter.  AFAIK, all our Windows perf bots have constant_tsc, so we can be a bit more direct about using rdtsc directly: it'll always tick at the max CPU frequency.

Now, the question remains, what is the max CPU frequency to divide through by?  It looks like QueryPerformanceFrequency actually gives the CPU frequency in kHz, suspiciously exactly what we need to divide through to get elapsed milliseconds.  That was a freebie.

I did some before/after comparison on slow benchmarks.  Timings look the same.  Going to land this without review tonight to see what happens on the bots; happy to review carefully tomorrow.

R=mtklein@google.com
TBR=bungeman

BUG=skia:

Review URL: https://codereview.chromium.org/394363003
2014-07-16 19:59:32 -04:00
Mike Klein
e3631364e9 Try to debug Windows infinite loops in nanobench.
NOTREECHECKS=true

BUG=skia:

Review URL: https://codereview.chromium.org/396753004
2014-07-15 17:56:37 -04:00
Mike Klein
3944a1d237 destroyContexts after each bench instead of before
BUG=skia:

Review URL: https://codereview.chromium.org/394893002
2014-07-15 13:40:19 -04:00
mtklein
6238688af0 Give windows boring bars and use 'us' for microseconds.
NOTREECHECKS=true

BUG=skia:
R=bsalomon@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/393673006
2014-07-15 10:30:31 -07:00
mtklein
1e319f7342 Add --resetGpuContext to both DM and nanobench.
Defaulting to true to be conservative for now.

BUG=skia:

NOTREECHECKS=true
R=mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/398483005
2014-07-15 08:27:06 -07:00
mtklein
a189ccdb4d nanobench: add --runOnce.
BUG=skia:
R=egdaniel@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/392583005
2014-07-14 12:28:48 -07:00
mtklein
60317d0ffb nanobench: support --outResultsFile
BUG=skia:
R=kelvinly@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/390933002
2014-07-14 11:30:37 -07:00
mtklein
5d9d10e821 nanobench: add a cute bar chart
Give this a try?  Helpful, or gets in the way?

BUG=skia:
R=krajcevski@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/390483002
2014-07-11 11:57:07 -07:00
mtklein
40b32be371 nanobench: add median and --cpu/--gpu
BUG=skia:
R=krajcevski@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/377283002
2014-07-09 08:46:50 -07:00
mtklein
9bc86ed052 Hey, maybe we should _call_ glFinish.
BUG=skia:

CQ_EXTRA_TRYBOTS=tryserver.skia:Build-Mac10.7-Clang-x86-Debug-Trybot
R=mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/345063004
2014-07-01 10:02:43 -07:00
mtklein
bb6a028239 draft gpu support in nanobench
BUG=skia:
R=bsalomon@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/359473004
2014-07-01 08:43:42 -07:00
mtklein
f372321de3 Add nanobench, a fresh look at our microbenchmark harness.
out/Release/nanobench runs 4-5x faster on my Z620 and ~2x faster on my N5 than out/Release/bench with the same configs.

(Debug isn't really comparable.  I haven't added the Debug -> single loop check.)

BUG=skia:
R=djsollen@google.com, mtklein@google.com

Author: mtklein@chromium.org

Review URL: https://codereview.chromium.org/347443002
2014-06-25 14:08:00 -07:00