Commit Graph

31 Commits

Author SHA1 Message Date
Kevin Lubick
b03b5aca54 Add skpbench job
This entails a few parts:
1. Isolating tools/skpbench/ to be used by the perf task.

2. Adding skpbench to BUILD_PRODUCTS_ISOLATE_WHITELIST

3. Add recipe commands to run skpbench. While it is conceptually similar
to nanobench, it doesn't support images nor svgs, only skps, so we don't
need to copy those to devices or bring them in via CIPD.

4. Add recipe commands to parse skpbench output and upload to Perf.

5. Update gen_tasks.go to handle skpbench jobs and tasks.

BUG=skia:

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=4551

Change-Id: I09bf0b078067279e138e95eca9316e55539ec735
Reviewed-on: https://skia-review.googlesource.com/4551
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2016-11-14 20:34:04 +00:00
Ben Wagner
255095866b PixelC is also TegraX1, but doesn't have GL.
BUG=skia:

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=4397
CQ_INCLUDE_TRYBOTS=master.client.skia.android:Test-Android-Clang-NVIDIA_Shield-GPU-TegraX1-arm64-Release-GN_Android-Trybot,Perf-Android-Clang-PixelC-GPU-TegraX1-arm64-Release-GN_Android-Trybot,Test-Android-Clang-PixelC-GPU-TegraX1-arm64-Debug-GN_Android-Trybot,Test-Android-Clang-PixelC-GPU-TegraX1-arm64-Release-GN_Android-Trybot,Perf-Android-Clang-PixelC-GPU-TegraX1-arm64-Debug-GN_Android-Trybot,Perf-Android-Clang-PixelC-GPU-TegraX1-arm64-Release-GN_Android-Trybot
NOTRY=true

Change-Id: I60a03f2b515cd2a7f1e3a00e4ed33985a5d4347d
Reviewed-on: https://skia-review.googlesource.com/4397
Reviewed-by: Chris Dalton <csmartdalton@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2016-11-04 17:00:00 +00:00
Ben Wagner
5224515ca0 Freshen blacklists.
- GalaxyS3 and GalaxyS4 bots no longer exist.
   - Add gpudft config for all bots, per conversation with bsalomon. We may need to scale this back if the tests take too long to run.
 - Daisy and Tegra2 bots no longer exist.
 - Re-enable interlaced PNGs per conversation with scroggo.
 - Re-enable msaa on iOS per conversation with bsalomon and stephana.

BUG=skia:

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=4311
CQ_INCLUDE_TRYBOTS=master.client.skia.android:Test-Android-Clang-GalaxyS7-GPU-Adreno530-arm64-Debug-GN_Android-Trybot;master.client.skia:Test-iOS-Clang-iPad4-GPU-SGX554-Arm7-Release-Trybot,Test-Win-MSVC-ShuttleC-GPU-iHD530-x86_64-Debug-Trybot

Change-Id: I12d6e1e2182f9706fda6bd88e1fd987dc2bc57cf
Reviewed-on: https://skia-review.googlesource.com/4311
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2016-11-02 19:24:57 +00:00
borenet
96b38426b6 Fix recipes for Gerrit property rename
NOTREECHECKS=true
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2455523004

Review-Url: https://codereview.chromium.org/2455523004
2016-10-27 06:35:52 -07:00
borenet
bfa5b45578 Add key params for buildbotless bots in DM/nanobench
BUG=skia:5626
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2424793005

Review-Url: https://chromiumcodereview.appspot.com/2424793005
2016-10-19 10:13:32 -07:00
Mike Klein
b9eb887f8b Some GN-related recipe cleanup.
All our bots but the iOS ones are on GN now.  This cleans up a bunch of GYP and other obsolete stuff.

  - Nothing's using default_flavor.py any more except as a base class.
  - There are no -CMake, -Shared or -VisualBench bots anymore.
  - Only the iOS bots care about GYP_DEFINES.

You'll see the PDFium bot's GYP_DEFINES change, but that doesn't matter... it's using PDFium's own (GN) build system, and it ignores GYP_DEFINES.

BUG=skia:

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3567

Change-Id: I699e10f013ea77df4dcaa1cb559c51c5bf55dfdb
Reviewed-on: https://skia-review.googlesource.com/3567
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2016-10-18 16:54:25 +00:00
Brian Salomon
f61abec299 Disable ANGLE GL backed on windows perf bots
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=3501

Change-Id: I0e3a166cd07560501cb350b780e264c274c81601
Reviewed-on: https://skia-review.googlesource.com/3501
Commit-Queue: Brian Salomon <bsalomon@google.com>
Reviewed-by: Jim Van Verth <jvanverth@google.com>
2016-10-14 20:11:06 +00:00
bsalomon
11abd8d6cb Explicit control in tools of ANGLE frontend and backend
Update the ANGLE test GL context, GrContextFactory, and config parsing to allow explicit control of ANGLE front/backend.

This will allow us to explicitly test ES2 vs ES3 interfaces to ANGLE as well as D3D9, D3D11, and OpenGL backends.

Also makes the angle api types valid in all builds (but will just fail when SK_ANGLE=1 or not on windows for the d3d backends).

BUG=skia:5804
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2381033002

CQ_INCLUDE_TRYBOTS=master.client.skia:Test-Win-MSVC-ShuttleC-GPU-GTX960-x86_64-Debug-ANGLE-Trybot

Review-Url: https://codereview.chromium.org/2381033002
2016-10-14 08:13:48 -07:00
borenet
2dbbfa5d37 Add infra_tests.py, recipe, buildbotless bot
Have the presubmit (upload) run infra_tests.py if anything in infra/ changed.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2415193002

Review-Url: https://codereview.chromium.org/2415193002
2016-10-14 06:32:09 -07:00
stephana
7fbacbb1e8 Disable svgs on nanobench when running on Valgrind
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2398173002

Review-Url: https://codereview.chromium.org/2398173002
2016-10-07 06:36:17 -07:00
borenet
2f036965f0 Use calendar.timegm instead of time.mktime in recipes
Timezones are hard.

BUG=skia:5823
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2392143004

Review-Url: https://codereview.chromium.org/2392143004
2016-10-05 11:43:41 -07:00
bsalomon
7c2213ba61 Revert of Explicit control in tools of ANGLE frontend and backend (patchset #6 id:90001 of https://codereview.chromium.org/2381033002/ )
Reason for revert:
Broke bots

Original issue's description:
> Explicit control in tools of ANGLE frontend and backend
>
> Update the ANGLE test GL context, GrContextFactory, and config parsing to allow explicit control of ANGLE front/backend.
>
> This will allow us to explicitly test ES2 vs ES3 interfaces to ANGLE as well as D3D9, D3D11, and OpenGL backends.
>
> Also makes the angle api types valid in all builds (but will just fail when SK_ANGLE=1 or not on windows for the d3d backends).
>
> BUG=skia:5804
> GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2381033002
>
> Committed: https://skia.googlesource.com/skia/+/50094fb489543655df026be4e4f99e09e57a1f49

TBR=brianosman@google.com
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=skia:5804

Review-Url: https://codereview.chromium.org/2384483003
2016-09-29 10:07:22 -07:00
bsalomon
50094fb489 Explicit control in tools of ANGLE frontend and backend
Update the ANGLE test GL context, GrContextFactory, and config parsing to allow explicit control of ANGLE front/backend.

This will allow us to explicitly test ES2 vs ES3 interfaces to ANGLE as well as D3D9, D3D11, and OpenGL backends.

Also makes the angle api types valid in all builds (but will just fail when SK_ANGLE=1 or not on windows for the d3d backends).

BUG=skia:5804
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2381033002
NOTREECHECKS=true
NOTRY=true
NOPRESUBMIT=true

Review-Url: https://codereview.chromium.org/2381033002
2016-09-29 09:49:06 -07:00
borenet
8b5ec39095 Include timestamp in nanobench JSON file name
BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2366623003
TBR=jcgregorio

Review-Url: https://codereview.chromium.org/2366623003
2016-09-26 10:25:32 -07:00
mtklein
90939aa22f Run commandbuffer config on CommandBuffer Perf bots.
Uh, so hey, I just noticed that while the CommandBuffer Test bot runs
with --config commandbuffer, the Perf bot just runs as a vanilla GPU bot.
Shouldn't we pass --config commandbuffer?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2364483002

CQ_INCLUDE_TRYBOTS=master.client.skia:Perf-Mac-Clang-MacMini6.2-GPU-HD4000-x86_64-Debug-CommandBuffer-Trybot

Review-Url: https://codereview.chromium.org/2364483002
2016-09-22 06:50:24 -07:00
rmistry
304e6745dc Reland of Run SVGs through nanobench on all Perf bots (patchset #1 id:1 of https://codereview.chromium.org/2348563003/ )
Reason for revert:
skbug/5770 has been fixed.
Trying out trybots to see if this can be relanded.

Original issue's description:
> Revert of Run SVGs through nanobench on all Perf bots (patchset #5 id:120001 of https://codereview.chromium.org/2343903002/ )
>
> Reason for revert:
> Causing failures:
> https://uberchromegw.corp.google.com/i/client.skia/builders/Perf-Mac-Clang-MacMini4.1-GPU-GeForce320M-x86_64-Debug-GN/builds/812
>
> Original issue's description:
> > Run SVGs through nanobench on all Perf bots.
> >
> > Tested with:
> > python infra/bots/recipes.py run --workdir=../ swarm_trigger path_config=kitchen buildername=Perf-Ubuntu-GCC-ShuttleA-GPU-GTX660-x86_64-Release-GN-Trybot mastername=client.skia slavename=skiabot-linux-swarm-010 buildnumber=1 revision=d8c2476a8b1e1e1a1771b17e8dd4db8645914f8c issue=2343903002 rietveld=https://codereview.chromium.org patchset=80001
> >
> > Example output is here:
> > https://luci-milo.appspot.com/swarming/task/3147c9789b2d0410
> > Nanobench did run on SVGs.
> >
> > BUG=skia:5757
> > GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2343903002
> >
> > Committed: https://skia.googlesource.com/skia/+/1c58dcb909642454d91ba36f71e55597511b5d52
>
> TBR=jcgregorio@google.com,fmalita@chromium.org,mtklein@google.com
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=skia:5757
>
> Committed: https://skia.googlesource.com/skia/+/be7e3c0db3caa7a86ae233a6bd26281817b83882

TBR=jcgregorio@google.com,fmalita@chromium.org,mtklein@google.com
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=skia:5757
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2350233002

Review-Url: https://codereview.chromium.org/2350233002
2016-09-19 13:44:25 -07:00
mtklein
122fac3061 GN: remove old Android recipe code.
All the Android builders, testers, and perfers are on gn_android_flavor now.

Not as scary as it looks... all the big line count changes are deletes.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2352653002

Review-Url: https://codereview.chromium.org/2352653002
2016-09-19 07:26:42 -07:00
rmistry
be7e3c0db3 Revert of Run SVGs through nanobench on all Perf bots (patchset #5 id:120001 of https://codereview.chromium.org/2343903002/ )
Reason for revert:
Causing failures:
https://uberchromegw.corp.google.com/i/client.skia/builders/Perf-Mac-Clang-MacMini4.1-GPU-GeForce320M-x86_64-Debug-GN/builds/812

Original issue's description:
> Run SVGs through nanobench on all Perf bots.
>
> Tested with:
> python infra/bots/recipes.py run --workdir=../ swarm_trigger path_config=kitchen buildername=Perf-Ubuntu-GCC-ShuttleA-GPU-GTX660-x86_64-Release-GN-Trybot mastername=client.skia slavename=skiabot-linux-swarm-010 buildnumber=1 revision=d8c2476a8b1e1e1a1771b17e8dd4db8645914f8c issue=2343903002 rietveld=https://codereview.chromium.org patchset=80001
>
> Example output is here:
> https://luci-milo.appspot.com/swarming/task/3147c9789b2d0410
> Nanobench did run on SVGs.
>
> BUG=skia:5757
> GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2343903002
>
> Committed: https://skia.googlesource.com/skia/+/1c58dcb909642454d91ba36f71e55597511b5d52

TBR=jcgregorio@google.com,fmalita@chromium.org,mtklein@google.com
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=skia:5757

Review-Url: https://codereview.chromium.org/2348563003
2016-09-16 12:06:56 -07:00
rmistry
1c58dcb909 Run SVGs through nanobench on all Perf bots.
Tested with:
python infra/bots/recipes.py run --workdir=../ swarm_trigger path_config=kitchen buildername=Perf-Ubuntu-GCC-ShuttleA-GPU-GTX660-x86_64-Release-GN-Trybot mastername=client.skia slavename=skiabot-linux-swarm-010 buildnumber=1 revision=d8c2476a8b1e1e1a1771b17e8dd4db8645914f8c issue=2343903002 rietveld=https://codereview.chromium.org patchset=80001

Example output is here:
https://luci-milo.appspot.com/swarming/task/3147c9789b2d0410
Nanobench did run on SVGs.

BUG=skia:5757
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2343903002

Review-Url: https://codereview.chromium.org/2343903002
2016-09-16 11:32:34 -07:00
mtklein
896ddb706d 565 is not this important.
Reduce its testing and perfing to just the fast Linux bots,
and remove it from the default set of dm/nanobench configs.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2344553002

Review-Url: https://codereview.chromium.org/2344553002
2016-09-14 10:33:13 -07:00
mtklein
df5df81084 GN: Android perf/tests
Just to take inventory of which old problems still linger and which are now moot, I've gone out of my way to have this stand alone.  All of gn_android_flavor's logic is self contained, without any dependency on the platform_tools scripts.

The tricky bits turn out to be, copying directories containing symlinks---or really any copying involving more than one file---and getting the exit code back from adb shell.  Luckily the ADB I've got on my desktop and my Nexus 5x seems to handle this all without the awkward workarounds you see here, so there's hope that One Day Soon the weird parts (basically, anything with inline python) can go away.  Once we've got these bots landed green, I'll go see whether the fixes are due to ADB updates, Android updates, or perhaps something else like hardware.

The parts marked TEMPORARY are a nod to the fact that the devices are used by gn_android_flavor and android_flavor both today.  It's mostly about not stepping on each other's toes or leaving anything laying around that might confuse each other.  The marked parts can go away when bots are either gn_ or non-gn_ but not both.

I have omitted a few steps that may be important, but which are easy independent follow-ups:
  - running as root
  - locking clocks
  - waiting on battery levels
  - fancier wait-for-ready than adb wait-for-usb-device
It'd be nice to, e.g., reaffirm that locking clocks helps perf stability, and that we're locking to the best policy.  I've tried to keep this CL as trim as possible, leaving any of these vaguely optional steps for later.

As of PS 41 or so, it looks like the trybots are all behaving as expected.
We should expect no new images in Gold.  Can we see trybots in Perf yet?

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2320153002

CQ_INCLUDE_TRYBOTS=master.client.skia.android:Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Perf-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-CPU-MT6582-arm-Release-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Debug-GN_Android-Trybot,Test-Android-Clang-AndroidOne-GPU-Mali400MP2-arm-Release-GN_Android-Trybot

Review-Url: https://codereview.chromium.org/2320153002
2016-09-12 12:24:38 -07:00
mtklein
2b3c2a3ff9 GN: add sanitize arg
Attempt to take over all *SAN builds.

MSAN has a lot of coordination required between gn/BUILD.gn and gn_flavor.py.
I'd like to follow up to move more of this into gn/BUILD.gn, to make it easier
to use locally.

The compile steps should be much faster now.  We no longer build CMake
and Clang for every run, instead using the clang_linux CIPD package.  This
removes the need for all the third_party/externals/llvm/... dependencies.

Similarly, since we're using the clang_linux package, we no longer depend
on Chrome's Clang, and thus no longer need to sync chromium on these bots.

Instead of packaging up MSAN libraries and llvm-symbolizer in the compile
output, I have the test / perf bots also depend on the clang_linux package.
These do not vary from build to build.

No more need for the xsan.blacklist -include hack: Clang, GN, and Ninja
all track changes to xsan.blacklist without our help.

This has the incidental effect of upgrading the compiler used by *SAN
bots from Clang 3.8 to Clang 3.9.

BUG=skia:
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2289343002

Review-Url: https://codereview.chromium.org/2289343002
2016-09-08 08:39:34 -07:00
rmistry
3f1c9c0a72 Apply gerrit ref if it is a Gerrit patch.
Also:
* Pass through a new property 'patch_storage' to DM/Nanobench/Coverage. This will be used by the different frameworks to figure out if it is Rietveld or Gerrit issue.
* Calculate issue and patchset for Gerrit patches similar to Rietveld.

BUG=skia:5627
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2263323002

Review-Url: https://codereview.chromium.org/2263323002
2016-08-24 05:07:06 -07:00
rmistry
bcba19dd02 Use SVGs CIPD package and use SVG as a DM source.
Currently only turned on for one bot. May turn it on for more after more conversations.

BUG=skia:5628

GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2231943002

Review-Url: https://codereview.chromium.org/2231943002
2016-08-11 12:36:58 -07:00
borenet
2c3112d517 [recipes] Increase test coverage
Remove a bunch of "pragma: no cover" by removing unused code or adding test bots to cover.

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2215443003

Review-Url: https://codereview.chromium.org/2215443003
2016-08-05 06:30:10 -07:00
borenet
e2251acbaa [recipes] Separate builder_spec into compile/test/perf recipes
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2205473002

Review-Url: https://codereview.chromium.org/2205473002
2016-08-05 05:40:59 -07:00
borenet
d460a3c256 [recipes] Remove build environment vars from default_env
BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2209423002

Review-Url: https://codereview.chromium.org/2209423002
2016-08-05 05:18:06 -07:00
borenet
bc20a701d8 [Recipes] Move test and perf steps into test and perf recipes
This looks like another big change, but I really just moved blocks of code around.  Again, no expectations diffs because no behavior is changed.

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2198973002

Review-Url: https://codereview.chromium.org/2198973002
2016-08-03 10:38:44 -07:00
borenet
1436a09e1f Re-organize Skia recipes
Break Skia recipe module into:
  - skia_vars: defines and stores variables (eg. paths)
  - skia_step: utilities for running Skia steps
  - skia_flavor: flavor-specific stuff
  - skia: top-level setup, checkout, test/perf steps, etc etc

This establishes a saner dependency structure for the recipes; skia_vars
is at the bottom level, skia_step depends on it, skia_flavor depends on
both of them, skia depends on all of the above, and the recipes
themselves may depend on any or all of them.

Next steps:
  - Merge buildbot_spec into skia_vars
  - Move test_steps and perf_steps from skia recipe_module into
    swarm_test and swarm_perf recipes
  - Cleaner checkout_steps process

BUG=skia:5578
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2198173002

Review-Url: https://codereview.chromium.org/2198173002
2016-08-03 08:23:10 -07:00
borenet
f5b675efbe Stop running nanobench on Test.*Debug bots
We'll need to add Perf.*Debug bots to achieve the same coverage.

BUG=skia:4768
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2189393002
NOTREECHECKS=true

Review-Url: https://codereview.chromium.org/2189393002
2016-08-01 05:08:52 -07:00
borenet
1ed2ae45f5 Move Skia recipes from build repo
BUG=skia:5563
GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2175373002

Review-Url: https://codereview.chromium.org/2175373002
2016-07-26 11:52:17 -07:00