Commit Graph

309 Commits

Author SHA1 Message Date
Yuqian Li
228da62fa7 Output calmbench json in nanobench format
Meanwhile, we also make the bot name more similar to the perf bot.

Bug: skia:
Change-Id: I8b0e6aa2d927aa4a0ce57606345589adb6e920b3
Reviewed-on: https://skia-review.googlesource.com/64082
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Yuqian Li <liyuqian@google.com>
2017-10-30 13:52:08 +00:00
Ben Wagner
b2fd61a058 Move most remaining ANGLE bots to Clang.
Sort job names.

Docs-Preview: https://skia.org/?cl=63080
No-Try: true
Change-Id: Ia72c055c4ddb903063b8a4be5ebd6ba92fc8b6d6
Reviewed-on: https://skia-review.googlesource.com/63080
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2017-10-24 18:01:21 +00:00
Yuqian Li
19aaabf8fd Add compiler configuration to calmbench bots
Now the calmbench bot has the same name schema as the build bots
and we're testing using Clang instead of GCC. In the future, we
may also import some configuration schemas from perf bots.

Bug: skia:
Change-Id: Idd9556cff36b749a6c9139d34f70ac50ca48fbba
Reviewed-on: https://skia-review.googlesource.com/62920
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Yuqian Li <liyuqian@google.com>
2017-10-23 17:43:19 +00:00
Mike Klein
30578893c5 add -SafeStack bots
These bots use https://clang.llvm.org/docs/SafeStack.html.

This separates the safe (constant size?) parts of the stack like return
address and some local variables apart from the unsafe over- and
underflow prone array uses.  These go on the "unsafe stack".

In theory this makes sense on {Linux,Mac}x{Debug,Release},
but let's just start with Linux/Debug.

Change-Id: I9f5cbeb7081e7d9ac8e1298ca0d86880a6085cfa
Reviewed-on: https://skia-review.googlesource.com/62743
Reviewed-by: Yuqian Li <liyuqian@google.com>
Commit-Queue: Mike Klein <mtklein@chromium.org>
2017-10-23 15:32:28 +00:00
Kevin Lubick
32f318bad4 Shard GCE Coverage bot to decrease latency
Bug: skia:
Change-Id: I28d28f514b46ad0c30109beba90ae8a1bd8fc4e3
Reviewed-on: https://skia-review.googlesource.com/59961
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
2017-10-17 21:10:17 +00:00
Yuqian Li
c81aaaad0b Try calmbench on bots
Bug: skia:
Change-Id: I869776e3d9d47b4a1a11089f729a0b17581eb9f7
Reviewed-on: https://skia-review.googlesource.com/57400
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Yuqian Li <liyuqian@google.com>
2017-10-16 17:33:40 +00:00
Kevin Lubick
9ef6de7824 Add in test_filter to build configuration
The first "sharding" technique we will try is just by test config
(e.g. 8888, gles, etc).  Thus, for backwards compatibility,
the default "shard" is All, as in all configs

Bug: skia:
NOTRY=true
Change-Id: Ia02362477a5d97f8f74d688b5f0c4f45fc129375
Reviewed-on: https://skia-review.googlesource.com/59563
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-10-16 11:09:49 +00:00
Kevin Lubick
7c1eae5c90 Fix asset version detection
Since we went to retrying the adb commands, the return value
got lost, which was used to read the SKP_VERSIOn, etc.

This fixes it.

Bug: skia:
NOTRY=true
Change-Id: Ifc61ec4cbb13a61c571b2fb89a9c0ed215f8fbd3
Reviewed-on: https://skia-review.googlesource.com/59441
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
2017-10-13 17:20:37 +00:00
Kevin Lubick
02f709e416 Fix Chromebook output step and skpbench
They pull stuff of using scp, which does not perform well
if a directory does not exist.

Skpbench also needs to be updated to use new adb.

Bug: skia:
NOTRY=true
Change-Id: I1e9f2dbddaab05e1aeb4d71d97f1e6e4bfdd299a
Reviewed-on: https://skia-review.googlesource.com/59081
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-10-12 19:18:11 +00:00
Kevin Lubick
587afc9df0 Update to adb 1.0.35
The newer adb acts more like cp when pulling. We used to 
create the dm folder on the host machine before pulling.

This used to be fine, 
/device/dm_out/dm.json -> /host/dm/dm.json

However, with the update, adb would do
/device/dm_out/dm.json -> /host/dm/dm_out/dm.json

This breaks the upload step.  To make this transition
a smooth one, /usr/bin/adb on the RPI is staying the same
(for a while, at least) and /usr/bin/adb.1.0.35 is being added.

That way we can use the new adb on tests after this commit, but
when we backfill, we don't break because of the unexpected folder.

Bug: skia:
Change-Id: Icbed38594fca0e17af1f8d01d75c42ce03f710b9
Reviewed-on: https://skia-review.googlesource.com/58880
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-10-12 17:22:12 +00:00
Ben Wagner
a317a99807 Switch most Win Vulkan jobs to Clang.
Update examples in recipes.

Change-Id: I096f259e8daa46af8626158a820fe226e99cd159
Reviewed-on: https://skia-review.googlesource.com/58200
Reviewed-by: Greg Daniel <egdaniel@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2017-10-11 15:00:10 +00:00
Kevin Lubick
e0240455a8 Add Linux GPU Coverage bot
Adds it for Vulkan and non-Vulkan

Bug: skia:7080
NOTRY=true
Change-Id: I5c4b30b91b96ad6ce9694c42550c4e909de4ba41
Reviewed-on: https://skia-review.googlesource.com/57560
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2017-10-10 15:15:22 +00:00
Kevin Lubick
c795a4c886 Add Linux CPU Coverage Bot
This simply uploads the results of an LLVM coverage to GCS
for later ingestion/display.

Bug: skia:7080
Change-Id: I7dcfa2307a239734a614990aca899ea37129126b
Reviewed-on: https://skia-review.googlesource.com/53880
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-10-09 19:58:34 +00:00
Ben Wagner
9fb285e322 Switch many jobs to Clang.
No-Try: true
Change-Id: Ie255d46ff50d13d25d045791c1c3066f06ab9243
Reviewed-on: https://skia-review.googlesource.com/53601
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
2017-10-03 16:17:19 +00:00
Ben Wagner
5655ba4398 Add Linux nVidia Vulkan jobs.
Change-Id: Ib323136e4c6f660923433a2508e1e0965712d698
Reviewed-on: https://skia-review.googlesource.com/53460
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
2017-10-02 15:08:28 +00:00
Eric Boren
8e0c2c9832 Manually roll recipe DEPS
Bug: skia:
Change-Id: I55641aa4bef5a7ac863e3aae3d2902ef408f0384
Reviewed-on: https://skia-review.googlesource.com/52121
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2017-09-27 17:24:38 +00:00
Ben Wagner
6e0a6b3c40 [recipes] Update example builder names.
Change-Id: If3532bcb5bef5fad8c950d6844135ad3597d2674
Reviewed-on: https://skia-review.googlesource.com/51380
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2017-09-26 20:26:56 +00:00
Eric Boren
6f67cf7c4e [infra] Bring recipe DEPS up to date
Bug: skia:
Change-Id: Iee3a94e56d1c8992f844f2b67d0bb565dd896684
Reviewed-on: https://skia-review.googlesource.com/50262
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-09-22 20:05:38 +00:00
Mike Klein
fddc21547d Set report_signal_unsafe=0 on TSAN bots.
We don't really care about report_signal_unsafe.
Our signal handlers all just dump out a stack trace and crash.
If they crash themselves due to signal unsafe code, no big deal.
(I've never seen that happen.)

Change-Id: Ibb90b2b323d88a56d64e6ea62b9c0924a4e41444
Reviewed-on: https://skia-review.googlesource.com/50120
Reviewed-by: Jim Van Verth <jvanverth@google.com>
Commit-Queue: Mike Klein <mtklein@chromium.org>
2017-09-22 18:48:08 +00:00
Kevin Lubick
ebf648e57a [WASM] Add POC compile bot for WebAssembly
Fix core.gni to use not use Assembler for none cpu.

Right now, there are no outputs because we aren't compiling
dm or nanobench.  However, this still compiles the skia
library and creates two executables, so it's a good canary
for a real WASM build.

Additional note: the two executables in question don't draw
anything to the screen via GL, which is still not possible with
Skia+WASM.

Bug: skia:
Change-Id: I0d767467e94e40d01070e34223dd90e96f1c96f2
Reviewed-on: https://skia-review.googlesource.com/49540
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
Reviewed-by: Eric Boren <borenet@google.com>
2017-09-21 18:24:43 +00:00
Eric Boren
fd4d60eac0 [infra] Rename internal_bot_id -> internal_hardware_label
Bug: skia:
Change-Id: I1a2442898c8eca8cf36d8e85d2acf1d048ebfb54
Reviewed-on: https://skia-review.googlesource.com/46849
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-09-15 14:54:43 +00:00
Eric Boren
053d7a43e5 [infra] Add internal_bot_id flag to recipes
Allows differentiation between internal bots in recipes.

Bug: skia:
Change-Id: I7130f2adc13dd01f7572856ea64c428a6adf56ba
Reviewed-on: https://skia-review.googlesource.com/46844
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-09-15 13:44:14 +00:00
Ben Wagner
adf17dc52f Add new task to run the float-cast-overflow sanitizer.
Bug: skia:4632
No-Try: true
Change-Id: I85a0b23037d5885e5d762788d8bf5b7cc6fc19b2
Reviewed-on: https://skia-review.googlesource.com/45980
Reviewed-by: Mike Klein <mtklein@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
2017-09-13 03:59:40 +00:00
Ben Wagner
79a1256c22 Simpler version of wait-for-device.
This is a followup to https://skia-review.googlesource.com/43564

No-Try: true
Change-Id: I4bc2a24fe08e6b07f5512e3a1e03eac7e3397155
Reviewed-on: https://skia-review.googlesource.com/45120
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
2017-09-12 17:25:30 +00:00
Ben Wagner
cf9365a0bc Retry adb commands on flaky devices.
No-Try: true
Change-Id: I3519131cf064d0a9676abd0fb9d7e946646ce27f
Reviewed-on: https://skia-review.googlesource.com/43564
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2017-09-11 14:09:20 +00:00
Kevin Lubick
5d303ed44b Fix chromecast perfs
Bug: skia:
NOTRY=true
Change-Id: I5efe4bc59efa4bc81e5f8811fed7867ec20447fb
Reviewed-on: https://skia-review.googlesource.com/41541
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
2017-08-31 17:50:01 +00:00
Kevin Lubick
2dafbd796d Add Correctness tests for Chromecast
Bug: skia:
NOTRY=true
Change-Id: Ib604bad8eda6ba4382b850f7c5f318fb0f114401
Reviewed-on: https://skia-review.googlesource.com/20051
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-08-31 14:40:51 +00:00
Kevin Lubick
b40902b6b8 Add upload to task name schema file
Bug: skia:
Change-Id: I65cd5761bb01e648db706878b902e6b6ed67fab9
Reviewed-on: https://skia-review.googlesource.com/33761
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
2017-08-15 11:58:25 +00:00
Eric Boren
bca3b8d8e0 Revert "Bring recipes up to date."
This reverts commit e4024d5dab.

Reason for revert: Breaking Win bots

Original change's description:
> Bring recipes up to date.
> 
>   * Move recipe_engine URL to new canonical URL
>   * Add README.recipes.md (auto-generated doc from docstrings in repo,
>     regenerated with `recipes.py test train`).
>   * Bring depot_tools up to date
> 
> recipe_engine:
> 464fc26 Update a couple more URLs.
> 38808d0 Switch recipes-py CQ to Gerrit
> c7cdc9c Update canonical URL.
> 6bac51f Fix urls to googlesource.
> 5cf5e58 Convert code review to Gerrit
> e92def4 Add source manifest proto to recipe_engine.
> e0ddd60 Move internal downstream recipe roll trybots to production
> 3cae8e9 Add file.filesizes to recipe engine core modules.
> 862f333 Vendor 'luci_context' library.
> 6de5241 Add add_json_log='on_failure' to json module.
> e183af0 [recipe_engine] Set presentation.status before OutputPlaceholder.result.
> 3b99cdd Fix "recipes.py test diff" not raising a failure
> 8e8339a [path] Fix root path mocking.
> 00799cc Add experimental internal downstream recipe trybots
> 0ca88ae Pretty up all recipe_engine documentation.
> de3238c Fix UnboundLocalError in find_recipe.
> 9b47d24 Move downstream recipe roll trybots to production
> da3467b Recursively apply recipe overrides
> 3d76281 [uuid] Remove unused uuid module
> 9959b07 Remove shutil module.
> 26475d6 Add more experimental downstream trybots
> 5c349eb [file] allow glob to handle nested patterns
> 12e7ca4 recipes.py test: enforce recipe expectations literally
> 241ddc2 [file] fix glob_paths when NO paths match, add tests.
> 11d7598 [file] Allow placeholders in file.copy
> 9419a0e [doc] Put all external URLs in a single section.
> 4b6b21e [doc] Fix randomization of DEPS link order, give DEPS links fully qualified names
> 3ac778e [recipe_autoroller] regen docs on trivial rolls too.
> c8d78bf [doc] Automatically generate and update cross-indexed recipe markdown files.
> 5873ef2 Fix "env_prefixes" rendering.
> 
> depot_tools:
> fb3fa785 Roll recipe dependencies (trivial).
> a643e9e3 Update patch_projects crap for googlesource repos in infra.
> fb391fcd [gclient config] Update luci project URLs (ugh)
> 590a4189 Roll recipe dependencies (trivial).
> 8e6a5e6e Roll recipe dependencies (trivial).
> 0e5f57ec [recipes.cfg] Update recipes.cfg to new canonical recipe URL.
> 704d6ad8 Roll recipe dependencies (trivial).
> f1205599 Roll recipe dependencies (trivial).
> 243c9643 Roll recipe dependencies (trivial).
> e866182f depot_tools: Fix executable bit for gerrit_*.py
> 4759fc62 Roll recipe dependencies (trivial).
> 7ab47354 Roll recipe dependencies (trivial).
> 554e2bff Roll recipe dependencies (trivial).
> 9e8c78d2 Roll recipe dependencies (trivial).
> b0ad16ee gclient flatten: do not recurse into deps not in recursedeps
> d9f97bc5 Use https for short URLs wherever applicable (MoarTLS).
> c9353605 gclient flatten: fixes for chromium/src
> ecdfa41d Fix quoting in update_depot_tools.
> 1407d00d gclient: make variables from parent DEPS scope override ones in inner scope
> 07a6bea9 Expose ast via InputApi
> e864a3f2 Remove presubmit check for DISALLOW_* macros
> e1054034 fetch_end_to_end_test: add recipe
> 8b478f04 PRESUBMIT: Be resilient to changes with no Code-Review label
> 8038dfe3 bot_update recipe: parameterize step name in apply_gerrit_ref
> 5aeeb72c [bootstraps] Sync at gclient, suppress output (#2)
> eb1feb90 [cipd] Bump version, log at warning (#2).
> 9f9d82ac Revert "[cipd] Bump version, log at warning."
> 18aa2d93 [cipd] Bump version, log at warning.
> fc6196b3 gclient flatten: preserve variable placeholders (reland #1)
> 5aa5cd76 depot_tools: add gerrit repo and ref to apply_gerrit_ref()
> 56766f1d Add experimental downstream recipe trybots to depot_tools
> 44855b94 Revert "[bootstraps] Sync at gclient, suppress output."
> 393ba066 Revert "gclient flatten: preserve variable placeholders"
> e79ddeaa gclient flatten: preserve variable placeholders
> 5a80eab0 Use -prerelease flag to vswhere when packaging VS
> 4d92fe43 [bootstraps] Sync at gclient, suppress output.
> 357415cb gclient flatten: fix a bug with some recursedeps not being processed
> 3d6363b3 Roll recipe dependencies (trivial).
> b396315c Bump led version.
> a3a515a4 Revert "Generalize git index refresh to bot_update."
> 06e1aa07 Generalize git index refresh to bot_update.
> 7f12a2e4 [bootstrap/win] Roll bleeding edge entry bat.
> f7543cdd git-cl: accept --force for -f everywhere
> 642dd84c [win/bootstrap] Use "for" instead of "set".
> ca01e2c8 git cl issue 0: Remove Change-Id from message, not description
> 290f5f56 Remove "create-ntfs-junction".
> ff9a5dec Roll recipe dependencies (trivial).
> d2dbea66 Roll recipe dependencies (trivial).
> fc62f768 Simplify notify behavior in git-cl
> cfbb775f Roll recipe dependencies (trivial).
> ae895075 Use doseq=True when re-encoding url query parameters
> 98681d11 Remove obsolete destination-branch-check flag
> 8aa774cf Fix bot_update example
> dd1bc7c0 [cipd] Generate other platform suffixes.
> 383ace4b Fix git-crrev-parse on Mac.
> e5adf612 Fix typo with --send-email [sic] flags
> 0ed616a4 Roll recipe dependencies (trivial).
> 636b13fc Let Changelist().AddComment() mark changes as Ready
> 68b54e78 [bootstrap/win] Upgrade Python to 2.7.13.
> f3d5a114 Enable Gerrit-destination-branch check by default
> 31f3e63c Revert "Revert "[tryserver] Remove unused methods.""
> c4dd3e82 [bootstrap/win] Remove support for legacy installs
> cc27ecb0 Revert "[tryserver] Remove unused methods."
> 68f6f279 [bootstrap/win] Always use "cipd.bat".
> 3a8717e1 [bootstrap/win] Remove legacy Git support.
> 6bac8a84 GitHyperBlameMainTest.testBadFilename: Work around git-blame bug.
> 4e5207d6 git-cl issue 0: Don't operate on empty description
> 22a9cf54 git_footers: fix bug stdin-based message input.
> 02e59414 Fix bot_update's usage of the destination branch
> 83eb1cd5 Roll recipe dependencies (trivial).
> 400e989b 'git cl issue 0': Remove Change-Id
> 133ac1ab [tryserver] Remove unused methods.
> b08ba657 git_footer: be more resilient to malformed footers
> 9d842bab [recipe_modules/git] Remove logging.
> 3d9b5881 [win_tools] Use bundled Python CIPD packages (#2).
> 209a681f [git] Remove hard-coded "depot_tools" Git.
> e8eed65f Always copy "python.bat".
> aab09ca6 Whitelist "python_bin_reldir.txt".
> af5c20f5 Revert "[win_tools] Use bundled Python CIPD packages."
> 7dadf05d Query Gerrit for destination branch in bot_update
> bf144679 [win_tools] Use bundled Python CIPD packages.
> b80fac66 Roll recipe dependencies (trivial).
> 265255f5 [bootstrap/win] Replace "python.bat" only on downgrade.
> 24025d3e tryserver: add support for gerrit footers
> 903ea731 [activity.py] Identify Gerrit-style bug annotations in commit messages
> 49b8147b Add support to query a CL's branch to gerrit recipe module
> 4cb9d742 [bootstrap/win] Enable revert from upcoming patch.
> 13e88538 Ignore the ".bleeding_edge" file.
> c8173825 git cl creds-check: improve report and give better recommendation.
> 4978917c Adjust CQ config for luci.infra.try -> luci.infra.depot_tools.try bucket move
> 9cb802ca Improve gerrit recipe_module example
> 79e897b0 [git_bootstrap] Support new Git bundles.
> 8af46469 Roll recipe dependencies (trivial).
> 1f767e17 Fix autoninja.py for ninja -t compatibility
> 7466d1a9 [cipd] Run client in .bat to preserve STDIN.
> 13f258bf Remove SVN bootstrapping and some tooling.
> b12c5256 Roll recipe dependencies (trivial).
> 016f92e6 Roll recipe dependencies (trivial).
> 68506985 Roll recipe dependencies (trivial).
> 215ded77 Fix testFlattenPinAllDeps
> 271a1680 gclient flatten: implement --pin-all-deps
> fb022017 gclient flatten: include vars
> e36a8431 Roll recipe dependencies (trivial).
> de37c012 Fix git-numberer swallowing of footers for Rietveld.
> 4e89144a git-cl: fix auth error message computation
> 7c2e05b4 [cipd] Fix CIPD bootstraps run concurrently.
> aaf93f48 gclient flatten: refactor flatten code to a class
> 807caf84 gclient flatten: remove --require-pinned-revisions
> 6fbf12f2 Enable query for changes in gerrit recipe module
> c6846aa2 gclient flatten: fix a bug with deps_os getting omitted for recursed deps
> 4baaa11c gclient: avoid adding potentially duplicate entries to work queue
> d3790251 gclient: apply custom_vars when evaluating conditions
> a8fdab96 Roll recipe dependencies (trivial).
> e82c0dec [cipd] Fail if setup has an error.
> b1ad2f04 Update CIPD client, vpython and "LUCI editor".
> 6a1ab843 Roll recipe dependencies (trivial).
> 32f0e5d7 Roll recipe dependencies (trivial).
> bc9c5ccf Roll recipe dependencies (trivial).
> 54adb2a3 Generate initial recipes README.
> 603545b7 Roll recipe dependencies (trivial).
> e015878d gclient flatten: fix an issue with an deps_os entry getting added to deps
> 6014b566 gclient flatten: also handle allowed_hosts
> 70b3f3aa [bot_update] Pass spec as file.
> 1af0a337 Roll recipe dependencies (trivial).
> dc598e82 [bot_update] Prefix PATH, cleanup.
> 
> Bug: skia:
> Change-Id: I0eea2dd126c130299ac39fe9b54a714cb0612563
> Reviewed-on: https://skia-review.googlesource.com/32420
> Reviewed-by: Eric Boren <borenet@google.com>
> Commit-Queue: Eric Boren <borenet@google.com>

TBR=borenet@google.com,iannucci@google.com

Change-Id: I6ed89db0757ffeb366da96d655372e9710aa589e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:
Reviewed-on: https://skia-review.googlesource.com/32520
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-08-09 13:03:15 +00:00
Robert Iannucci
e4024d5dab Bring recipes up to date.
* Move recipe_engine URL to new canonical URL
  * Add README.recipes.md (auto-generated doc from docstrings in repo,
    regenerated with `recipes.py test train`).
  * Bring depot_tools up to date

recipe_engine:
464fc26 Update a couple more URLs.
38808d0 Switch recipes-py CQ to Gerrit
c7cdc9c Update canonical URL.
6bac51f Fix urls to googlesource.
5cf5e58 Convert code review to Gerrit
e92def4 Add source manifest proto to recipe_engine.
e0ddd60 Move internal downstream recipe roll trybots to production
3cae8e9 Add file.filesizes to recipe engine core modules.
862f333 Vendor 'luci_context' library.
6de5241 Add add_json_log='on_failure' to json module.
e183af0 [recipe_engine] Set presentation.status before OutputPlaceholder.result.
3b99cdd Fix "recipes.py test diff" not raising a failure
8e8339a [path] Fix root path mocking.
00799cc Add experimental internal downstream recipe trybots
0ca88ae Pretty up all recipe_engine documentation.
de3238c Fix UnboundLocalError in find_recipe.
9b47d24 Move downstream recipe roll trybots to production
da3467b Recursively apply recipe overrides
3d76281 [uuid] Remove unused uuid module
9959b07 Remove shutil module.
26475d6 Add more experimental downstream trybots
5c349eb [file] allow glob to handle nested patterns
12e7ca4 recipes.py test: enforce recipe expectations literally
241ddc2 [file] fix glob_paths when NO paths match, add tests.
11d7598 [file] Allow placeholders in file.copy
9419a0e [doc] Put all external URLs in a single section.
4b6b21e [doc] Fix randomization of DEPS link order, give DEPS links fully qualified names
3ac778e [recipe_autoroller] regen docs on trivial rolls too.
c8d78bf [doc] Automatically generate and update cross-indexed recipe markdown files.
5873ef2 Fix "env_prefixes" rendering.

depot_tools:
fb3fa785 Roll recipe dependencies (trivial).
a643e9e3 Update patch_projects crap for googlesource repos in infra.
fb391fcd [gclient config] Update luci project URLs (ugh)
590a4189 Roll recipe dependencies (trivial).
8e6a5e6e Roll recipe dependencies (trivial).
0e5f57ec [recipes.cfg] Update recipes.cfg to new canonical recipe URL.
704d6ad8 Roll recipe dependencies (trivial).
f1205599 Roll recipe dependencies (trivial).
243c9643 Roll recipe dependencies (trivial).
e866182f depot_tools: Fix executable bit for gerrit_*.py
4759fc62 Roll recipe dependencies (trivial).
7ab47354 Roll recipe dependencies (trivial).
554e2bff Roll recipe dependencies (trivial).
9e8c78d2 Roll recipe dependencies (trivial).
b0ad16ee gclient flatten: do not recurse into deps not in recursedeps
d9f97bc5 Use https for short URLs wherever applicable (MoarTLS).
c9353605 gclient flatten: fixes for chromium/src
ecdfa41d Fix quoting in update_depot_tools.
1407d00d gclient: make variables from parent DEPS scope override ones in inner scope
07a6bea9 Expose ast via InputApi
e864a3f2 Remove presubmit check for DISALLOW_* macros
e1054034 fetch_end_to_end_test: add recipe
8b478f04 PRESUBMIT: Be resilient to changes with no Code-Review label
8038dfe3 bot_update recipe: parameterize step name in apply_gerrit_ref
5aeeb72c [bootstraps] Sync at gclient, suppress output (#2)
eb1feb90 [cipd] Bump version, log at warning (#2).
9f9d82ac Revert "[cipd] Bump version, log at warning."
18aa2d93 [cipd] Bump version, log at warning.
fc6196b3 gclient flatten: preserve variable placeholders (reland #1)
5aa5cd76 depot_tools: add gerrit repo and ref to apply_gerrit_ref()
56766f1d Add experimental downstream recipe trybots to depot_tools
44855b94 Revert "[bootstraps] Sync at gclient, suppress output."
393ba066 Revert "gclient flatten: preserve variable placeholders"
e79ddeaa gclient flatten: preserve variable placeholders
5a80eab0 Use -prerelease flag to vswhere when packaging VS
4d92fe43 [bootstraps] Sync at gclient, suppress output.
357415cb gclient flatten: fix a bug with some recursedeps not being processed
3d6363b3 Roll recipe dependencies (trivial).
b396315c Bump led version.
a3a515a4 Revert "Generalize git index refresh to bot_update."
06e1aa07 Generalize git index refresh to bot_update.
7f12a2e4 [bootstrap/win] Roll bleeding edge entry bat.
f7543cdd git-cl: accept --force for -f everywhere
642dd84c [win/bootstrap] Use "for" instead of "set".
ca01e2c8 git cl issue 0: Remove Change-Id from message, not description
290f5f56 Remove "create-ntfs-junction".
ff9a5dec Roll recipe dependencies (trivial).
d2dbea66 Roll recipe dependencies (trivial).
fc62f768 Simplify notify behavior in git-cl
cfbb775f Roll recipe dependencies (trivial).
ae895075 Use doseq=True when re-encoding url query parameters
98681d11 Remove obsolete destination-branch-check flag
8aa774cf Fix bot_update example
dd1bc7c0 [cipd] Generate other platform suffixes.
383ace4b Fix git-crrev-parse on Mac.
e5adf612 Fix typo with --send-email [sic] flags
0ed616a4 Roll recipe dependencies (trivial).
636b13fc Let Changelist().AddComment() mark changes as Ready
68b54e78 [bootstrap/win] Upgrade Python to 2.7.13.
f3d5a114 Enable Gerrit-destination-branch check by default
31f3e63c Revert "Revert "[tryserver] Remove unused methods.""
c4dd3e82 [bootstrap/win] Remove support for legacy installs
cc27ecb0 Revert "[tryserver] Remove unused methods."
68f6f279 [bootstrap/win] Always use "cipd.bat".
3a8717e1 [bootstrap/win] Remove legacy Git support.
6bac8a84 GitHyperBlameMainTest.testBadFilename: Work around git-blame bug.
4e5207d6 git-cl issue 0: Don't operate on empty description
22a9cf54 git_footers: fix bug stdin-based message input.
02e59414 Fix bot_update's usage of the destination branch
83eb1cd5 Roll recipe dependencies (trivial).
400e989b 'git cl issue 0': Remove Change-Id
133ac1ab [tryserver] Remove unused methods.
b08ba657 git_footer: be more resilient to malformed footers
9d842bab [recipe_modules/git] Remove logging.
3d9b5881 [win_tools] Use bundled Python CIPD packages (#2).
209a681f [git] Remove hard-coded "depot_tools" Git.
e8eed65f Always copy "python.bat".
aab09ca6 Whitelist "python_bin_reldir.txt".
af5c20f5 Revert "[win_tools] Use bundled Python CIPD packages."
7dadf05d Query Gerrit for destination branch in bot_update
bf144679 [win_tools] Use bundled Python CIPD packages.
b80fac66 Roll recipe dependencies (trivial).
265255f5 [bootstrap/win] Replace "python.bat" only on downgrade.
24025d3e tryserver: add support for gerrit footers
903ea731 [activity.py] Identify Gerrit-style bug annotations in commit messages
49b8147b Add support to query a CL's branch to gerrit recipe module
4cb9d742 [bootstrap/win] Enable revert from upcoming patch.
13e88538 Ignore the ".bleeding_edge" file.
c8173825 git cl creds-check: improve report and give better recommendation.
4978917c Adjust CQ config for luci.infra.try -> luci.infra.depot_tools.try bucket move
9cb802ca Improve gerrit recipe_module example
79e897b0 [git_bootstrap] Support new Git bundles.
8af46469 Roll recipe dependencies (trivial).
1f767e17 Fix autoninja.py for ninja -t compatibility
7466d1a9 [cipd] Run client in .bat to preserve STDIN.
13f258bf Remove SVN bootstrapping and some tooling.
b12c5256 Roll recipe dependencies (trivial).
016f92e6 Roll recipe dependencies (trivial).
68506985 Roll recipe dependencies (trivial).
215ded77 Fix testFlattenPinAllDeps
271a1680 gclient flatten: implement --pin-all-deps
fb022017 gclient flatten: include vars
e36a8431 Roll recipe dependencies (trivial).
de37c012 Fix git-numberer swallowing of footers for Rietveld.
4e89144a git-cl: fix auth error message computation
7c2e05b4 [cipd] Fix CIPD bootstraps run concurrently.
aaf93f48 gclient flatten: refactor flatten code to a class
807caf84 gclient flatten: remove --require-pinned-revisions
6fbf12f2 Enable query for changes in gerrit recipe module
c6846aa2 gclient flatten: fix a bug with deps_os getting omitted for recursed deps
4baaa11c gclient: avoid adding potentially duplicate entries to work queue
d3790251 gclient: apply custom_vars when evaluating conditions
a8fdab96 Roll recipe dependencies (trivial).
e82c0dec [cipd] Fail if setup has an error.
b1ad2f04 Update CIPD client, vpython and "LUCI editor".
6a1ab843 Roll recipe dependencies (trivial).
32f0e5d7 Roll recipe dependencies (trivial).
bc9c5ccf Roll recipe dependencies (trivial).
54adb2a3 Generate initial recipes README.
603545b7 Roll recipe dependencies (trivial).
e015878d gclient flatten: fix an issue with an deps_os entry getting added to deps
6014b566 gclient flatten: also handle allowed_hosts
70b3f3aa [bot_update] Pass spec as file.
1af0a337 Roll recipe dependencies (trivial).
dc598e82 [bot_update] Prefix PATH, cleanup.

Bug: skia:
Change-Id: I0eea2dd126c130299ac39fe9b54a714cb0612563
Reviewed-on: https://skia-review.googlesource.com/32420
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-08-09 12:49:04 +00:00
Ravi Mistry
0ac06e4726 Bot updates for SkiaCT tasks.
Use gcc-4.8 when compiling for Ubuntu14. Rename the CT tasks to make it
clear their child tasks run on Ubuntu14.

Move to Debian-9.1 bots created with the changes in
https://skia-review.googlesource.com/c/31281 and
https://skia-review.googlesource.com/c/31820

Change the cache dir used by the ct_skps recipe so that changing the
bot name doesn't invalidate the cache.

Bug: skia:6921
Bug: skia:6818
No-Try: true
Change-Id: Idaac8187427b5c2c7bffe66ad6674496b8cd0c0e
Reviewed-on: https://skia-review.googlesource.com/31242
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2017-08-08 14:17:49 +00:00
Ravi Mistry
698edfecef Chromium lkgr is no longer updated. Use lkcr
Bug: skia:6916
Change-Id: I16badf80c3b34e517b8baab161150c9434f325aa
Reviewed-on: https://skia-review.googlesource.com/30100
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-08-03 12:41:36 +00:00
Mike Klein
8e3c42b1ee add a Win/Clang build bot
BUG=skia:2679

CQ_INCLUDE_TRYBOTS=skia.primary:Build-Win-Clang-x86_64-Release

Change-Id: I861366ae1340abcc67e87041904d08337b465bbe
Reviewed-on: https://skia-review.googlesource.com/28864
Reviewed-by: Eric Boren <borenet@google.com>
2017-07-31 19:53:51 +00:00
Greg Daniel
293d9bd3c2 Add Metal build bot
Bug: skia:6896
Change-Id: If2ed083703e6300b9d2a86cfb298011bbf54e018
Reviewed-on: https://skia-review.googlesource.com/28180
Reviewed-by: Eric Boren <borenet@google.com>
2017-07-28 18:41:43 +00:00
Eric Boren
f7928b4f33 Add a bot to verify that generated SKSL files have not been hand-edited
Bug: skia:6893
Change-Id: I54480ddd1ad1989661393500044f9164a99ce7b0
Reviewed-on: https://skia-review.googlesource.com/27700
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-07-28 12:20:47 +00:00
Eric Boren
fe40abc14b Add new Android builder which uses API_LEVEL 26
Bug: skia:6870
Change-Id: Idd9a27518c1123c0a729b4d07461047f5530aae3
Reviewed-on: https://skia-review.googlesource.com/24960
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
2017-07-20 18:25:12 +00:00
Leon Scroggins III
51b2f1b64c Stop adding Android-specific defines to SkUserConfig
Remove #defines that lived in gn_to_bp.py and
android_framework_defines.gni. These have been moved into a new file
in Android, SkUserConfigManual.h, in https://googleplex-android-review.git.corp.google.com/#/c/2519600/

Update gn_to_bp.py to include SkUserConfigManual.h, so it will still
result in using the same #defines.

Lately, we've found it difficult to guard changes behind a flag. e.g.
a change to drawing causes a CTS failure in Android, so we have to do
the following:
- put the change behind a flag, and add it to gn_to_bp.py or
android_framework_defines.gni
- generate new images on Android (by running CTS with external/skia
modified to not define the flag)
- create a CL in CTS that uses the new images
- land a CL in Skia that stops defining the flag
- when the Skia change lands, wait for the auto-roller to create a CL
that includes the change, stop the auto-roller, add the topic to the CTS
CL so the two can land at the same time
- land both Android changes (with TreeHugger)
- restart the Android auto-roller

With SkUserConfigManual.h (which lives in Android), the process will
be similar to Chromium:
- land a CL in Android's external/skia that defines a flag e.g.
SK_SUPPORT_LEGACY_FEATURE. Land without TreeHugger because it isn't used
in Skia and does not do anything
- land a change in Skia that changes behavior unless
SK_SUPPORT_LEGACY_FEATURE is defined. This will safely go through the
Android roll and not change any behavior for Android
- create two Android CLs - one in CTS to use the new images, and one in
external/skia to delete SK_SUPPORT_LEGACY_FEATURE. Set them to the same
topic and land them with TreeHugger

In the new process, there is no need to mess with the Android roll.

A downside to the new process is that we cannot test the android
framework defines without checking in to Android. But given how much
we've progressed in automating Android testing, this is fine.

Bug: b/63429612
Change-Id: Idfbaef2f4cae641a75fb6e7bf70428733a441336
Reviewed-on: https://skia-review.googlesource.com/22072
Commit-Queue: Leon Scroggins <scroggo@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-07-11 20:43:35 +00:00
Robert Iannucci
8cd50418af Remove dependency on recipe_engine/shutil.
R=borenet@google.com

Bug: chromium:739984
Change-Id: I077c4e6f494eb7e93851201120048ba1ef5faa07
Reviewed-on: https://skia-review.googlesource.com/21921
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-07-10 13:46:35 +00:00
Brian Salomon
0c26a9dbd0 GR_TEST_UTILS fixes
Fixes build of Skia lib when GR_TEST_UTILS=0

Makes GR_TEST_UTILS=0 for official builds

Makes "Mini" builder bot exercise building GPU with is_official_build=true

Bug: skia:6786
Change-Id: I6186683a3a216d2e779645bd9e8276a66bcff4d5
Reviewed-on: https://skia-review.googlesource.com/21524
Commit-Queue: Brian Salomon <bsalomon@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2017-07-06 14:35:45 +00:00
Eric Boren
89cd35795f Fix recipes/tests for new Debian bots
Bug: skia:6612
Change-Id: I2920bb7b1aa3ebb021223ed6345f13f1f9c8bc24
Reviewed-on: https://skia-review.googlesource.com/21141
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-06-28 18:46:16 +00:00
Eric Boren
3e2ffd74b5 [recipes] Switch to file module from recipe_engine
Bug: skia:6704
Change-Id: Id3cefa643316e5676d2038a75240e16d929d6d63
Reviewed-on: https://skia-review.googlesource.com/20146
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2017-06-16 17:41:28 +00:00
Ben Wagner
1791a064f1 [recipes] Fix internal_test tryjobs.
>>> 'internal_test.git'.rstrip('.git')
'internal_tes'

Change-Id: If023784a7387a89b2ba1972ea43d68a0c07df635
Reviewed-on: https://skia-review.googlesource.com/19546
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
2017-06-13 15:18:22 +00:00
recipe-roller
a6e46865ac Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
depot_tools:
  https://crrev.com/59a74ed5585087e85ff4b7fa2ac18ef3648d05b0 Add cleanup dir, use with bot_update. (dnj@chromium.org)


Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9021158a90cb8ba0c939a45c2d480271506da65b
Reviewed-on: https://skia-review.googlesource.com/19099
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-06-12 11:44:40 +00:00
Eric Boren
f4a5fc7af0 Use chrome-infra Git CIPD package
Add Git recipe module to easily use this version of Git anywhere.
This fixes recipe bundling and unblocks the recipe roll.

Bug: skia:
Change-Id: Ib4d1361b7a52676e1992025b29e630ea3ada173b
Reviewed-on: https://skia-review.googlesource.com/18833
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-06-06 12:45:11 +00:00
Robert Iannucci
dec4c0b298 Rename example.py -> examples/full.py for all modules
`example.py` is going away; examples folder can now hold multiple
example recipes.

R=borenet@google.com

Bug: skia:
Change-Id: Ib28910987342217bc52040d4d966444174ce27a3
Reviewed-on: https://skia-review.googlesource.com/18432
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-06-02 11:40:36 +00:00
Kevin Lubick
8692b4a5ac Use ssh to trigger chromecast test
Using adb seemed to ignore errors in some cases, primarily
if nanobench crashed/hung.

Bug: skia:6706
Change-Id: I5def49d5b4d20bed6c486b7e85040190be856ac8
Reviewed-on: https://skia-review.googlesource.com/18404
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-06-01 20:11:47 +00:00
recipe-roller
e0f7e164ee Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
depot_tools:
  https://crrev.com/fc1c6f4c1fac4de7647da83425f4398dde6e77ca Correct Git show's path format on Windows (manzagop@chromium.org)
recipe_engine:
  https://crrev.com/458ce956c9d91d184e0107c5fa6b34a8267b8e9b [step_runner] run _merge_envs in simulation too. (iannucci@chromium.org)


R=martiniss@chromium.org, tansell@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ic938b59f304189fdfb78a466e6a4a0e47f0548a6
Reviewed-on: https://skia-review.googlesource.com/18300
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Robbie Iannucci <iannucci@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2017-06-01 19:21:14 +00:00
Robert Iannucci
baf41bd1c8 Convert last step.context stragglers.
These methods are removed in an upcoming version of the recipe engine.

R=borenet@google.com

Bug: skia:
Change-Id: I2db5cc683dac35b9ce65fe1b175e9f9a6d6b5efe
Reviewed-on: https://skia-review.googlesource.com/18324
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-06-01 11:39:57 +00:00
Eric Boren
d696df74d4 Add CIPD asset for Valgrind
Bug: skia:6668
Change-Id: I324602b1381f701ec780c02b26775c5e9e09cb84
Reviewed-on: https://skia-review.googlesource.com/17834
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
2017-05-31 19:41:13 +00:00
Eric Boren
30095a1166 [recipes] Try removing the possibly-over-conservative git step
Bug: skia:
Change-Id: Ie6ea54d26eff7a165f32afbb5c8037b6c1921a5d
Reviewed-on: https://skia-review.googlesource.com/18138
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2017-05-31 11:51:57 +00:00