Commit Graph

152 Commits

Author SHA1 Message Date
Stephan Altmueller
88df8d2e5a Test SKQP on Firebase Testlab and Upload
- adds building the testlab driver (run_testlab) as a separate step
- adds gcloud isolate necessary to run testlab
- adds Testlab support and uploading a verified AKP to GCS (with
  meta data attached).

Bug: skia:
Change-Id: I1bf265f46c99360eb3a9eb684886f93de48085fe
Reviewed-on: https://skia-review.googlesource.com/111603
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Stephan Altmueller <stephana@google.com>
2018-03-07 20:44:50 +00:00
Hal Canary
8080a6e705 SkQP: Host APK list on cloud; no more commits
No-Try: true
Change-Id: I7f77c828cba2aa3763d495bec86d016eb65cb485
Reviewed-on: https://skia-review.googlesource.com/111741
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-03-02 16:54:50 +00:00
Hal Canary
84fdd4fcba SkQP: optionally build make_universal_apk with debug on.
* Respects SKQP_DEBUG=true.
  * Also, be more selective about cleaning. (So I can test on master.)
  * Also, stop using `gradlew --daemon`, which was getting confused
    when switching back and forth build types.

No-Try:true
Change-Id: Id93f77ab7831a82cad9a3a64dbb8a93af4757d57
Reviewed-on: https://skia-review.googlesource.com/108222
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-02-27 20:42:29 +00:00
Hal Canary
65a4178ff7 SkQP: do test filtering correctly
PLEASE NOTE: Instructions for running a single test have changed!

No-Try: true
Change-Id: I1923240e879daa7ff0556737ddd5aa3f58e0097c
Reviewed-on: https://skia-review.googlesource.com/109566
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
2018-02-23 15:57:46 +00:00
Hal Canary
f637cc01f8 SkQP: run a single test
PLEASE NOTE:  Instructions on running `am instrument` for the
SkQP APK have changed.

To run a single test, see the section "Running a single test"
in `tools/skqp/README.md`.

No-Try: true
Change-Id: I0a2cbc47755929d6c6a927a3591ff98046779c77
Reviewed-on: https://skia-review.googlesource.com/108780
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-21 20:49:56 +00:00
Ben Wagner
483c772cfd Add nativeFonts flag to Viewer.
This moves DMFontMgr and several related files which are tightly related
to fonts into tools/fonts, moves some flags around to prevent
duplication, and adds the nativeFonts handling to Viewer.

Change-Id: Id1bdad708a6b74319ac5ac9adfe21025db4ca0b2
Reviewed-on: https://skia-review.googlesource.com/108904
Commit-Queue: Ben Wagner <bungeman@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
2018-02-21 03:31:14 +00:00
Hal Canary
24889c90d2 SkQP: update README.md
No-Try: true
Change-Id: I7b12f683901ece91e3a7e2d9fa432aa05fbcf12a
Reviewed-on: https://skia-review.googlesource.com/108568
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-20 17:58:43 +00:00
Hal Canary
c126191fd1 SkQP: debug option, and fix a bug
Change-Id: I642c65815fdab5adcd7735956e1955ede8ff229a
Reviewed-on: https://skia-review.googlesource.com/106805
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-13 16:16:50 +00:00
Hal Canary
a58bcf7ef8 SkQP: Slimmer APKS
Currently produces 30% smaller APKs.

No-Try: true
Change-Id: Ic04b4bebef78e9a3f8b8dde4bace78ce74e43d66
Reviewed-on: https://skia-review.googlesource.com/106165
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
2018-02-12 17:05:22 +00:00
Hal Canary
c87fbf93d0 Android Tools: build x86_64 correctly.
- Fix some naming confusion between `x64` and `x86-64`.
  - SkQP Universal (multi-arch apk) Build Script
    [tools/skqp/make_universal_apk]
  - skqp/build.gradle: add `universal` product flavor
  - skqp manifest:  make debug build optional.

No-Try: true
Change-Id: Ic8cd88719a34786ab2d53771749b2beb9db98be5
Reviewed-on: https://skia-review.googlesource.com/105023
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-02-09 22:13:23 +00:00
Mike Reed
7fcfb62199 move a bunch of helpers from SkImageInfo.h into priv
Bug: skia:
Change-Id: I8c91cfdb89e4f22448d1201d391556fe43d86dca
Reviewed-on: https://skia-review.googlesource.com/105289
Commit-Queue: Mike Reed <reed@google.com>
Reviewed-by: Mike Klein <mtklein@chromium.org>
Reviewed-by: Cary Clark <caryclark@google.com>
2018-02-09 20:38:32 +00:00
Hal Canary
703d9c47a8 SkQP: download_model complains when files.checksum is missing
Change-Id: I4fad015181f15e4ffb9c9607723a1313a041621b
Reviewed-on: https://skia-review.googlesource.com/103763
Reviewed-by: Stephan Altmueller <stephana@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-06 19:48:55 +00:00
Derek Sollenberger
82831b18a6 Add a global error tolerance of 4 to the SkQP Android.bp build file
Change-Id: Iceebd0e153351cad1c2784daf6cabcc8f3bb8ebf
Reviewed-on: https://skia-review.googlesource.com/103720
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Derek Sollenberger <djsollen@google.com>
2018-02-05 17:24:46 +00:00
Hal Canary
5f73c217cc SkQP: use multiprocessing over threading
BUG=skia:7579
NOTRY=true
Change-Id: I4c7d2331e88171943f724592eb1481a5ea41b0ba
Reviewed-on: https://skia-review.googlesource.com/103700
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-05 17:06:16 +00:00
Stephan Altmueller
dd3eca1fc5 [skqp] More args for skqp related scripts
- allows to choose api versions in run_testlab.go
- adds api_version and work_arounds to generate_gn_args

Bug: skia:
Change-Id: I87516b493f4d8eef3689d1722a5dd864131a44f8
Reviewed-on: https://skia-review.googlesource.com/102900
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Stephan Altmueller <stephana@google.com>
2018-02-02 20:48:59 +00:00
Hal Canary
553ca26136 SkQP: stop using global across jni calls
BIG=skia:7572
Change-Id: I9231e5264305eb871e31ec7b9d23c042135bbefc
Reviewed-on: https://skia-review.googlesource.com/103022
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-02 19:00:19 +00:00
Derek Sollenberger
5d3f7704bf Add gn_to_bp script for SkQP.
This refactors the current gn_to_bp script and moves the common
components into a utils class.  Both gn_to_bp scripts now also
accept an optional param (--gn <gn_path>) to make it easier to run
within the Android tree where we don't have the DEPS synced.

Change-Id: Idc4de7b3993e63e21a5b7137b1873d82a8e1843e
Reviewed-on: https://skia-review.googlesource.com/102184
Commit-Queue: Derek Sollenberger <djsollen@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
2018-02-02 15:12:20 +00:00
Hal Canary
a3ca5f857a SkQP: flatten report images
Change-Id: If590ceb362b4d2e23da72543f40f6255502c63f4
Reviewed-on: https://skia-review.googlesource.com/102486
Reviewed-by: Stephan Altmueller <stephana@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-02 15:10:19 +00:00
Hal Canary
2331c82e0d SkQP: skia_skqp_global_error_tolerance
Change-Id: I991f6a5a6fde3815cd5487227f22014570cfb5d5
Reviewed-on: https://skia-review.googlesource.com/102681
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-01 21:10:49 +00:00
Hal Canary
b296f017a6 SkQP: README: note need java8 not other version
Change-Id: I4743df622bfbcf75d3335226cc5ea4c63de92762
Reviewed-on: https://skia-review.googlesource.com/102323
Reviewed-by: Hal Canary <halcanary@google.com>
2018-02-01 20:12:48 +00:00
Hal Canary
2e34f6b6e4 SkQP: remove model slack, README
Change-Id: Ifd258ccd633fba11a13ac3ffc92fb684789d3e57
Reviewed-on: https://skia-review.googlesource.com/102205
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-01 15:49:43 +00:00
Hal Canary
9c17391cef SkQP: report, use image-rendering:pixelated
Change-Id: If0542634b96dd98e0b86407bbfe0a346a0e42cbd
Reviewed-on: https://skia-review.googlesource.com/102343
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-31 21:13:56 +00:00
Hal Canary
75454f58f5 SkQP: report formatting, junit classname
Change-Id: Ibc5b55c98242c50b62b16134e0a504c04a458f6c
Reviewed-on: https://skia-review.googlesource.com/102320
Reviewed-by: Hal Canary <halcanary@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-31 18:52:45 +00:00
Hal Canary
4689b543a2 SkQP: skia_skqp_enable_driver_correctness_workarounds
Change-Id: If61341423ff03efa48814a393f16eaaee02cecbd
Reviewed-on: https://skia-review.googlesource.com/102183
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-31 17:18:25 +00:00
Hal Canary
ce243ba109 SkQP: fix upload/download scripts
Change-Id: If2b2ec786630e1ecea159e0804547a79478138f5
Reviewed-on: https://skia-review.googlesource.com/101844
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 21:40:40 +00:00
Hal Canary
51494f6615 SkQP: Allow checking against nearby pixel's expectations.
Also more verbose logging.

Change-Id: Ib2d9e3074d155140f14ab8818a332ce926488b79
Reviewed-on: https://skia-review.googlesource.com/101543
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 20:29:59 +00:00
Hal Canary
537d9c0229 SkQP: remove skia_embed_resources option
Motivation: delete unnecessary code.  ResourceFactory.h provides a much more
flexible way of fixing the same problem.

Change-Id: Ib8a3ce25ce98e4f752dc1e7ce88eb9ceb95a4372
Reviewed-on: https://skia-review.googlesource.com/101920
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 19:24:15 +00:00
Hal Canary
ff2742eba8 SkQP: slim deps. Also skia_enable_fontmgr_empty
Change-Id: I24ee2fb3a4a4ca03a60ee2f1a90e022825eee409
Reviewed-on: https://skia-review.googlesource.com/101820
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 18:57:45 +00:00
Hal Canary
b0c427b1f2 SkQP: error image now opaque
Also more explanation in the report.

Change-Id: I1d91efb5b5af064d90da5383e4182c8cef197e0c
Reviewed-on: https://skia-review.googlesource.com/101541
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-29 22:02:49 +00:00
Hal Canary
b4d01a95c9 SkQP: ResourceFactory
Change-Id: Ic57a86ec115e3ace52615c0b8e9a22a915988850
Reviewed-on: https://skia-review.googlesource.com/101320
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-29 21:23:33 +00:00
Hal Canary
323df5720c SkQP: more slack
NOTRY=true
Change-Id: Ibc4efe070cb6e1e69f57f34808b277bf0257ce06
Reviewed-on: https://skia-review.googlesource.com/100421
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Hal Canary <halcanary@google.com>
2018-01-26 22:07:44 +00:00
Hal Canary
71eabfe9e9 SkQP: re-enable workarounds for unit tests. Also, more logging
Change-Id: I74de5477c3cc9a034553363ac525d8f264478153
Reviewed-on: https://skia-review.googlesource.com/99862
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-26 20:44:50 +00:00
Hal Canary
fd1d20b4bb SkQP: run_skqp_exe script
Change-Id: I9e3e4994e15856edd86a1f0f0ab91297c325dcca
Reviewed-on: https://skia-review.googlesource.com/99662
Reviewed-by: Hal Canary <halcanary@google.com>
2018-01-25 16:30:22 +00:00
Hal Canary
7c434fe238 SkQP: Fix errors in README.md
NOTRY=true
Change-Id: I5d9e1834adbff72122c825c63345432a324d5856
Reviewed-on: https://skia-review.googlesource.com/99760
Reviewed-by: Hal Canary <halcanary@google.com>
2018-01-25 15:02:03 +00:00
Hal Canary
181ec2f02f SkQP: Stop requiring users to use posix shell
Also update README.md

Change-Id: I62f5ac38ff4a8c3aa19d895441a76664cb8e8176
Reviewed-on: https://skia-review.googlesource.com/99302
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-24 21:11:19 +00:00
Hal Canary
9ef0989f4c SkQP: add slack to model
Change-Id: Id14e5d4d6c5d517c29cd09b815ede1b52604d96a
Reviewed-on: https://skia-review.googlesource.com/99322
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-24 20:47:59 +00:00
Hal Canary
c341d3829d SkQP: make report.html smaller
Change-Id: Ie43bf83fbd94fdf7edc79cfe403b01474894199d
Reviewed-on: https://skia-review.googlesource.com/99280
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-24 16:46:10 +00:00
Hal Canary
3da15bb1c8 SkQP: fix upload script
Also report transparency

Change-Id: I38c803d1abe1892dc4d7ee6d917f5a39eac6c1c3
Reviewed-on: https://skia-review.googlesource.com/98762
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-23 22:37:30 +00:00
Hal Canary
8d4a9f01f6 SkQP: upload_directory_contents.sh is sharded
NOTRY=true
Change-Id: I1976778cd2c2bdc82f551285b3d36212034f8f7a
Reviewed-on: https://skia-review.googlesource.com/98643
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-23 19:46:39 +00:00
Hal Canary
16b56f5dcf SkQP: set fDisableDriverCorrectnessWorkarounds
Change-Id: I43ea362c5ff726278e2b74bea73cd54a9ef07761
Reviewed-on: https://skia-review.googlesource.com/98843
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
2018-01-23 19:36:20 +00:00
Hal Canary
23fda7ad07 SkQP: Cloud Scripts
- tools/skqp/generate_gn_args.sh - single script to generate
    configuration used to build SkQP.
  - tools/skqp/make_model.sh - script to execute make_gmkb.go and put
    the results in the correct place.
  - tools/skqp/{up,down}load_directory_contents.sh - scripts to move
    large assets into the cloud and retrieve them.
  - tools/skqp/gm_runner.cpp - treat missing text files as empty.
  - tools/skqp/make_apk.sh - make use of generate_gn_args.sh.
  - tools/skqp/make_{model,known_tests}.sh - better error handling,
    and also stop calling `git add`.

Change-Id: Ib1803b4d68e63945f2c47f8eb6cb96375d24e3be
Reviewed-on: https://skia-review.googlesource.com/98842
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-23 19:07:11 +00:00
Hal Canary
4080305f57 SkQP: add some scripts for admin
Change-Id: Idfba4c3e91eb048792ab395d4ab2e4a98ed8888c
Reviewed-on: https://skia-review.googlesource.com/98323
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-22 22:28:06 +00:00
Hal Canary
a9de760a21 SkQP: replace blacklist with: DoNotExecuteInExperimentalMode and NoScoreInCompatibilityTestMode
Also clean up some things, fix docs, whitelist.

Change-Id: I2818d973978ffe1b8ce0cc9c69f8d91ab4a0ef22
Reviewed-on: https://skia-review.googlesource.com/91805
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-22 20:11:57 +00:00
Hal Canary
b69a2f6644 SkQP: use bitmap.pixmap()
Change-Id: I8e372f75a1a0b32db88808b0b4d7314c7fba9cda
Reviewed-on: https://skia-review.googlesource.com/97161
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-19 18:07:43 +00:00
Hal Canary
8bad7e01ba SkQP: better logging
Change-Id: If0b093cfe646f5805cc4d0660c581150fbf992f4
Reviewed-on: https://skia-review.googlesource.com/96702
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-01-19 16:19:23 +00:00
Stephan Altmueller
8ee7fb0083 Fix directory creation in skqp app
Bug: skia:
Change-Id: Id26832ccce7d09d3d3d640bcdd11a424f18943f7
Reviewed-on: https://skia-review.googlesource.com/92682
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-16 21:33:39 +00:00
Stephan Altmueller
c35959f3cb First version of SkQP app to run on Firebase Testlab
Adds activities to the skqp app so it can run as an Android
app (as opposed to just instrumentation tests).
A user can trigger the tests via a button.
Adds the an intent receiver so the tests can be triggered on
Firebase Testlab via the gameloop option.

It adds the run_testlab.go script to run an apk across devices
on Firebase Testlab.

Bug: skia:
Change-Id: I3ff5c37d743fa47913a916a0fa1e7db3c2cc79c7
Reviewed-on: https://skia-review.googlesource.com/89163
Commit-Queue: Stephan Altmueller <stephana@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
2018-01-08 22:42:21 +00:00
Hal Canary
3de24bf20c SkQP: dump csv report
Change-Id: I7485ed3315acdc3dd3b795a42846ed3deedb6b25
Reviewed-on: https://skia-review.googlesource.com/91403
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-07 03:07:46 +00:00
Hal Canary
4e9a3504aa SkQP: more README
Change-Id: Id22a094445ea75a7dc60270a195b9f42bb6849f1
Reviewed-on: https://skia-review.googlesource.com/88680
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2017-12-21 21:11:44 +00:00
Hal Canary
2a7f0aa9eb SkQP: better error report workflow
* Generate report in app after running all tests.
  * Add script to unpack backup.ab file.
  * Remove unused index file.
  * Streamline instructions in README.md.

Change-Id: I44c80b17332eb4496ee31578287b691bd646d71a
Reviewed-on: https://skia-review.googlesource.com/86742
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Hal Canary <halcanary@google.com>
2017-12-20 19:23:46 +00:00
Hal Canary
28f89389f0 SkQP: add junit app
Change-Id: Ic32eaec6cce1509f07e7cf610717d3b12d335c89
Reviewed-on: https://skia-review.googlesource.com/83921
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2017-12-18 18:58:52 +00:00
Hal Canary
d7b3845f3d SkQP: make_gmkb, gm_knowledge (GM Knowledgebase)
Add a real implementation for gm_knowledge.h  This depends on
the presence of files in the form $GMK_DIR/foo/{max,min}.png

The implementation also writes out failures in a report directory.

Add a utility: experimental/make_gmkb which is a stand-alone
go executable that generates the foo/{max,min}.png data.

tools/skqp/README.md has instructions on running SkQP.

Also: add SkFontMgrPriv.h

Change-Id: Ibe1e9a7e7de143d14eee3877f5f2d2d8713f7f49
Reviewed-on: https://skia-review.googlesource.com/65380
Reviewed-by: Yuqian Li <liyuqian@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2017-12-15 17:23:48 +00:00