Commit Graph

140 Commits

Author SHA1 Message Date
Derek Sollenberger
82831b18a6 Add a global error tolerance of 4 to the SkQP Android.bp build file
Change-Id: Iceebd0e153351cad1c2784daf6cabcc8f3bb8ebf
Reviewed-on: https://skia-review.googlesource.com/103720
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Derek Sollenberger <djsollen@google.com>
2018-02-05 17:24:46 +00:00
Hal Canary
5f73c217cc SkQP: use multiprocessing over threading
BUG=skia:7579
NOTRY=true
Change-Id: I4c7d2331e88171943f724592eb1481a5ea41b0ba
Reviewed-on: https://skia-review.googlesource.com/103700
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-05 17:06:16 +00:00
Stephan Altmueller
dd3eca1fc5 [skqp] More args for skqp related scripts
- allows to choose api versions in run_testlab.go
- adds api_version and work_arounds to generate_gn_args

Bug: skia:
Change-Id: I87516b493f4d8eef3689d1722a5dd864131a44f8
Reviewed-on: https://skia-review.googlesource.com/102900
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Stephan Altmueller <stephana@google.com>
2018-02-02 20:48:59 +00:00
Hal Canary
553ca26136 SkQP: stop using global across jni calls
BIG=skia:7572
Change-Id: I9231e5264305eb871e31ec7b9d23c042135bbefc
Reviewed-on: https://skia-review.googlesource.com/103022
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-02 19:00:19 +00:00
Derek Sollenberger
5d3f7704bf Add gn_to_bp script for SkQP.
This refactors the current gn_to_bp script and moves the common
components into a utils class.  Both gn_to_bp scripts now also
accept an optional param (--gn <gn_path>) to make it easier to run
within the Android tree where we don't have the DEPS synced.

Change-Id: Idc4de7b3993e63e21a5b7137b1873d82a8e1843e
Reviewed-on: https://skia-review.googlesource.com/102184
Commit-Queue: Derek Sollenberger <djsollen@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
2018-02-02 15:12:20 +00:00
Hal Canary
a3ca5f857a SkQP: flatten report images
Change-Id: If590ceb362b4d2e23da72543f40f6255502c63f4
Reviewed-on: https://skia-review.googlesource.com/102486
Reviewed-by: Stephan Altmueller <stephana@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-02 15:10:19 +00:00
Hal Canary
2331c82e0d SkQP: skia_skqp_global_error_tolerance
Change-Id: I991f6a5a6fde3815cd5487227f22014570cfb5d5
Reviewed-on: https://skia-review.googlesource.com/102681
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-01 21:10:49 +00:00
Hal Canary
b296f017a6 SkQP: README: note need java8 not other version
Change-Id: I4743df622bfbcf75d3335226cc5ea4c63de92762
Reviewed-on: https://skia-review.googlesource.com/102323
Reviewed-by: Hal Canary <halcanary@google.com>
2018-02-01 20:12:48 +00:00
Hal Canary
2e34f6b6e4 SkQP: remove model slack, README
Change-Id: Ifd258ccd633fba11a13ac3ffc92fb684789d3e57
Reviewed-on: https://skia-review.googlesource.com/102205
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-02-01 15:49:43 +00:00
Hal Canary
9c17391cef SkQP: report, use image-rendering:pixelated
Change-Id: If0542634b96dd98e0b86407bbfe0a346a0e42cbd
Reviewed-on: https://skia-review.googlesource.com/102343
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-31 21:13:56 +00:00
Hal Canary
75454f58f5 SkQP: report formatting, junit classname
Change-Id: Ibc5b55c98242c50b62b16134e0a504c04a458f6c
Reviewed-on: https://skia-review.googlesource.com/102320
Reviewed-by: Hal Canary <halcanary@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-31 18:52:45 +00:00
Hal Canary
4689b543a2 SkQP: skia_skqp_enable_driver_correctness_workarounds
Change-Id: If61341423ff03efa48814a393f16eaaee02cecbd
Reviewed-on: https://skia-review.googlesource.com/102183
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-31 17:18:25 +00:00
Hal Canary
ce243ba109 SkQP: fix upload/download scripts
Change-Id: If2b2ec786630e1ecea159e0804547a79478138f5
Reviewed-on: https://skia-review.googlesource.com/101844
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 21:40:40 +00:00
Hal Canary
51494f6615 SkQP: Allow checking against nearby pixel's expectations.
Also more verbose logging.

Change-Id: Ib2d9e3074d155140f14ab8818a332ce926488b79
Reviewed-on: https://skia-review.googlesource.com/101543
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 20:29:59 +00:00
Hal Canary
537d9c0229 SkQP: remove skia_embed_resources option
Motivation: delete unnecessary code.  ResourceFactory.h provides a much more
flexible way of fixing the same problem.

Change-Id: Ib8a3ce25ce98e4f752dc1e7ce88eb9ceb95a4372
Reviewed-on: https://skia-review.googlesource.com/101920
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 19:24:15 +00:00
Hal Canary
ff2742eba8 SkQP: slim deps. Also skia_enable_fontmgr_empty
Change-Id: I24ee2fb3a4a4ca03a60ee2f1a90e022825eee409
Reviewed-on: https://skia-review.googlesource.com/101820
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-30 18:57:45 +00:00
Hal Canary
b0c427b1f2 SkQP: error image now opaque
Also more explanation in the report.

Change-Id: I1d91efb5b5af064d90da5383e4182c8cef197e0c
Reviewed-on: https://skia-review.googlesource.com/101541
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-29 22:02:49 +00:00
Hal Canary
b4d01a95c9 SkQP: ResourceFactory
Change-Id: Ic57a86ec115e3ace52615c0b8e9a22a915988850
Reviewed-on: https://skia-review.googlesource.com/101320
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-29 21:23:33 +00:00
Hal Canary
323df5720c SkQP: more slack
NOTRY=true
Change-Id: Ibc4efe070cb6e1e69f57f34808b277bf0257ce06
Reviewed-on: https://skia-review.googlesource.com/100421
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Hal Canary <halcanary@google.com>
2018-01-26 22:07:44 +00:00
Hal Canary
71eabfe9e9 SkQP: re-enable workarounds for unit tests. Also, more logging
Change-Id: I74de5477c3cc9a034553363ac525d8f264478153
Reviewed-on: https://skia-review.googlesource.com/99862
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-26 20:44:50 +00:00
Hal Canary
fd1d20b4bb SkQP: run_skqp_exe script
Change-Id: I9e3e4994e15856edd86a1f0f0ab91297c325dcca
Reviewed-on: https://skia-review.googlesource.com/99662
Reviewed-by: Hal Canary <halcanary@google.com>
2018-01-25 16:30:22 +00:00
Hal Canary
7c434fe238 SkQP: Fix errors in README.md
NOTRY=true
Change-Id: I5d9e1834adbff72122c825c63345432a324d5856
Reviewed-on: https://skia-review.googlesource.com/99760
Reviewed-by: Hal Canary <halcanary@google.com>
2018-01-25 15:02:03 +00:00
Hal Canary
181ec2f02f SkQP: Stop requiring users to use posix shell
Also update README.md

Change-Id: I62f5ac38ff4a8c3aa19d895441a76664cb8e8176
Reviewed-on: https://skia-review.googlesource.com/99302
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-24 21:11:19 +00:00
Hal Canary
9ef0989f4c SkQP: add slack to model
Change-Id: Id14e5d4d6c5d517c29cd09b815ede1b52604d96a
Reviewed-on: https://skia-review.googlesource.com/99322
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-24 20:47:59 +00:00
Hal Canary
c341d3829d SkQP: make report.html smaller
Change-Id: Ie43bf83fbd94fdf7edc79cfe403b01474894199d
Reviewed-on: https://skia-review.googlesource.com/99280
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-24 16:46:10 +00:00
Hal Canary
3da15bb1c8 SkQP: fix upload script
Also report transparency

Change-Id: I38c803d1abe1892dc4d7ee6d917f5a39eac6c1c3
Reviewed-on: https://skia-review.googlesource.com/98762
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-23 22:37:30 +00:00
Hal Canary
8d4a9f01f6 SkQP: upload_directory_contents.sh is sharded
NOTRY=true
Change-Id: I1976778cd2c2bdc82f551285b3d36212034f8f7a
Reviewed-on: https://skia-review.googlesource.com/98643
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-23 19:46:39 +00:00
Hal Canary
16b56f5dcf SkQP: set fDisableDriverCorrectnessWorkarounds
Change-Id: I43ea362c5ff726278e2b74bea73cd54a9ef07761
Reviewed-on: https://skia-review.googlesource.com/98843
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
2018-01-23 19:36:20 +00:00
Hal Canary
23fda7ad07 SkQP: Cloud Scripts
- tools/skqp/generate_gn_args.sh - single script to generate
    configuration used to build SkQP.
  - tools/skqp/make_model.sh - script to execute make_gmkb.go and put
    the results in the correct place.
  - tools/skqp/{up,down}load_directory_contents.sh - scripts to move
    large assets into the cloud and retrieve them.
  - tools/skqp/gm_runner.cpp - treat missing text files as empty.
  - tools/skqp/make_apk.sh - make use of generate_gn_args.sh.
  - tools/skqp/make_{model,known_tests}.sh - better error handling,
    and also stop calling `git add`.

Change-Id: Ib1803b4d68e63945f2c47f8eb6cb96375d24e3be
Reviewed-on: https://skia-review.googlesource.com/98842
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-23 19:07:11 +00:00
Hal Canary
4080305f57 SkQP: add some scripts for admin
Change-Id: Idfba4c3e91eb048792ab395d4ab2e4a98ed8888c
Reviewed-on: https://skia-review.googlesource.com/98323
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-22 22:28:06 +00:00
Hal Canary
a9de760a21 SkQP: replace blacklist with: DoNotExecuteInExperimentalMode and NoScoreInCompatibilityTestMode
Also clean up some things, fix docs, whitelist.

Change-Id: I2818d973978ffe1b8ce0cc9c69f8d91ab4a0ef22
Reviewed-on: https://skia-review.googlesource.com/91805
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-22 20:11:57 +00:00
Hal Canary
b69a2f6644 SkQP: use bitmap.pixmap()
Change-Id: I8e372f75a1a0b32db88808b0b4d7314c7fba9cda
Reviewed-on: https://skia-review.googlesource.com/97161
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-19 18:07:43 +00:00
Hal Canary
8bad7e01ba SkQP: better logging
Change-Id: If0b093cfe646f5805cc4d0660c581150fbf992f4
Reviewed-on: https://skia-review.googlesource.com/96702
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Stephan Altmueller <stephana@google.com>
2018-01-19 16:19:23 +00:00
Stephan Altmueller
8ee7fb0083 Fix directory creation in skqp app
Bug: skia:
Change-Id: Id26832ccce7d09d3d3d640bcdd11a424f18943f7
Reviewed-on: https://skia-review.googlesource.com/92682
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-16 21:33:39 +00:00
Stephan Altmueller
c35959f3cb First version of SkQP app to run on Firebase Testlab
Adds activities to the skqp app so it can run as an Android
app (as opposed to just instrumentation tests).
A user can trigger the tests via a button.
Adds the an intent receiver so the tests can be triggered on
Firebase Testlab via the gameloop option.

It adds the run_testlab.go script to run an apk across devices
on Firebase Testlab.

Bug: skia:
Change-Id: I3ff5c37d743fa47913a916a0fa1e7db3c2cc79c7
Reviewed-on: https://skia-review.googlesource.com/89163
Commit-Queue: Stephan Altmueller <stephana@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Derek Sollenberger <djsollen@google.com>
2018-01-08 22:42:21 +00:00
Hal Canary
3de24bf20c SkQP: dump csv report
Change-Id: I7485ed3315acdc3dd3b795a42846ed3deedb6b25
Reviewed-on: https://skia-review.googlesource.com/91403
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2018-01-07 03:07:46 +00:00
Hal Canary
4e9a3504aa SkQP: more README
Change-Id: Id22a094445ea75a7dc60270a195b9f42bb6849f1
Reviewed-on: https://skia-review.googlesource.com/88680
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2017-12-21 21:11:44 +00:00
Hal Canary
2a7f0aa9eb SkQP: better error report workflow
* Generate report in app after running all tests.
  * Add script to unpack backup.ab file.
  * Remove unused index file.
  * Streamline instructions in README.md.

Change-Id: I44c80b17332eb4496ee31578287b691bd646d71a
Reviewed-on: https://skia-review.googlesource.com/86742
Commit-Queue: Hal Canary <halcanary@google.com>
Reviewed-by: Hal Canary <halcanary@google.com>
2017-12-20 19:23:46 +00:00
Hal Canary
28f89389f0 SkQP: add junit app
Change-Id: Ic32eaec6cce1509f07e7cf610717d3b12d335c89
Reviewed-on: https://skia-review.googlesource.com/83921
Reviewed-by: Hal Canary <halcanary@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2017-12-18 18:58:52 +00:00
Hal Canary
d7b3845f3d SkQP: make_gmkb, gm_knowledge (GM Knowledgebase)
Add a real implementation for gm_knowledge.h  This depends on
the presence of files in the form $GMK_DIR/foo/{max,min}.png

The implementation also writes out failures in a report directory.

Add a utility: experimental/make_gmkb which is a stand-alone
go executable that generates the foo/{max,min}.png data.

tools/skqp/README.md has instructions on running SkQP.

Also: add SkFontMgrPriv.h

Change-Id: Ibe1e9a7e7de143d14eee3877f5f2d2d8713f7f49
Reviewed-on: https://skia-review.googlesource.com/65380
Reviewed-by: Yuqian Li <liyuqian@google.com>
Commit-Queue: Hal Canary <halcanary@google.com>
2017-12-15 17:23:48 +00:00