Commit Graph

100 Commits

Author SHA1 Message Date
epoger
984b97c496 rebaseline_server: delete no-longer-used weightedDiffMeasure
(replaced by perceptual diff)

NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/334533003
2014-06-11 20:35:59 -07:00
epoger
cbbc7bf971 rebaseline_server: change 2 image expectations within unittests
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/329933002
2014-06-11 20:02:26 -07:00
epoger
6b28120e4c rebaseline_server: merge identical results (across multiple builders/configs) into a single row
BUG=skia:2534
R=jcgregorio@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/326013002
2014-06-11 14:01:35 -07:00
epoger
b144271179 reland "rebaseline_server: download actual-results.json files from GCS instead of SVN"
relands https://codereview.chromium.org/310093003 with modifications.

BUG=skia:2641
R=jcgregorio@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/313343003
2014-06-05 10:30:37 -07:00
epoger
77f68b623e Revert "rebaseline_server: download actual-results.json files from GCS instead of SVN"
This reverts commit f4666045f3.

BUG=skia:2641
NOTREECHECKS=True
NOTRY=True
R=jcgregorio@google.com
TBR=jcgregorio

Author: epoger@google.com

Review URL: https://codereview.chromium.org/317783004
2014-06-04 15:09:18 -07:00
epoger
f4666045f3 rebaseline_server: download actual-results.json files from GCS instead of SVN
BUG=skia:553
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/310093003
2014-06-04 10:02:58 -07:00
epoger
55ada0630e download_actuals.py: download JSON files from Google Storage instead of skia-autogen
uses google-api-python-client instead of gsutil binary to interact with Google Storage

BUG=skia:553
R=rmistry@google.com, bensong@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/309653005
2014-06-03 10:35:28 -07:00
commit-bot@chromium.org
4cef1be821 rebaseline_server: apply ignored-tests.txt within rebaseline_server, not just on the bots
(SkipBuildbotRuns)
BUG=skia:2514
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/287623006

git-svn-id: http://skia.googlecode.com/svn/trunk@14740 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-14 18:04:46 +00:00
commit-bot@chromium.org
68a3815401 rename rebaseline_server JSON constants to be more consistent
(no behavioral change, no change to JSON file format)

(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/287473002

git-svn-id: http://skia.googlecode.com/svn/trunk@14701 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-12 20:40:29 +00:00
commit-bot@chromium.org
97f0b089da rebaseline_server JSON: pass category values as values, not keys
BUG=skia:1758
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/270413002

git-svn-id: http://skia.googlecode.com/svn/trunk@14658 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-08 21:15:20 +00:00
commit-bot@chromium.org
0ecbfffa82 rebaseline_server: add link to other result sets from main UI
BUG=skia:2529
NOTREECHECKS=True
NOTRY=True
(SkipBuildbotRuns)

R=caryclark@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/267363010

git-svn-id: http://skia.googlecode.com/svn/trunk@14648 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-08 15:40:06 +00:00
commit-bot@chromium.org
a60d0370b1 make compare_rendered_pictures_test.py run end-to-end test
1. generate test SKPs using skpmaker
2. render those SKPs using render_pictures
3. compare results using compare_rendered_pictures.py

BUG=skia:1942,skia:2230
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/274623002

git-svn-id: http://skia.googlecode.com/svn/trunk@14624 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-07 19:29:03 +00:00
commit-bot@chromium.org
7418bd8cad make compare_rendered_pictures process render_pictures's new JSON output format
BUG=skia:1942,skia:2230
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/265793013

git-svn-id: http://skia.googlecode.com/svn/trunk@14591 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-06 15:31:31 +00:00
commit-bot@chromium.org
658e59507d tweaks to tab CSS for rebaseline server page; moar pretty
BUG=skia:
R=epoger@google.com

Author: humper@google.com

Review URL: https://codereview.chromium.org/264853004

git-svn-id: http://skia.googlecode.com/svn/trunk@14512 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-01 19:29:53 +00:00
commit-bot@chromium.org
6323844cb3 rebaseline_server: ignore builders we don't have any expectations for
(SkipBuildbotRuns)
BUG=skia:2486
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/269543002

git-svn-id: http://skia.googlecode.com/svn/trunk@14488 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-30 21:32:37 +00:00
commit-bot@chromium.org
9d3fdbeb7b rebaseline_server unittests: output tkdiff lines when actuals don't match expectations
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/252583005

git-svn-id: http://skia.googlecode.com/svn/trunk@14385 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-25 19:59:44 +00:00
commit-bot@chromium.org
c9b511f0e6 Revert of rebaseline_server: multithreaded loading/diffing of images (https://codereview.chromium.org/235923002/)
Reason for revert:
Caused https://code.google.com/p/skia/issues/detail?id=2423 ('"Deadline exceeded" error when connecting to live rebaseline_server').  Reverting until I can figure it out.

Original issue's description:
> rebaseline_server: multithreaded loading/diffing of images
>
> BUG=skia:2414
> NOTRY=True
>
> Committed: http://code.google.com/p/skia/source/detail?r=14184

R=rmistry@google.com
TBR=rmistry@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:2414

Author: epoger@google.com

Review URL: https://codereview.chromium.org/239623002

git-svn-id: http://skia.googlecode.com/svn/trunk@14207 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-15 18:50:12 +00:00
commit-bot@chromium.org
280ea8296f rebaseline_server: multithreaded loading/diffing of images
BUG=skia:2414
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/235923002

git-svn-id: http://skia.googlecode.com/svn/trunk@14184 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-14 18:15:29 +00:00
commit-bot@chromium.org
e93a4c0fea rebaseline_server: fix sorting by perceptual diff column
(SkipBuildbotRuns)
BUG=skia:2410
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/235993002

git-svn-id: http://skia.googlecode.com/svn/trunk@14167 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-11 21:45:41 +00:00
commit-bot@chromium.org
1f75ab9c66 add some .gitignores related to gm and tools
NOTREECHECKS=True
NOTRY=True
R=scroggo@google.com
TBR=scroggo

Author: epoger@google.com

Review URL: https://codereview.chromium.org/234603004

git-svn-id: http://skia.googlecode.com/svn/trunk@14150 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-11 14:49:50 +00:00
commit-bot@chromium.org
d0b8904521 rebaseline_server: eliminate meaningless 404 errors while waiting for content to load
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=jcgregorio@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/231513004

git-svn-id: http://skia.googlecode.com/svn/trunk@14138 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-10 20:41:13 +00:00
commit-bot@chromium.org
defe6fdbc8 rebaseline_server: allow user to specify which builders to process
BUG=skia:1543,skia:1915
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/232103002

git-svn-id: http://skia.googlecode.com/svn/trunk@14131 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-10 15:05:24 +00:00
commit-bot@chromium.org
3eb77e4d5a teach rebaseline_server how to compare results of multiple render_pictures runs
BUG=skia:2230,skia:1942
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/216103004

git-svn-id: http://skia.googlecode.com/svn/trunk@14062 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-04 16:40:25 +00:00
commit-bot@chromium.org
6844958fff rebaseline_server: display as much info as possible if expected/actual image is missing
BUG=skia:2368
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/221423003

git-svn-id: http://skia.googlecode.com/svn/trunk@14019 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-01 22:16:33 +00:00
commit-bot@chromium.org
31d0b3d806 rebaseline_server: add --compare-configs option
This allows us to compare GMs between configs across all builders, so we can see the largest deviations between raster and GPU renderings.

BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/215503002

git-svn-id: http://skia.googlecode.com/svn/trunk@13991 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-31 15:17:52 +00:00
commit-bot@chromium.org
c0df2fb5d0 rebaseline_server: add option to read actuals from local disk
(SkipBuildbotRuns)
BUG=skia:1908
NOTREECHECKS=True
NOTRY=True
R=halcanary@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/212673013

git-svn-id: http://skia.googlecode.com/svn/trunk@13971 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-28 14:28:04 +00:00
commit-bot@chromium.org
da0ceb2299 rebaseline_server: delete invalid image files from local disk cache
BUG=skia:2247
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/211733004

git-svn-id: http://skia.googlecode.com/svn/trunk@13951 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-26 13:38:29 +00:00
commit-bot@chromium.org
41056232e4 fix compare_to_expectations command line launch
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/210093003

git-svn-id: http://skia.googlecode.com/svn/trunk@13921 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-24 19:51:57 +00:00
commit-bot@chromium.org
b463d5668a rebaseline_server: rename results.py to compare_to_expectations.py
BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/208243003

git-svn-id: http://skia.googlecode.com/svn/trunk@13894 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-21 17:54:14 +00:00
commit-bot@chromium.org
a25c4e4fef rebaseline_server: fix --editable mode
complete revert of https://skia.googlesource.com/skia/+/736be3527214ddbd3dd7e42a89a374c62d98f370

partial revert of https://skia.googlesource.com/skia/+/579942387bed9259b03419c593503022e1399931

and glue made necessary by the above.

(SkipBuildbotRuns)
BUG=skia:2319
NOTREECHECKS=True
NOTRY=True
R=senorblanco@google.com, senorblanco@chromium.org

Author: epoger@google.com

Review URL: https://codereview.chromium.org/208473004

git-svn-id: http://skia.googlecode.com/svn/trunk@13893 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-21 17:30:12 +00:00
commit-bot@chromium.org
736be35272 rebaseline_server: when first started, tell client results are still being generated
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=senorblanco@google.com, senorblanco@chromium.org

Author: epoger@google.com

Review URL: https://codereview.chromium.org/206723002

git-svn-id: http://skia.googlecode.com/svn/trunk@13879 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-20 18:31:00 +00:00
commit-bot@chromium.org
579942387b rebaseline_server: serve JSON from static file, rather than generating it live
BUG=skia:1455, skia:2230
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/197213033

git-svn-id: http://skia.googlecode.com/svn/trunk@13876 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-20 17:27:46 +00:00
commit-bot@chromium.org
b42105a4a2 rebaseline_server: image URLs are relative to JSON file's location
BUG=skia:1455, skia:2230
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/205273002

git-svn-id: http://skia.googlecode.com/svn/trunk@13874 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-20 15:27:34 +00:00
commit-bot@chromium.org
d1c85d2920 rebaseline_server: make intermediate JSON specify base urls for diff images
BUG=skia:1919,skia:2230
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/196533021

git-svn-id: http://skia.googlecode.com/svn/trunk@13830 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-17 14:22:02 +00:00
commit-bot@chromium.org
addcddeda6 make rebaseline_server client check schema version number
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/195763013

git-svn-id: http://skia.googlecode.com/svn/trunk@13792 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 17:44:01 +00:00
commit-bot@chromium.org
ea770f155e add schema version to rebaseline_server intermediate JSON
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/196343009

git-svn-id: http://skia.googlecode.com/svn/trunk@13790 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 16:33:36 +00:00
commit-bot@chromium.org
7498d95bde rebaseline_server: generate JSON that can be viewed without a live server
BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/195943004

git-svn-id: http://skia.googlecode.com/svn/trunk@13783 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 14:56:29 +00:00
commit-bot@chromium.org
6e3ee3339c rebaseline_server: disable image magnifier, for huge UI speedup
BUG=skia:2248
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/193073003

git-svn-id: http://skia.googlecode.com/svn/trunk@13727 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-10 19:12:53 +00:00
commit-bot@chromium.org
5865ec5f3a rebaseline_server: add actual_repo and actual_revision command line options
So, if you want to see divergences from expectations as of https://code.google.com/p/skia-autogen/source/detail?r=29365 , specify --actuals-revision 29365

This is also handy for testing rebaseline_server performance improvements against a static set of results.

BUG=skia:2248
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com, bsalomon@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/193023002

git-svn-id: http://skia.googlecode.com/svn/trunk@13726 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-10 18:09:25 +00:00
commit-bot@chromium.org
ed19107c1b rebaseline_server: fix reload delay when there is client/server clock skew
Currently, if there is >10 second clock skew between client and server, the
client will attempt to reload constantly while the server is still loading
results.

NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/192833003

git-svn-id: http://skia.googlecode.com/svn/trunk@13725 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-10 18:05:15 +00:00
commit-bot@chromium.org
fadc749bd0 rebaseline_server: display how long it took to render the list
BUG=skia:2248
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/189833008

git-svn-id: http://skia.googlecode.com/svn/trunk@13720 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-10 15:36:06 +00:00
commit-bot@chromium.org
61f1606236 rebaseline_server: add "show thumbnails" option, and skip display of nonexistent images
BUG=skia:2248,skia:2231
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/183893030

git-svn-id: http://skia.googlecode.com/svn/trunk@13676 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-05 19:46:17 +00:00
commit-bot@chromium.org
8cc39a6567 rebaseline_server: better reporting for bad URL fetch
context: https://codereview.chromium.org/183763025/ ('vm_run_skia_try/write_json_summary: run skpdiff within imagediffdb, not on its own')

NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/183833019

git-svn-id: http://skia.googlecode.com/svn/trunk@13654 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-04 16:46:22 +00:00
commit-bot@chromium.org
3b98759529 rebaseline_server: keep going on exception in ImagePair.__init__()
BUG=skia:2247
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com
TBR=borenet

Author: epoger@google.com

Review URL: https://codereview.chromium.org/184103022

git-svn-id: http://skia.googlecode.com/svn/trunk@13650 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-04 00:58:21 +00:00
commit-bot@chromium.org
8dab8ecc93 rebaseline_server: fix "Results current as of <timestamp>" banner
(SkipBuildbotRuns)
BUG=skia:2232
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/183523002

git-svn-id: http://skia.googlecode.com/svn/trunk@13612 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-27 18:13:07 +00:00
commit-bot@chromium.org
16f418080f rebaseline_server: use new intermediate JSON format as described in https://goto.google.com/ChangingRbsJson
There are still some places in the frontend (HTML+Javascript) code where
we assume we are handling expectations vs actuals, but there are just a few
and we should be able to remove them easily in a coming CL.
At that point, the frontend will work just as well for displaying any set
of image pairs.

BUG=skia:1919
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/178253010

git-svn-id: http://skia.googlecode.com/svn/trunk@13598 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-26 19:05:20 +00:00
commit-bot@chromium.org
f679e2b90d rebaseline_server: extract common unittest code, and rename tests/ -> testdata/
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/170803005

git-svn-id: http://skia.googlecode.com/svn/trunk@13499 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-19 15:38:13 +00:00
commit-bot@chromium.org
4d0f008895 rebaseline_server: fail fast if skpdiff binary is not available
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/169943002

git-svn-id: http://skia.googlecode.com/svn/trunk@13484 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-18 14:38:22 +00:00
commit-bot@chromium.org
536b15ffb0 rebaseline_server: create ImagePairSet-- holds a number of ImagePairs to examine
See https://goto.google.com/ChangingRbsJson and bug 1919 for additional context

BUG=skia:1919
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/139343018

git-svn-id: http://skia.googlecode.com/svn/trunk@13434 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-13 17:17:05 +00:00
commit-bot@chromium.org
44546f85d9 Add the perceptual difference metric to the rebaseline server
* Finds the location of the skpdiff binary.
* Calculates the perceptual difference percentage using the skpdiff binary (treats -1 as 0% perceptual similarity).
* Replaces the weightedDiffMeasure in view.html with perceptualDifferent.

BUG=skia:2019
NOTRY=true

Committed: http://code.google.com/p/skia/source/detail?r=13398

R=epoger@google.com

Author: rmistry@google.com

Review URL: https://codereview.chromium.org/147453003

git-svn-id: http://skia.googlecode.com/svn/trunk@13410 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-11 18:21:26 +00:00