epoger
7909f47b42
rebaseline_server live queries: allow comparison against summary files within the Skia repo
...
This is a step towards providing SKP comparisons against expectations, not just
other actual results... we still need to allow the user to select
JSONKEY_EXPECTEDRESULTS vs JSONKEY_ACTUALRESULTS within the summary
files found.
BUG=skia:1942
NOTREECHECKS=true
NOTRY=true
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/442203002
2014-08-06 08:37:53 -07:00
epoger
0b7127635d
teach rebaseline_server to generate diffs of rendered SKPs
...
Creates a new live-view.html page, served by the rebaseline_server, that will display diffs between two sets of rendered SKP images.
BUG=skia:1942
NOTRY=true
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/424263005
2014-08-05 10:07:22 -07:00
epoger
66ed8dc4bf
combine base_unittest.py modules from gm and tools
...
general cleanup, which will also help with http://skbug.com/2752 ('split existing "gpu" GM results into "gl" and "gles"')
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/397103003
2014-07-17 12:54:16 -07:00
epoger
03f3db02c1
roll common DEPS, now using recursedeps
...
BUG=skia:2618
R=borenet@google.com , cmp@chromium.org
Author: epoger@google.com
Review URL: https://codereview.chromium.org/392353003
2014-07-16 14:45:48 -07:00
epoger
33ac9506fb
make gm self-tests launch python unittests automatically
...
(not just within the rebaseline_server subdirectory)
Generally useful, but needed asap for my work on http://skbug.com/2752 ('split existing "gpu" GM results into "gl" and "gles"')
BUG=skia:677
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/398043002
2014-07-16 08:28:23 -07:00
epoger
133931f4ab
roll "common" DEPS, and replace tools/pyutils with it
...
BUG=skia:2682
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/385783002
2014-07-11 08:52:35 -07:00
epoger
6132b436d8
rebaseline_server: cache results in long-running ImageDiffDB instance
...
Rather than rebuilding the ImageDiffDB from scratch every time we update expected/actual results, keep the same ImageDiffDB instance around and add image pairs to it.
This makes updating results within a long-running server.py *much* faster, and tests out an idea I'm ruminating in https://goto.google.com/LongRunningImageDiffDBServer : we could run an ImageDiffDB server that developers could connect to from their locally running rebaseline_server instances, for much faster launch times.
BUG=skia:2414
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/379563005
2014-07-09 07:59:06 -07:00
epoger
b4edbffd7c
rebaseline_server: allow JSON to control column filtering
...
Makes the rebaseline_server client more generic, allowing the server to tweak display properties by writing directives into the JSON file.
Adds two new fields to the rebaseline_server JSON file (and thus increments VALUE__HEADER__SCHEMA_VERSION):
1. KEY__ROOT__EXTRACOLUMNORDER: order in which the client should display columns
2. KEY__EXTRACOLUMNHEADERS__USE_FREEFORM_FILTER: whether a column should be filtered using a freeform text field or checkboxes
BUG=skia:2230
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/376623002
2014-07-09 06:19:20 -07:00
epoger
df55f0632a
rebaseline_server: handle column filtering more generically
...
This is a step towards making rebaseline_server more flexible, and useful for displaying output of the cluster telemetry server as well as actuals-on-configA-vs-actuals-on-configB.
BUG=skia:2230
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/369133004
2014-07-07 08:49:14 -07:00
epoger
46e51e10ae
rebaseline_server: if urlSchemaVersion is not specified, assume current
...
Modification to https://codereview.chromium.org/368933002/ ('rebaseline_server: add urlSchemaVersion parameter')
Means we don't need https://codereview.chromium.org/367173002/ ('add urlSchemaVersion to rebaseline_server link on buildbot console page')
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com
TBR=borenet
Author: epoger@google.com
Review URL: https://codereview.chromium.org/364253003
2014-07-03 10:50:30 -07:00
epoger
e5481eca7e
rebaseline_server: add urlSchemaVersion parameter
...
In https://codereview.chromium.org/369473005 ('rebaseline_server: handle category filtering more generically'), I am changing the meaning of some URL parameters. If a user pastes an old URL into an updated rebaseline_server instance, they may get unexpected results. Adding a urlSchemaVersion parameter allows us to automatically convert the URL, or at least warn the user.
BUG=skia:2230
NOTRY=True
R=jcgregorio@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/368933002
2014-07-02 13:51:47 -07:00
epoger
54f1ad8bb5
rebaseline_server: use just skpdiff, not Python Image Library
...
BUG=skia:2414
R=djsollen@google.com , borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/325413003
2014-07-02 07:43:04 -07:00
Mike Klein
a40239ca1e
Drop --clone from commands in skia repo.
...
--clone, and SkPicture::clone(), are no longer.
CQ_EXTRA_TRYBOTS=tryserver.skia:Housekeeper-PerCommit-Trybot
BUG=skia:
R=rmistry@google.com
Review URL: https://codereview.chromium.org/343813006
2014-06-27 17:09:39 -04:00
epoger
984b97c496
rebaseline_server: delete no-longer-used weightedDiffMeasure
...
(replaced by perceptual diff)
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/334533003
2014-06-11 20:35:59 -07:00
epoger
cbbc7bf971
rebaseline_server: change 2 image expectations within unittests
...
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/329933002
2014-06-11 20:02:26 -07:00
epoger
6b28120e4c
rebaseline_server: merge identical results (across multiple builders/configs) into a single row
...
BUG=skia:2534
R=jcgregorio@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/326013002
2014-06-11 14:01:35 -07:00
epoger
b144271179
reland "rebaseline_server: download actual-results.json files from GCS instead of SVN"
...
relands https://codereview.chromium.org/310093003 with modifications.
BUG=skia:2641
R=jcgregorio@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/313343003
2014-06-05 10:30:37 -07:00
epoger
77f68b623e
Revert "rebaseline_server: download actual-results.json files from GCS instead of SVN"
...
This reverts commit f4666045f3
.
BUG=skia:2641
NOTREECHECKS=True
NOTRY=True
R=jcgregorio@google.com
TBR=jcgregorio
Author: epoger@google.com
Review URL: https://codereview.chromium.org/317783004
2014-06-04 15:09:18 -07:00
epoger
f4666045f3
rebaseline_server: download actual-results.json files from GCS instead of SVN
...
BUG=skia:553
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/310093003
2014-06-04 10:02:58 -07:00
epoger
55ada0630e
download_actuals.py: download JSON files from Google Storage instead of skia-autogen
...
uses google-api-python-client instead of gsutil binary to interact with Google Storage
BUG=skia:553
R=rmistry@google.com , bensong@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/309653005
2014-06-03 10:35:28 -07:00
commit-bot@chromium.org
4cef1be821
rebaseline_server: apply ignored-tests.txt within rebaseline_server, not just on the bots
...
(SkipBuildbotRuns)
BUG=skia:2514
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/287623006
git-svn-id: http://skia.googlecode.com/svn/trunk@14740 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-14 18:04:46 +00:00
commit-bot@chromium.org
68a3815401
rename rebaseline_server JSON constants to be more consistent
...
(no behavioral change, no change to JSON file format)
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/287473002
git-svn-id: http://skia.googlecode.com/svn/trunk@14701 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-12 20:40:29 +00:00
commit-bot@chromium.org
97f0b089da
rebaseline_server JSON: pass category values as values, not keys
...
BUG=skia:1758
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/270413002
git-svn-id: http://skia.googlecode.com/svn/trunk@14658 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-08 21:15:20 +00:00
commit-bot@chromium.org
0ecbfffa82
rebaseline_server: add link to other result sets from main UI
...
BUG=skia:2529
NOTREECHECKS=True
NOTRY=True
(SkipBuildbotRuns)
R=caryclark@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/267363010
git-svn-id: http://skia.googlecode.com/svn/trunk@14648 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-08 15:40:06 +00:00
commit-bot@chromium.org
a60d0370b1
make compare_rendered_pictures_test.py run end-to-end test
...
1. generate test SKPs using skpmaker
2. render those SKPs using render_pictures
3. compare results using compare_rendered_pictures.py
BUG=skia:1942,skia:2230
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/274623002
git-svn-id: http://skia.googlecode.com/svn/trunk@14624 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-07 19:29:03 +00:00
commit-bot@chromium.org
7418bd8cad
make compare_rendered_pictures process render_pictures's new JSON output format
...
BUG=skia:1942,skia:2230
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/265793013
git-svn-id: http://skia.googlecode.com/svn/trunk@14591 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-06 15:31:31 +00:00
commit-bot@chromium.org
658e59507d
tweaks to tab CSS for rebaseline server page; moar pretty
...
BUG=skia:
R=epoger@google.com
Author: humper@google.com
Review URL: https://codereview.chromium.org/264853004
git-svn-id: http://skia.googlecode.com/svn/trunk@14512 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-01 19:29:53 +00:00
commit-bot@chromium.org
6323844cb3
rebaseline_server: ignore builders we don't have any expectations for
...
(SkipBuildbotRuns)
BUG=skia:2486
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/269543002
git-svn-id: http://skia.googlecode.com/svn/trunk@14488 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-30 21:32:37 +00:00
commit-bot@chromium.org
9d3fdbeb7b
rebaseline_server unittests: output tkdiff lines when actuals don't match expectations
...
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/252583005
git-svn-id: http://skia.googlecode.com/svn/trunk@14385 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-25 19:59:44 +00:00
commit-bot@chromium.org
c9b511f0e6
Revert of rebaseline_server: multithreaded loading/diffing of images ( https://codereview.chromium.org/235923002/ )
...
Reason for revert:
Caused https://code.google.com/p/skia/issues/detail?id=2423 ('"Deadline exceeded" error when connecting to live rebaseline_server'). Reverting until I can figure it out.
Original issue's description:
> rebaseline_server: multithreaded loading/diffing of images
>
> BUG=skia:2414
> NOTRY=True
>
> Committed: http://code.google.com/p/skia/source/detail?r=14184
R=rmistry@google.com
TBR=rmistry@google.com
NOTREECHECKS=true
NOTRY=true
BUG=skia:2414
Author: epoger@google.com
Review URL: https://codereview.chromium.org/239623002
git-svn-id: http://skia.googlecode.com/svn/trunk@14207 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-15 18:50:12 +00:00
commit-bot@chromium.org
280ea8296f
rebaseline_server: multithreaded loading/diffing of images
...
BUG=skia:2414
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/235923002
git-svn-id: http://skia.googlecode.com/svn/trunk@14184 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-14 18:15:29 +00:00
commit-bot@chromium.org
e93a4c0fea
rebaseline_server: fix sorting by perceptual diff column
...
(SkipBuildbotRuns)
BUG=skia:2410
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/235993002
git-svn-id: http://skia.googlecode.com/svn/trunk@14167 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-11 21:45:41 +00:00
commit-bot@chromium.org
1f75ab9c66
add some .gitignores related to gm and tools
...
NOTREECHECKS=True
NOTRY=True
R=scroggo@google.com
TBR=scroggo
Author: epoger@google.com
Review URL: https://codereview.chromium.org/234603004
git-svn-id: http://skia.googlecode.com/svn/trunk@14150 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-11 14:49:50 +00:00
commit-bot@chromium.org
d0b8904521
rebaseline_server: eliminate meaningless 404 errors while waiting for content to load
...
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=jcgregorio@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/231513004
git-svn-id: http://skia.googlecode.com/svn/trunk@14138 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-10 20:41:13 +00:00
commit-bot@chromium.org
defe6fdbc8
rebaseline_server: allow user to specify which builders to process
...
BUG=skia:1543,skia:1915
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/232103002
git-svn-id: http://skia.googlecode.com/svn/trunk@14131 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-10 15:05:24 +00:00
commit-bot@chromium.org
3eb77e4d5a
teach rebaseline_server how to compare results of multiple render_pictures runs
...
BUG=skia:2230,skia:1942
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/216103004
git-svn-id: http://skia.googlecode.com/svn/trunk@14062 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-04 16:40:25 +00:00
commit-bot@chromium.org
6844958fff
rebaseline_server: display as much info as possible if expected/actual image is missing
...
BUG=skia:2368
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/221423003
git-svn-id: http://skia.googlecode.com/svn/trunk@14019 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-01 22:16:33 +00:00
commit-bot@chromium.org
31d0b3d806
rebaseline_server: add --compare-configs option
...
This allows us to compare GMs between configs across all builders, so we can see the largest deviations between raster and GPU renderings.
BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/215503002
git-svn-id: http://skia.googlecode.com/svn/trunk@13991 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-31 15:17:52 +00:00
commit-bot@chromium.org
c0df2fb5d0
rebaseline_server: add option to read actuals from local disk
...
(SkipBuildbotRuns)
BUG=skia:1908
NOTREECHECKS=True
NOTRY=True
R=halcanary@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/212673013
git-svn-id: http://skia.googlecode.com/svn/trunk@13971 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-28 14:28:04 +00:00
commit-bot@chromium.org
da0ceb2299
rebaseline_server: delete invalid image files from local disk cache
...
BUG=skia:2247
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/211733004
git-svn-id: http://skia.googlecode.com/svn/trunk@13951 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-26 13:38:29 +00:00
commit-bot@chromium.org
41056232e4
fix compare_to_expectations command line launch
...
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry
Author: epoger@google.com
Review URL: https://codereview.chromium.org/210093003
git-svn-id: http://skia.googlecode.com/svn/trunk@13921 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-24 19:51:57 +00:00
commit-bot@chromium.org
b463d5668a
rebaseline_server: rename results.py to compare_to_expectations.py
...
BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/208243003
git-svn-id: http://skia.googlecode.com/svn/trunk@13894 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-21 17:54:14 +00:00
commit-bot@chromium.org
a25c4e4fef
rebaseline_server: fix --editable mode
...
complete revert of https://skia.googlesource.com/skia/+/736be3527214ddbd3dd7e42a89a374c62d98f370
partial revert of https://skia.googlesource.com/skia/+/579942387bed9259b03419c593503022e1399931
and glue made necessary by the above.
(SkipBuildbotRuns)
BUG=skia:2319
NOTREECHECKS=True
NOTRY=True
R=senorblanco@google.com , senorblanco@chromium.org
Author: epoger@google.com
Review URL: https://codereview.chromium.org/208473004
git-svn-id: http://skia.googlecode.com/svn/trunk@13893 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-21 17:30:12 +00:00
commit-bot@chromium.org
736be35272
rebaseline_server: when first started, tell client results are still being generated
...
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=senorblanco@google.com , senorblanco@chromium.org
Author: epoger@google.com
Review URL: https://codereview.chromium.org/206723002
git-svn-id: http://skia.googlecode.com/svn/trunk@13879 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-20 18:31:00 +00:00
commit-bot@chromium.org
579942387b
rebaseline_server: serve JSON from static file, rather than generating it live
...
BUG=skia:1455, skia:2230
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/197213033
git-svn-id: http://skia.googlecode.com/svn/trunk@13876 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-20 17:27:46 +00:00
commit-bot@chromium.org
b42105a4a2
rebaseline_server: image URLs are relative to JSON file's location
...
BUG=skia:1455, skia:2230
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/205273002
git-svn-id: http://skia.googlecode.com/svn/trunk@13874 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-20 15:27:34 +00:00
commit-bot@chromium.org
d1c85d2920
rebaseline_server: make intermediate JSON specify base urls for diff images
...
BUG=skia:1919,skia:2230
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/196533021
git-svn-id: http://skia.googlecode.com/svn/trunk@13830 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-17 14:22:02 +00:00
commit-bot@chromium.org
addcddeda6
make rebaseline_server client check schema version number
...
(SkipBuildbotRuns)
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/195763013
git-svn-id: http://skia.googlecode.com/svn/trunk@13792 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 17:44:01 +00:00
commit-bot@chromium.org
ea770f155e
add schema version to rebaseline_server intermediate JSON
...
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/196343009
git-svn-id: http://skia.googlecode.com/svn/trunk@13790 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 16:33:36 +00:00
commit-bot@chromium.org
7498d95bde
rebaseline_server: generate JSON that can be viewed without a live server
...
BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
Author: epoger@google.com
Review URL: https://codereview.chromium.org/195943004
git-svn-id: http://skia.googlecode.com/svn/trunk@13783 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 14:56:29 +00:00