Commit Graph

30 Commits

Author SHA1 Message Date
stephana
7260d7292b For certain input skpdiff segfaulted. This prevents the segfault and should fix the failed test in housekeeping.
BUG=skia:2902
R=mtklein@google.com, rmistry@google.com

Author: stephana@google.com

Review URL: https://codereview.chromium.org/546753003
2014-09-05 13:21:42 -07:00
rmistry
2529f2e72c Add ability to output ImageBaseGSUrl to render_picture and use in rebaseline server.
BUG=skia:2230
R=epoger@google.com

Author: rmistry@google.com

Review URL: https://codereview.chromium.org/479613002
2014-08-22 04:46:30 -07:00
epoger
2c4352bb9d rebaseline_server: allow users to generate SKP diff patches on a shared instance
This will allow developers to maintain SKP baselines without ever running their
own rebaseline_server instance!

For now, the developer must manually apply the resulting patchset to his local
Skia checkout to actually modify expectations; in the near future, we hope to
make the UI upload the patchset to Rietveld on the user's behalf.

BUG=skia:1918
NOTRY=true
R=stephana@google.com, rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/487853004
2014-08-20 08:00:28 -07:00
epoger
caa80b9a46 rebaseline_server: constrain setADir/setBDir to consistently refer to single directories, not lists
Before this CL, we supported multiple setADirs/setBDirs for some operations but
not others... now we consistently support a single pair of directories.

NOTRY=true
R=stephana@google.com
TBR=stephana

Author: epoger@google.com

Review URL: https://codereview.chromium.org/480293002
2014-08-18 15:14:46 -07:00
stephana
3b5c86c7a2 This adds renderMode and builder as columns to the front end when looking at SKPs.
BUG=skia:2833
NOTRY=true
R=epoger@google.com

Author: stephana@google.com

Review URL: https://codereview.chromium.org/480153002
2014-08-18 13:37:59 -07:00
stephana
a1aa5c2806 This passes the URL of the diff image to the front-end, thus removing the need to keep frontend code for URL generation in sync with the backend.
BUG=skia:
NOTREECHECKS=true
NOTRY=true
R=epoger@google.com

Author: stephana@google.com

Review URL: https://codereview.chromium.org/471033003
2014-08-15 06:53:23 -07:00
epoger
5c4b137592 rebaseline_server: include setA/setB descriptions in output JSON
This is needed for rebaselining using the shared rebaseline_server instance:
by telling the client which git revision was used to fetch expectations, we
allow the client to pass that back along with new baselines (so the server
knows which revision to patch).

This is also handy in that it allows the server to tell the client more about the
different result sets (which files they were generated from).

BUG=skia:1918
NOTRY=true
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/474813002
2014-08-14 07:12:46 -07:00
epoger
8d66dd1d43 change setA/setB labels if viewing actuals vs expectations
NOTRY=true
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/451053004
2014-08-12 08:29:05 -07:00
epoger
acc1258600 don't list out all sourceSkpFile names in the rebaseline_server UI
(the list is long and crowds out the results themselves; better to use
a text filter to select the sourceSkpFiles of interest)

R=stephana@google.com
TBR=stephana
NOTREECHECKS=true
NOTRY=true

Author: epoger@google.com

Review URL: https://codereview.chromium.org/451853002
2014-08-07 14:55:34 -07:00
epoger
e73cd5ab0f rebaseline_server: allow live queries to examine JSONKEY_EXPECTEDRESULTS or JSONKEY_ACTUALRESULTS
See https://goto.google.com/CorrectnessTestingBigIssues for full context

BUG=skia:1942
NOTRY=true
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/453433004
2014-08-07 12:13:13 -07:00
epoger
7909f47b42 rebaseline_server live queries: allow comparison against summary files within the Skia repo
This is a step towards providing SKP comparisons against expectations, not just
other actual results... we still need to allow the user to select
JSONKEY_EXPECTEDRESULTS vs JSONKEY_ACTUALRESULTS within the summary
files found.

BUG=skia:1942
NOTREECHECKS=true
NOTRY=true
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/442203002
2014-08-06 08:37:53 -07:00
epoger
0b7127635d teach rebaseline_server to generate diffs of rendered SKPs
Creates a new live-view.html page, served by the rebaseline_server, that will display diffs between two sets of rendered SKP images.

BUG=skia:1942
NOTRY=true
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/424263005
2014-08-05 10:07:22 -07:00
epoger
b4edbffd7c rebaseline_server: allow JSON to control column filtering
Makes the rebaseline_server client more generic, allowing the server to tweak display properties by writing directives into the JSON file.

Adds two new fields to the rebaseline_server JSON file (and thus increments VALUE__HEADER__SCHEMA_VERSION):
1. KEY__ROOT__EXTRACOLUMNORDER: order in which the client should display columns
2. KEY__EXTRACOLUMNHEADERS__USE_FREEFORM_FILTER: whether a column should be filtered using a freeform text field or checkboxes

BUG=skia:2230
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/376623002
2014-07-09 06:19:20 -07:00
epoger
54f1ad8bb5 rebaseline_server: use just skpdiff, not Python Image Library
BUG=skia:2414
R=djsollen@google.com, borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/325413003
2014-07-02 07:43:04 -07:00
epoger
984b97c496 rebaseline_server: delete no-longer-used weightedDiffMeasure
(replaced by perceptual diff)

NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/334533003
2014-06-11 20:35:59 -07:00
epoger
cbbc7bf971 rebaseline_server: change 2 image expectations within unittests
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/329933002
2014-06-11 20:02:26 -07:00
commit-bot@chromium.org
4cef1be821 rebaseline_server: apply ignored-tests.txt within rebaseline_server, not just on the bots
(SkipBuildbotRuns)
BUG=skia:2514
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/287623006

git-svn-id: http://skia.googlecode.com/svn/trunk@14740 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-14 18:04:46 +00:00
commit-bot@chromium.org
97f0b089da rebaseline_server JSON: pass category values as values, not keys
BUG=skia:1758
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/270413002

git-svn-id: http://skia.googlecode.com/svn/trunk@14658 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-08 21:15:20 +00:00
commit-bot@chromium.org
a60d0370b1 make compare_rendered_pictures_test.py run end-to-end test
1. generate test SKPs using skpmaker
2. render those SKPs using render_pictures
3. compare results using compare_rendered_pictures.py

BUG=skia:1942,skia:2230
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/274623002

git-svn-id: http://skia.googlecode.com/svn/trunk@14624 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-07 19:29:03 +00:00
commit-bot@chromium.org
7418bd8cad make compare_rendered_pictures process render_pictures's new JSON output format
BUG=skia:1942,skia:2230
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/265793013

git-svn-id: http://skia.googlecode.com/svn/trunk@14591 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-05-06 15:31:31 +00:00
commit-bot@chromium.org
6323844cb3 rebaseline_server: ignore builders we don't have any expectations for
(SkipBuildbotRuns)
BUG=skia:2486
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com
TBR=rmistry

Author: epoger@google.com

Review URL: https://codereview.chromium.org/269543002

git-svn-id: http://skia.googlecode.com/svn/trunk@14488 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-30 21:32:37 +00:00
commit-bot@chromium.org
1f75ab9c66 add some .gitignores related to gm and tools
NOTREECHECKS=True
NOTRY=True
R=scroggo@google.com
TBR=scroggo

Author: epoger@google.com

Review URL: https://codereview.chromium.org/234603004

git-svn-id: http://skia.googlecode.com/svn/trunk@14150 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-11 14:49:50 +00:00
commit-bot@chromium.org
3eb77e4d5a teach rebaseline_server how to compare results of multiple render_pictures runs
BUG=skia:2230,skia:1942
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/216103004

git-svn-id: http://skia.googlecode.com/svn/trunk@14062 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-04 16:40:25 +00:00
commit-bot@chromium.org
6844958fff rebaseline_server: display as much info as possible if expected/actual image is missing
BUG=skia:2368
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/221423003

git-svn-id: http://skia.googlecode.com/svn/trunk@14019 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-04-01 22:16:33 +00:00
commit-bot@chromium.org
31d0b3d806 rebaseline_server: add --compare-configs option
This allows us to compare GMs between configs across all builders, so we can see the largest deviations between raster and GPU renderings.

BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/215503002

git-svn-id: http://skia.googlecode.com/svn/trunk@13991 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-31 15:17:52 +00:00
commit-bot@chromium.org
b463d5668a rebaseline_server: rename results.py to compare_to_expectations.py
BUG=skia:1919
NOTREECHECKS=True
NOTRY=True
R=borenet@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/208243003

git-svn-id: http://skia.googlecode.com/svn/trunk@13894 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-21 17:54:14 +00:00
commit-bot@chromium.org
d1c85d2920 rebaseline_server: make intermediate JSON specify base urls for diff images
BUG=skia:1919,skia:2230
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/196533021

git-svn-id: http://skia.googlecode.com/svn/trunk@13830 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-17 14:22:02 +00:00
commit-bot@chromium.org
ea770f155e add schema version to rebaseline_server intermediate JSON
NOTREECHECKS=True
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/196343009

git-svn-id: http://skia.googlecode.com/svn/trunk@13790 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-13 16:33:36 +00:00
commit-bot@chromium.org
16f418080f rebaseline_server: use new intermediate JSON format as described in https://goto.google.com/ChangingRbsJson
There are still some places in the frontend (HTML+Javascript) code where
we assume we are handling expectations vs actuals, but there are just a few
and we should be able to remove them easily in a coming CL.
At that point, the frontend will work just as well for displaying any set
of image pairs.

BUG=skia:1919
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/178253010

git-svn-id: http://skia.googlecode.com/svn/trunk@13598 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-26 19:05:20 +00:00
commit-bot@chromium.org
f679e2b90d rebaseline_server: extract common unittest code, and rename tests/ -> testdata/
NOTRY=True
R=rmistry@google.com

Author: epoger@google.com

Review URL: https://codereview.chromium.org/170803005

git-svn-id: http://skia.googlecode.com/svn/trunk@13499 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-02-19 15:38:13 +00:00