epoger@google.com
3e7399fe28
Fix Housekeeper bot after r9961
...
git-svn-id: http://skia.googlecode.com/svn/trunk@9965 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 17:23:47 +00:00
epoger@google.com
89fa4b9ee6
rebaseline.py: error out early if --expectations-root not pointing at gm-expected dir
...
BUG=https://code.google.com/p/skia/issues/detail?id=1403
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18416005
git-svn-id: http://skia.googlecode.com/svn/trunk@9961 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 16:54:10 +00:00
epoger@google.com
9de25e3fd4
rebaseline.py: use specified configs/tests as a FILTER over actually-run tests
...
Now that we have the actual-results.json file for every bot type, we know
the full set of tests that actual results are available for. If configs and/or
tests are specified, just use those as FILTERS over the actually available
results.
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18324018
git-svn-id: http://skia.googlecode.com/svn/trunk@9956 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 15:27:18 +00:00
epoger@google.com
a783f2bee8
rebaseline.py: if expectations dir contains JSON format results, update those instead of image files
...
Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18348018
git-svn-id: http://skia.googlecode.com/svn/trunk@9910 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-08 17:51:58 +00:00
epoger@google.com
34b73a2da1
rebaseline.py self-tests: prepare to add tests of JSON-format rebaselining
...
For now, rebaseline.py still does not support JSON-format rebaselining, so
the one self-test reports failure. But this change will allow us to test
the JSON-format rebaselining once rebaseline.py supports it.
R=scroggo@google.com
Review URL: https://codereview.chromium.org/18348015
git-svn-id: http://skia.googlecode.com/svn/trunk@9896 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-03 17:43:11 +00:00
epoger@google.com
53953b457b
Create jsondiff tool to capture diffs between two JSON expectations files.
...
This will be used to view diffs before committing rebaselines, or to review
exactly what changed between two baseline revisions in the past.
Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail
R=scroggo@google.com
Review URL: https://codereview.chromium.org/18103005
git-svn-id: http://skia.googlecode.com/svn/trunk@9878 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-02 20:22:27 +00:00
epoger@google.com
8575b4df41
Fix rebaseline.py self-test to account for r9859
...
R=robertphillips@google.com
Review URL: https://codereview.chromium.org/18559002
git-svn-id: http://skia.googlecode.com/svn/trunk@9872 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-02 18:25:06 +00:00
scroggo@google.com
2b9424b342
Treat empty directory properly in skimage.
...
Instead of treating an empty directory as a file that failed to
decode, treat it as an empty directory.
Add self tests to check for this.
Builds upon from https://codereview.chromium.org/16866010 ,
which was reverted.
R=epoger@google.com
Review URL: https://codereview.chromium.org/17101029
git-svn-id: http://skia.googlecode.com/svn/trunk@9730 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-21 19:12:47 +00:00
epoger@google.com
99a8ec9705
rebaseline.py: split image-based rebaselining, which will go away soon, into its own script
...
(part of step 3 in https://goto.google.com/ChecksumTransitionDetail )
also adds new --expectations-root option
R=scroggo@google.com , senorblanco@chromium.org
Review URL: https://codereview.chromium.org/17379004
git-svn-id: http://skia.googlecode.com/svn/trunk@9689 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-19 18:56:59 +00:00
epoger@google.com
c7e1873c85
skimage: revert r9642
...
git-svn-id: http://skia.googlecode.com/svn/trunk@9647 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-18 02:53:16 +00:00
epoger@google.com
e0433f1ad3
skimage: if --readPath points at an empty dir, create an empty expectations file
...
TBR=scroggo
Review URL: https://codereview.chromium.org/17294006
git-svn-id: http://skia.googlecode.com/svn/trunk@9642 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-17 19:34:58 +00:00
epoger@google.com
74feb1593a
skimage self-tests: confirm correctness of bitmap hash values
...
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16866010
git-svn-id: http://skia.googlecode.com/svn/trunk@9591 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-13 19:12:05 +00:00
epoger@google.com
e78d20798d
rebaseline.py: try to download images from Google Storage before skia-autogen
...
This will allow us to complete Step 1 of https://goto.google.com/ChecksumTransitionDetail
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16311011
git-svn-id: http://skia.googlecode.com/svn/trunk@9534 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 17:44:14 +00:00
epoger@google.com
dad5310ab6
rebaseline.py: add --add-new option, and only add new expectations if it is set
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/16363006
git-svn-id: http://skia.googlecode.com/svn/trunk@9527 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 14:25:30 +00:00
epoger@google.com
cc2e1cf13c
rebaseline.py: improve readability of dry-run output
...
Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results')
Review URL: https://codereview.chromium.org/16509006
git-svn-id: http://skia.googlecode.com/svn/trunk@9507 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 16:24:37 +00:00
epoger@google.com
82f317815e
rebaseline.py: in option names, replace underscores with hyphens
...
Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results')
Review URL: https://codereview.chromium.org/15904018
git-svn-id: http://skia.googlecode.com/svn/trunk@9504 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 15:45:46 +00:00
scroggo@google.com
cf5eb6ab25
Preparations for running skimage tool on bots.
...
skimage_main:
Specify the full name of the expectations file to write on the command line,
rather than the folder to write 'results.json'. This is consistent with
writeJsonSummaryPath in GM, as well as the format for reading the file.
Further, it allows more flexibility to name the file.
Do not force opaque when writing bitmaps. skimage does its comparison on
the raw bitmaps. The images are written for visual comparison, and mostly
transparent bitmaps (like resources/plane.png) aren't meaningful when
forced opaque.
Provide an option to to write only mismatches to a directory. Handled
similar to the writePath.
Report a failure if there is no expectations file. This allows writing
to the mismatchPath so we can create an initial baseline.
Pass SkBitmap by const&.
skimage_self_test:
Include the name of the json file in the parameter passed to
createExpectationsPath. The name has been changed to be more specific.
.gitignore:
Ignore the new name of the file created by skimage_self_test.
run.sh:
Run skimage_self_tests.py
BUG=http://code.google.com/p/skia/issues/detail?id=1241
R=epoger@google.com
Review URL: https://codereview.chromium.org/15813016
git-svn-id: http://skia.googlecode.com/svn/trunk@9474 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-07 12:43:15 +00:00
scroggo@google.com
e762dad67c
Run skimage as part of run.sh
...
Test skimage to ensure that it does not break.
Split off from https://codereview.chromium.org/15813016/
BUG=http://code.google.com/p/skia/issues/detail?id=1241
R=epoger@google.com
Review URL: https://codereview.chromium.org/15774017
git-svn-id: http://skia.googlecode.com/svn/trunk@9473 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-07 12:40:35 +00:00
epoger@google.com
19e80c16a5
rebaseline.py self-tests: split test inputs and outputs into completely separate directories
...
Review URL: https://codereview.chromium.org/16557002
git-svn-id: http://skia.googlecode.com/svn/trunk@9459 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-06 15:11:11 +00:00
epoger@google.com
bf65378266
add more rebaseline test resources (no behavioral changes)
...
(SkipBuildbotRuns)
Review URL: https://codereview.chromium.org/15774019
git-svn-id: http://skia.googlecode.com/svn/trunk@9446 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-05 16:26:23 +00:00
epoger@google.com
99ba65a181
rebaseline.py: if --tests is not specified, get test list from actual-results.json
...
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16093025
git-svn-id: http://skia.googlecode.com/svn/trunk@9443 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-05 15:43:37 +00:00
epoger@google.com
db29a310f7
rebaseline.py : add self-tests
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/15660014
git-svn-id: http://skia.googlecode.com/svn/trunk@9426 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-04 14:58:47 +00:00
scroggo@google.com
4fed643cb9
Create a self test for skimage.
...
Runs skimage twice: once to create an expectations file, and a
second time comparing against the file. Uses the files in
resources as test files.
R=epoger@google.com
Review URL: https://codereview.chromium.org/14969007
git-svn-id: http://skia.googlecode.com/svn/trunk@9123 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-14 17:50:02 +00:00
epoger@google.com
bd9669ad27
bench_graph_svg: add r7618 results (NOT per-tile) to self-test source data
...
Review URL: https://codereview.appspot.com/7305098
git-svn-id: http://skia.googlecode.com/svn/trunk@7760 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-17 08:33:27 +00:00
epoger@google.com
aa6470af8e
bench_graph_svg self-test: add non-tile bench data
...
TBR=benchen
Review URL: https://codereview.appspot.com/7317048
git-svn-id: http://skia.googlecode.com/svn/trunk@7750 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-15 16:42:21 +00:00
epoger@google.com
673816c44f
tools/tests/run.sh: download test bench data from Google Storage rather than SVN
...
Review URL: https://codereview.appspot.com/7343045
git-svn-id: http://skia.googlecode.com/svn/trunk@7749 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-15 15:50:23 +00:00
epoger@google.com
2459a394b8
bench_graph_svg: generate output in consistently sorted order, to make diffs more useful
...
TBR=benchen
Review URL: https://codereview.appspot.com/7330044
git-svn-id: http://skia.googlecode.com/svn/trunk@7744 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-14 18:58:05 +00:00
epoger@google.com
9316d7a0cc
disable new bench_graph_svg test in tools/tests/run.sh that fails on housekeeper
...
TBR=bensong
Review URL: https://codereview.appspot.com/7303092
git-svn-id: http://skia.googlecode.com/svn/trunk@7739 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-14 15:35:23 +00:00
epoger@google.com
f4eeeabcdc
tools/tests/run.sh: add self-test for bench_graph_svg.py
...
Review URL: https://codereview.appspot.com/7319044
git-svn-id: http://skia.googlecode.com/svn/trunk@7735 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-02-14 15:10:24 +00:00
borenet@google.com
b00d670114
Enable per-tile logging in bench_pictures
...
- Modify bench_pictures buildstep to translate "key: True" --> "--key" rather
than requiring "key: value" --> "--key value"
- Add whacky TileArgs helper which includes "timeIndividualTiles: True"
Review URL: https://codereview.appspot.com/7092046
git-svn-id: http://skia.googlecode.com/svn/trunk@7141 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-01-11 20:45:24 +00:00
rmistry@google.com
ee5a5eee12
* Extended skdiff to report alpha channel differences.
...
* Created a tools/tests/rebaseline.sh to copy output-actual into output-expected.
* Sample results are available here: http://www.corp.google.com/~rmistry/skia/gm-playback-windows/images-skdiff/
Review URL: https://codereview.appspot.com/7038048
git-svn-id: http://skia.googlecode.com/svn/trunk@7003 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-01-03 19:23:22 +00:00
epoger@google.com
3aa3358c03
Add more warnings about https://code.google.com/p/skia/issues/detail?id=677 ('make tools/tests/run.sh work cross-platform')
...
git-svn-id: http://skia.googlecode.com/svn/trunk@6965 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-01-02 15:53:25 +00:00
borenet@google.com
3b98bfd402
Address comments for r6822: https://codereview.appspot.com/6946052/
...
Review URL: https://codereview.appspot.com/6943059
git-svn-id: http://skia.googlecode.com/svn/trunk@6847 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-12-17 17:21:04 +00:00
borenet@google.com
ddf36e73ac
Fix bench_pictures.cfg and add sanity check
...
Review URL: https://codereview.appspot.com/6946052
git-svn-id: http://skia.googlecode.com/svn/trunk@6822 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-12-14 14:56:37 +00:00
bungeman@google.com
e3c8ddfd03
Update skdiff.
...
https://codereview.appspot.com/6850115/
git-svn-id: http://skia.googlecode.com/svn/trunk@6681 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-12-05 20:13:12 +00:00
epoger@google.com
a413a53cb2
Add a self-test for gm, similar to our self-test for skdiff.
...
Review URL: https://codereview.appspot.com/6827068
git-svn-id: http://skia.googlecode.com/svn/trunk@6380 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-11-12 18:04:51 +00:00
jkleinert@google.com
ed01f12d13
Adding checkboxes to skdiff output so a user can select a subset of images to rebaseline.
...
BUG=745
Review URL: https://codereview.appspot.com/6497054
git-svn-id: http://skia.googlecode.com/svn/trunk@5360 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-08-30 22:05:53 +00:00
epoger@google.com
71329d809a
skdiff: recurse over subdirectories, unless --norecurse option is given
...
in anticipation of https://code.google.com/p/skia/issues/detail?id=743 ('move gm baselines outside of trunk, and modify naming convention')
Review URL: https://codereview.appspot.com/6465053
git-svn-id: http://skia.googlecode.com/svn/trunk@5121 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-08-16 13:42:13 +00:00
epoger@google.com
3af4ff46a7
skdiff output: indicate which result types would cause failure
...
Review URL: https://codereview.appspot.com/6419053
git-svn-id: http://skia.googlecode.com/svn/trunk@4676 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-07-19 17:35:04 +00:00
epoger@google.com
dfbf24e5e7
skdiff: replace --failonmismatches with repeatable --failonresult option
...
Review URL: https://codereview.appspot.com/6392064
git-svn-id: http://skia.googlecode.com/svn/trunk@4611 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-07-13 21:22:02 +00:00
epoger@google.com
46a45963fa
skdiff: add --failonmismatches and --listfilename options, plus cleanup
...
These changes are needed in order to switch the buildbots from using "gm -r" to "skdiff" to compare gm results, and should be generally good for humans too.
Review URL: https://codereview.appspot.com/6392054
git-svn-id: http://skia.googlecode.com/svn/trunk@4579 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-07-12 18:16:02 +00:00
epoger@google.com
28060e7c94
Improve sorting of skdiff output, and make it consistent across platforms
...
BUG=https://code.google.com/p/skia/issues/detail?id=677
Review URL: https://codereview.appspot.com/6351045
git-svn-id: http://skia.googlecode.com/svn/trunk@4388 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-06-28 16:47:34 +00:00
epoger@google.com
be6188d647
skdiff: return nonzero exit value if there are any mismatches
...
This will allow us to replace "gm -r" in our buildbots with "skdiff". This will speed things up, and work around http://code.google.com/p/skia/issues/detail?id=473 ('PDF gradtext gm image results are nondeterministic')
Review URL: https://codereview.appspot.com/6242071
git-svn-id: http://skia.googlecode.com/svn/trunk@4097 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-31 15:13:45 +00:00
epoger@google.com
8e4e51d69b
test-only: add test coverage of skdiff over input with identical bits or pixels
...
Review URL: https://codereview.appspot.com/6260055
git-svn-id: http://skia.googlecode.com/svn/trunk@4084 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-30 19:12:45 +00:00
epoger@google.com
46256ea736
Add to skdiff: test images for bitwise equality before testing for pixel-wise equality
...
- we now check for bitwise equality of files before bothering with pixel
comparisons (this will help with formats for which we don't have good
decoders, like PDF)
- unparsable files are now reported as differing, unless they are bitwise equal
(before, they were always reported as matching!)
- "largest area mismatch" is now based on same-size image pairs only;
previously, if any image pairs had mismatching size, it was 100%
- removed repetitive "image size mismatch, so no diff to display" messages
- changed format of leftmost table cells to be more readable
BUG=http://code.google.com/p/skia/issues/detail?id=473
Review URL: https://codereview.appspot.com/6208089
git-svn-id: http://skia.googlecode.com/svn/trunk@4027 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-22 13:45:35 +00:00
epoger@google.com
90cb9b325c
skdiff test-only change: add test of --nodiffs mode, make tests validate return value
...
Review URL: https://codereview.appspot.com/6206085
git-svn-id: http://skia.googlecode.com/svn/trunk@3999 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-18 18:25:26 +00:00
epoger@google.com
a44435296a
test-only change: update run.sh and rearrange dirs to ease addition of more tests
...
Review URL: https://codereview.appspot.com/6220044
git-svn-id: http://skia.googlecode.com/svn/trunk@3995 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-17 17:12:38 +00:00
epoger@google.com
b2aacb6dc2
test-only changes: add more image files to test more skdiff behavior
...
Tests output order for very vs. slightly different pixels, very vs. slightly different image sizes, etc.
TBR=tomhudson@google.com
Review URL: https://codereview.appspot.com/6213049
git-svn-id: http://skia.googlecode.com/svn/trunk@3977 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-16 18:06:54 +00:00
epoger@google.com
6600852667
skdiff: only write out diff images that are included in index.html
...
Not only are those files unnecessary, in some different-image-size cases, their contents are nondeterministic!
Review URL: https://codereview.appspot.com/6208073
git-svn-id: http://skia.googlecode.com/svn/trunk@3975 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-16 17:40:57 +00:00
epoger@google.com
a2b793c4d1
Add self-test for skdiff, and remove date/time from skdiff output so it passes.
...
BUG=http://code.google.com/p/skia/issues/detail?id=612
Review URL: https://codereview.appspot.com/6198074
git-svn-id: http://skia.googlecode.com/svn/trunk@3938 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-05-15 14:58:53 +00:00