commit-bot@chromium.org
bf8ba65cca
Add Win8 bots to rebaseline.py, add baselines for Win8
...
NOTRY=True
BUG=
R=epoger@google.com
Author: borenet@google.com
Review URL: https://codereview.chromium.org/117433003
git-svn-id: http://skia.googlecode.com/svn/trunk@12714 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-17 17:29:33 +00:00
scroggo@google.com
5187c4313c
More work to integrate skimage with rebaseline tools.
...
tools/skimage_main.cpp:
Add the ability to write the results to checksum based filenames,
much like GM uses. This will allow using the skpdiff server to
rebaseline images.
Write the keys in the JSON file as <original image>_<pref config>.png,
so it matches gm_json.IMAGE_FILENAME_PATTERN. Also replace '_' with
'-' in the original file name, to avoid confusing the pattern matcher.
The '_' to '-' replacement also happens on the output filename.
Read the keys in a similar manner.
In make_outname, no longer remove a suffix. This fixes a bug where
subset decoding writes multiple subsets to the same file.
tools/rebaseline.py:
Since the filenames written to json files now match
gm_json.IMAGE_FILENAME_PATTERN, enable the option to match based
on configs/tests when rebaselining skimage.
test json files:
Update to match the new format of output.
gm/gm_expectations:
Add a constructor that takes a BitmapAndDigest as input.
tools/tests/skimage_self_test.py:
Test that reading the expectations file just created by skimage with
the same file actually compares to the original file (rather than just
succeeding because expectations were missing).
Change the expectations files to match the new format.
Will require a buildbot change to use the new flag: https://codereview.chromium.org/27389002/
BUG=1466
R=epoger@google.com
Review URL: https://codereview.chromium.org/26297004
git-svn-id: http://skia.googlecode.com/svn/trunk@11902 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-22 00:42:46 +00:00
bsalomon@google.com
46484ec5bf
Add --add-ignored option to rebaseline.py
...
R=senorblanco@chromium.org , epoger@google.com
BUG=skia:1600
Review URL: https://codereview.chromium.org/27517002
git-svn-id: http://skia.googlecode.com/svn/trunk@11809 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-16 15:10:45 +00:00
scroggo@google.com
d23e683c51
Add ability to rebaseline skimage to rebaseline.py
...
Skip pattern matching for skimage, since the skimage results do not
match the pattern.
Automatically fill in expectations_root and actuals_base_url when
--skimage is specified.
Requires a change to move the expectations files to the right place.
Depends on https://codereview.chromium.org/26734006
(SkipBuildBotRuns)
BUG=skia:1466
R=epoger@google.com
Review URL: https://codereview.chromium.org/26666004
git-svn-id: http://skia.googlecode.com/svn/trunk@11717 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-10 21:09:24 +00:00
bungeman@google.com
575e82ad5d
Document how to get a specific revision when rebaselining.
...
This is helpful when using the trybots.
R=epoger@google.com
Review URL: https://codereview.chromium.org/25653008
git-svn-id: http://skia.googlecode.com/svn/trunk@11643 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-10-07 20:32:18 +00:00
senorblanco@chromium.org
bfc0d6bd5f
Add an option to rebaseline.py to mark tests as ignore-failure.
...
R=epoger@google.com , epoger
Review URL: https://codereview.chromium.org/24195018
git-svn-id: http://skia.googlecode.com/svn/trunk@11358 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-18 19:14:43 +00:00
epoger@google.com
6d28e8d6ad
rebaseline.py: make builder names copy-pastable from --help output
...
BUG=skia:1653
(SkipBuildbotRuns)
R=bsalomon@google.com
Review URL: https://codereview.chromium.org/23514068
git-svn-id: http://skia.googlecode.com/svn/trunk@11347 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-18 14:58:47 +00:00
borenet@google.com
b6e915da2e
Add --from-trybot flag in rebaseline.py to enable rebaselining from trybots
...
(SkipBuildbotRuns)
R=epoger@google.com
Review URL: https://codereview.chromium.org/24153003
git-svn-id: http://skia.googlecode.com/svn/trunk@11294 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-16 18:13:13 +00:00
epoger@google.com
80f9cf18cc
rebaseline.py: add bug-filing URL to error message
...
R=bsalomon@google.com
Review URL: https://codereview.chromium.org/23968002
git-svn-id: http://skia.googlecode.com/svn/trunk@11133 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-06 17:26:09 +00:00
epoger@google.com
7f5f6e6e30
Revert r11102 (mistakenly committed)
...
git-svn-id: http://skia.googlecode.com/svn/trunk@11103 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-05 17:19:37 +00:00
epoger@google.com
e5ee52e1f0
whitespace-only fix to manually-modified GM expectations
...
git-svn-id: http://skia.googlecode.com/svn/trunk@11102 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-05 17:18:08 +00:00
epoger@google.com
93acfd2b49
rebaseline.py: fix --add-new when there are no expectations at all
...
BUG=skia:1582
R=rmistry@google.com
Review URL: https://codereview.chromium.org/23899003
git-svn-id: http://skia.googlecode.com/svn/trunk@11063 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-03 18:27:19 +00:00
epoger@google.com
06e626d69e
rebaseline.py: add --bugs and --unreviewed flags
...
BUG=skia:1569
R=borenet@google.com
Review URL: https://codereview.chromium.org/23478011
git-svn-id: http://skia.googlecode.com/svn/trunk@11058 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-09-03 17:32:15 +00:00
epoger@google.com
44f165f6ce
temporarily hard-code builder list in rebaseline.py, for quick startup
...
BUG=skia:1544
R=borenet@google.com
Review URL: https://codereview.chromium.org/22866023
git-svn-id: http://skia.googlecode.com/svn/trunk@10908 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-24 20:45:31 +00:00
epoger@google.com
c192aa4937
Update references to buildbot globals deprecated by r10856
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/23011035
git-svn-id: http://skia.googlecode.com/svn/trunk@10860 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-21 17:35:59 +00:00
epoger@google.com
fd040116c5
Remove base-* directories from gm expected/actual paths; just use platform names
...
BUG=skia:1512
The buildbot master continues to set gm_image_subdir, but the slaves no longer
use that component when building GM paths (they just use builder_name instead).
But gm_image_subdir is still used for SKP storage paths under
gs://chromium-skia-gm/playback/gm-actual , for now...
Immediately before committing this CL, I will copy the actual-results.json files
from their old locations to their new locations (within the skia-autogen repo)
so that both old and new versions of rebaseline.py will be able to retrieve
actual results, like so:
svn cp base-android-galaxy-nexus/Test-Android-GalaxyNexus-SGX540-Arm7-Debug/base-android-galaxy-nexus Test-Android-GalaxyNexus-SGX540-Arm7-Debug
svn cp base-android-galaxy-nexus/Test-Android-GalaxyNexus-SGX540-Arm7-Release/base-android-galaxy-nexus Test-Android-GalaxyNexus-SGX540-Arm7-Release
svn cp base-android-nexus-10/Test-Android-Nexus10-MaliT604-Arm7-Debug/base-android-nexus-10 Test-Android-Nexus10-MaliT604-Arm7-Debug
...
svn cp base-shuttle-win7-intel-float/Test-Win7-ShuttleA-HD2000-x86-Debug/base-shuttle-win7-intel-float Test-Win7-ShuttleA-HD2000-x86-Debug
svn cp base-shuttle-win7-intel-float/Test-Win7-ShuttleA-HD2000-x86-Release/base-shuttle-win7-intel-float Test-Win7-ShuttleA-HD2000-x86-Release
Once we see this CL land successfully, I will delete the actual-results.json
files from their OLD location within skia-autogen, like so:
svn rm base-android-galaxy-nexus
svn rm base-android-nexus-10
...
svn rm base-shuttle-win7-intel-directwrite
svn rm base-shuttle-win7-intel-float
R=borenet@google.com , rmistry@google.com
Review URL: https://codereview.chromium.org/23120002
git-svn-id: http://skia.googlecode.com/svn/trunk@10821 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-20 16:21:55 +00:00
epoger@google.com
2a192a8577
Delete image-based rebaselining tool; we have switched to checksums
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/21901004
git-svn-id: http://skia.googlecode.com/svn/trunk@10524 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-08-02 20:54:46 +00:00
epoger@google.com
c60e745356
rebaseline.py: add ability to write new expectations to a different file
...
(instead of overwriting the original expectations file)
R=zachr@google.com
Review URL: https://codereview.chromium.org/19805007
git-svn-id: http://skia.googlecode.com/svn/trunk@10333 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-24 19:36:51 +00:00
epoger@google.com
27e1c0070d
rebaseline.py: set text mimetype for all .json files, so text diffs work
...
BUG=https://code.google.com/p/skia/issues/detail?id=1442
R=fmalita@google.com
Review URL: https://codereview.chromium.org/20017009
git-svn-id: http://skia.googlecode.com/svn/trunk@10311 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-24 15:38:39 +00:00
epoger@google.com
e94a7d2936
Move gm-expected to trunk/expectations/gm , and modify scripts that use it.
...
Left dummy content in the old gm-expected dir to keep Update step happy,
to avoid the breakages caused by r10281.
R=borenet@google.com
Review URL: https://codereview.chromium.org/19668009
git-svn-id: http://skia.googlecode.com/svn/trunk@10288 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-23 19:37:03 +00:00
epoger@google.com
7699eeb847
Revert r10281 -- it broke the Update step on Test builders
...
git-svn-id: http://skia.googlecode.com/svn/trunk@10285 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-23 18:32:55 +00:00
epoger@google.com
0ab790bef3
Move gm-expected to trunk/expectations/gm , and modify scripts that use it.
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/19697017
git-svn-id: http://skia.googlecode.com/svn/trunk@10281 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-23 17:35:43 +00:00
epoger@google.com
b248dd5294
rebaseline.py: re-record already succeeding test results, just in case
...
TBR=borenet
Review URL: https://codereview.chromium.org/19459002
git-svn-id: http://skia.googlecode.com/svn/trunk@10121 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-17 00:09:10 +00:00
epoger@google.com
4b38301a57
rebaseline.py: fix --add-new for JSON files
...
TBR=borenet
Review URL: https://codereview.chromium.org/19415002
git-svn-id: http://skia.googlecode.com/svn/trunk@10117 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-16 21:10:54 +00:00
epoger@google.com
61822a25d3
svndiff.py: add ability to compare before-and-after JSON files, not just raw images
...
This should complete step 3 of https://goto.google.com/ChecksumTransitionDetail !
R=borenet@google.com
Review URL: https://codereview.chromium.org/19112002
git-svn-id: http://skia.googlecode.com/svn/trunk@10113 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-16 18:56:32 +00:00
epoger@google.com
ffcbdbfe6a
rebaseline.py: add --keep-going-on-failure option, off by default
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/18092004
git-svn-id: http://skia.googlecode.com/svn/trunk@10109 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-16 17:35:39 +00:00
epoger@google.com
66ba9f978f
rebaseline.py: skip any platforms we don't have actual results for, with warning message
...
R=borenet@google.com
Review URL: https://codereview.chromium.org/19052007
git-svn-id: http://skia.googlecode.com/svn/trunk@10022 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-11 19:20:30 +00:00
epoger@google.com
3e7399fe28
Fix Housekeeper bot after r9961
...
git-svn-id: http://skia.googlecode.com/svn/trunk@9965 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 17:23:47 +00:00
epoger@google.com
89fa4b9ee6
rebaseline.py: error out early if --expectations-root not pointing at gm-expected dir
...
BUG=https://code.google.com/p/skia/issues/detail?id=1403
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18416005
git-svn-id: http://skia.googlecode.com/svn/trunk@9961 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 16:54:10 +00:00
epoger@google.com
9de25e3fd4
rebaseline.py: use specified configs/tests as a FILTER over actually-run tests
...
Now that we have the actual-results.json file for every bot type, we know
the full set of tests that actual results are available for. If configs and/or
tests are specified, just use those as FILTERS over the actually available
results.
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18324018
git-svn-id: http://skia.googlecode.com/svn/trunk@9956 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-10 15:27:18 +00:00
epoger@google.com
a783f2bee8
rebaseline.py: if expectations dir contains JSON format results, update those instead of image files
...
Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/18348018
git-svn-id: http://skia.googlecode.com/svn/trunk@9910 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-08 17:51:58 +00:00
robertphillips@google.com
63e9627fca
Add N4 to rebaseline.py
...
https://codereview.chromium.org/18023014/
(SkipBuildbotRuns)
git-svn-id: http://skia.googlecode.com/svn/trunk@9859 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-07-02 12:54:37 +00:00
epoger@google.com
99a8ec9705
rebaseline.py: split image-based rebaselining, which will go away soon, into its own script
...
(part of step 3 in https://goto.google.com/ChecksumTransitionDetail )
also adds new --expectations-root option
R=scroggo@google.com , senorblanco@chromium.org
Review URL: https://codereview.chromium.org/17379004
git-svn-id: http://skia.googlecode.com/svn/trunk@9689 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-19 18:56:59 +00:00
epoger@google.com
bd4af3a9a6
rebaseline.py: keep going if we encounter an "INVALID" entry in the JSON file
...
Review URL: https://codereview.chromium.org/16826003
git-svn-id: http://skia.googlecode.com/svn/trunk@9543 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 19:07:00 +00:00
epoger@google.com
e78d20798d
rebaseline.py: try to download images from Google Storage before skia-autogen
...
This will allow us to complete Step 1 of https://goto.google.com/ChecksumTransitionDetail
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16311011
git-svn-id: http://skia.googlecode.com/svn/trunk@9534 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 17:44:14 +00:00
epoger@google.com
dad5310ab6
rebaseline.py: add --add-new option, and only add new expectations if it is set
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/16363006
git-svn-id: http://skia.googlecode.com/svn/trunk@9527 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-12 14:25:30 +00:00
epoger@google.com
cc2e1cf13c
rebaseline.py: improve readability of dry-run output
...
Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results')
Review URL: https://codereview.chromium.org/16509006
git-svn-id: http://skia.googlecode.com/svn/trunk@9507 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 16:24:37 +00:00
epoger@google.com
bda4e91596
rebaseline.py: if rebaselining all platforms (not listing them explicitly), allow missing platforms
...
TBR=any-reviewer
Review URL: https://codereview.chromium.org/16782003
git-svn-id: http://skia.googlecode.com/svn/trunk@9506 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 16:16:02 +00:00
epoger@google.com
82f317815e
rebaseline.py: in option names, replace underscores with hyphens
...
Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results')
Review URL: https://codereview.chromium.org/15904018
git-svn-id: http://skia.googlecode.com/svn/trunk@9504 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-11 15:45:46 +00:00
epoger@google.com
99ba65a181
rebaseline.py: if --tests is not specified, get test list from actual-results.json
...
R=scroggo@google.com
Review URL: https://codereview.chromium.org/16093025
git-svn-id: http://skia.googlecode.com/svn/trunk@9443 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-05 15:43:37 +00:00
epoger@google.com
db29a310f7
rebaseline.py : add self-tests
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/15660014
git-svn-id: http://skia.googlecode.com/svn/trunk@9426 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-06-04 14:58:47 +00:00
epoger@google.com
9166bf57d8
rebaseline.py: use argparse command-line flags for more flexibility
...
--tests is the only mandatory argument.
If you used to run this:
rebaseline.py aaclip bigmatrix
Run this instead:
rebaseline.py --tests aaclip bigmatrix
That's the only change you NEED to make.
And then, if you WANT to specify --configs, --subdirs, etc. you CAN.
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/15675010
git-svn-id: http://skia.googlecode.com/svn/trunk@9348 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-30 15:46:19 +00:00
epoger@google.com
ec3397bb6c
Refactor rebaseline.py into functions ; no behavioral changes.
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/16160008
git-svn-id: http://skia.googlecode.com/svn/trunk@9318 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-29 17:09:43 +00:00
bsalomon@google.com
7d01ec478f
Make rebaseline.py pull msaa images
...
R=robertphillips@google.com
Review URL: https://codereview.chromium.org/15299005
git-svn-id: http://skia.googlecode.com/svn/trunk@9179 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-17 14:10:59 +00:00
bsalomon@google.com
3c58723202
Update rebaseline.py for N10.
...
R=jvanverth@google.com
Review URL: https://codereview.chromium.org/14581014
git-svn-id: http://skia.googlecode.com/svn/trunk@9126 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-14 18:20:00 +00:00
bsalomon@google.com
cb3c8d7e6a
Make rebaseline.py pull GN Debug images.
...
R=senorblanco@chromium.org
Review URL: https://codereview.chromium.org/14743004
git-svn-id: http://skia.googlecode.com/svn/trunk@8999 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-05-03 20:24:54 +00:00
borenet@google.com
e6598a04a7
Fix builder name references
...
... as a result of https://codereview.chromium.org/14517004/
(SkipBuildbotRuns)
R=bensong@google.com , rmistry@google.com , senorblanco@chromium.org
Review URL: https://codereview.chromium.org/14544003
git-svn-id: http://skia.googlecode.com/svn/trunk@8921 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-04-30 12:02:32 +00:00
senorblanco@chromium.org
0b6dc190bb
Rebaseline script: check for .git and .svn in the current directory as well. Add mesa to list of configs to update.
...
R=bsalomon@google.com
Review URL: https://codereview.appspot.com/6908050
git-svn-id: http://skia.googlecode.com/svn/trunk@6707 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-12-07 17:19:07 +00:00
bungeman@google.com
df6fe603a5
No longer generating 4444, remove from rebaseline tool.
...
https://codereview.appspot.com/6886048/
git-svn-id: http://skia.googlecode.com/svn/trunk@6692 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-12-06 19:16:36 +00:00
senorblanco@chromium.org
123a0b5fbf
rebaseline.py: Allow multiple testnames on the command line.
...
Review URL: https://codereview.appspot.com/6856118
git-svn-id: http://skia.googlecode.com/svn/trunk@6623 2bbb7eff-a529-9590-31e7-b0007b416f81
2012-11-29 21:50:34 +00:00