Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
#ifndef DMExpectations_DEFINED
|
|
|
|
#define DMExpectations_DEFINED
|
|
|
|
|
|
|
|
#include "DMTask.h"
|
|
|
|
#include "gm_expectations.h"
|
|
|
|
|
|
|
|
namespace DM {
|
|
|
|
|
|
|
|
struct Expectations {
|
|
|
|
virtual ~Expectations() {}
|
|
|
|
|
|
|
|
// Return true if bitmap is the correct output for task, else false.
|
|
|
|
virtual bool check(const Task& task, SkBitmap bitmap) const = 0;
|
|
|
|
};
|
|
|
|
|
|
|
|
class NoExpectations : public Expectations {
|
|
|
|
public:
|
|
|
|
NoExpectations() {}
|
|
|
|
bool check(const Task&, SkBitmap) const SK_OVERRIDE { return true; }
|
|
|
|
};
|
|
|
|
|
2014-05-16 18:11:51 +00:00
|
|
|
#ifdef SK_BUILD_JSON_WRITER
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
class JsonExpectations : public Expectations {
|
|
|
|
public:
|
|
|
|
explicit JsonExpectations(const char* path) : fGMExpectations(path) {}
|
|
|
|
|
|
|
|
bool check(const Task& task, SkBitmap bitmap) const SK_OVERRIDE {
|
|
|
|
SkString filename = task.name();
|
|
|
|
filename.append(".png");
|
|
|
|
const skiagm::Expectations expectations = fGMExpectations.get(filename.c_str());
|
|
|
|
|
|
|
|
if (expectations.ignoreFailure() || expectations.empty()) {
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Delay this calculation as long as possible. It's expensive.
|
|
|
|
const skiagm::GmResultDigest digest(bitmap);
|
|
|
|
return expectations.match(digest);
|
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
skiagm::JsonExpectationsSource fGMExpectations;
|
|
|
|
};
|
2014-05-16 18:11:51 +00:00
|
|
|
#endif
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
|
|
|
|
} // namespace DM
|
|
|
|
|
|
|
|
#endif // DMExpectations_DEFINED
|