2013-10-18 20:52:44 +00:00
|
|
|
#include "DMWriteTask.h"
|
|
|
|
|
|
|
|
#include "DMUtil.h"
|
2014-01-15 21:28:25 +00:00
|
|
|
#include "SkColorPriv.h"
|
2014-07-22 17:15:34 +00:00
|
|
|
#include "SkCommonFlags.h"
|
2014-09-09 14:36:57 +00:00
|
|
|
#include "SkData.h"
|
2013-10-18 20:52:44 +00:00
|
|
|
#include "SkImageEncoder.h"
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
#include "SkMD5.h"
|
2014-03-05 15:37:11 +00:00
|
|
|
#include "SkMallocPixelRef.h"
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
#include "SkOSFile.h"
|
DM: fix -w/-r problems stemming from PM/UPM conversions.
DM will now read and write a custom image format, which begins first with a
normal PNG, but also includes the bitmap's raw pixels. Most tools (file
browser, Preview, Chrome, Skia image decoder) will read the file as PNG. DM
skips the encoded PNG and reads the raw pixels instead.
This scheme allows perfect pixel comparisons that never go through PM -> UPM or
UPM -> PM conversions, as long as you compare on a machine with the same native
pixel format as the machine which generated the images. Since we've only ever
used this to do same-machine comparisons, that's more than good enough for now.
We could convert as needed to a standardized PM pixel format if we find we want
these files to be portable.
DM -w output now does increase to ~1.3G. If that turns out to be annoyingly
big, I'm sure I can come up with a simple pixelwise RLE instead of writing
uncompressed pixels.
BUG=skia:
R=reed@google.com, bsalomon@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/185343002
git-svn-id: http://skia.googlecode.com/svn/trunk@13638 2bbb7eff-a529-9590-31e7-b0007b416f81
2014-03-03 15:43:56 +00:00
|
|
|
#include "SkStream.h"
|
2014-03-05 15:37:11 +00:00
|
|
|
#include "SkString.h"
|
2013-10-18 20:52:44 +00:00
|
|
|
|
2014-09-08 18:33:48 +00:00
|
|
|
DEFINE_bool(nameByHash, false, "If true, write .../hash.png instead of .../mode/config/name.png");
|
|
|
|
|
2013-10-18 20:52:44 +00:00
|
|
|
namespace DM {
|
|
|
|
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
// Splits off the last N suffixes of name (splitting on _) and appends them to out.
|
|
|
|
// Returns the total number of characters consumed.
|
|
|
|
static int split_suffixes(int N, const char* name, SkTArray<SkString>* out) {
|
2013-12-02 13:50:38 +00:00
|
|
|
SkTArray<SkString> split;
|
|
|
|
SkStrSplit(name, "_", &split);
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
int consumed = 0;
|
|
|
|
for (int i = 0; i < N; i++) {
|
2013-12-02 13:50:38 +00:00
|
|
|
// We're splitting off suffixes from the back to front.
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
out->push_back(split[split.count()-i-1]);
|
|
|
|
consumed += out->back().size() + 1; // Add one for the _.
|
2013-12-02 13:50:38 +00:00
|
|
|
}
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
return consumed;
|
|
|
|
}
|
|
|
|
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
inline static SkString find_base_name(const Task& parent, SkTArray<SkString>* suffixList) {
|
2014-05-14 20:26:00 +00:00
|
|
|
const int suffixes = parent.depth() + 1;
|
|
|
|
const SkString& name = parent.name();
|
2014-06-03 20:57:14 +00:00
|
|
|
const int totalSuffixLength = split_suffixes(suffixes, name.c_str(), suffixList);
|
|
|
|
return SkString(name.c_str(), name.size() - totalSuffixLength);
|
2013-12-02 13:50:38 +00:00
|
|
|
}
|
2013-10-18 20:52:44 +00:00
|
|
|
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
WriteTask::WriteTask(const Task& parent, const char* sourceType, SkBitmap bitmap)
|
2014-06-03 20:57:14 +00:00
|
|
|
: CpuTask(parent)
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
, fBaseName(find_base_name(parent, &fSuffixes))
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
, fSourceType(sourceType)
|
2014-06-03 20:57:14 +00:00
|
|
|
, fBitmap(bitmap)
|
|
|
|
, fData(NULL)
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
, fExtension(".png") {
|
|
|
|
}
|
2014-06-03 20:57:14 +00:00
|
|
|
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
WriteTask::WriteTask(const Task& parent,
|
|
|
|
const char* sourceType,
|
|
|
|
SkStreamAsset *data,
|
|
|
|
const char* ext)
|
2014-06-03 20:57:14 +00:00
|
|
|
: CpuTask(parent)
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
, fBaseName(find_base_name(parent, &fSuffixes))
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
, fSourceType(sourceType)
|
2014-08-26 17:38:07 +00:00
|
|
|
, fData(data)
|
|
|
|
, fExtension(ext) {
|
|
|
|
SkASSERT(fData.get());
|
|
|
|
SkASSERT(fData->unique());
|
|
|
|
}
|
2014-06-03 20:57:14 +00:00
|
|
|
|
2013-12-02 13:50:38 +00:00
|
|
|
void WriteTask::makeDirOrFail(SkString dir) {
|
|
|
|
if (!sk_mkdir(dir.c_str())) {
|
|
|
|
this->fail();
|
|
|
|
}
|
2013-10-18 20:52:44 +00:00
|
|
|
}
|
|
|
|
|
2014-09-09 15:42:04 +00:00
|
|
|
static SkString get_md5(const void* ptr, size_t len) {
|
2014-09-08 19:42:23 +00:00
|
|
|
SkMD5 hasher;
|
2014-09-09 15:42:04 +00:00
|
|
|
hasher.write(ptr, len);
|
2014-09-08 18:33:48 +00:00
|
|
|
SkMD5::Digest digest;
|
2014-09-08 19:42:23 +00:00
|
|
|
hasher.finish(digest);
|
2014-09-08 18:33:48 +00:00
|
|
|
|
2014-09-08 19:42:23 +00:00
|
|
|
SkString md5;
|
2014-09-08 18:33:48 +00:00
|
|
|
for (int i = 0; i < 16; i++) {
|
2014-09-08 19:42:23 +00:00
|
|
|
md5.appendf("%02x", digest.data[i]);
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
}
|
2014-09-08 19:42:23 +00:00
|
|
|
return md5;
|
2014-09-08 18:33:48 +00:00
|
|
|
}
|
|
|
|
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
struct JsonData {
|
|
|
|
SkString name; // E.g. "ninepatch-stretch", "desk-gws_skp"
|
|
|
|
SkString config; // "gpu", "8888"
|
|
|
|
SkString sourceType; // "GM", "SKP"
|
|
|
|
SkString md5; // In ASCII, so 32 bytes long.
|
|
|
|
};
|
|
|
|
SkTArray<JsonData> gJsonData;
|
|
|
|
SK_DECLARE_STATIC_MUTEX(gJsonDataLock);
|
|
|
|
|
2014-09-08 18:33:48 +00:00
|
|
|
void WriteTask::draw() {
|
2014-09-09 15:42:04 +00:00
|
|
|
SkString md5;
|
|
|
|
{
|
|
|
|
SkAutoLockPixels lock(fBitmap);
|
|
|
|
md5 = fData ? get_md5(fData->getMemoryBase(), fData->getLength())
|
|
|
|
: get_md5(fBitmap.getPixels(), fBitmap.getSize());
|
2014-09-08 19:42:23 +00:00
|
|
|
}
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
|
2014-09-09 15:42:04 +00:00
|
|
|
JsonData entry = { fBaseName, fSuffixes[0], fSourceType, md5 };
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
{
|
|
|
|
SkAutoMutexAcquire lock(&gJsonDataLock);
|
|
|
|
gJsonData.push_back(entry);
|
|
|
|
}
|
|
|
|
|
2014-09-08 18:33:48 +00:00
|
|
|
SkString dir(FLAGS_writePath[0]);
|
|
|
|
#if SK_BUILD_FOR_IOS
|
|
|
|
if (dir.equals("@")) {
|
|
|
|
dir.set(FLAGS_resourcePath[0]);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
this->makeDirOrFail(dir);
|
|
|
|
|
|
|
|
SkString path;
|
|
|
|
if (FLAGS_nameByHash) {
|
|
|
|
// Flat directory of hash-named files.
|
2014-09-09 15:42:04 +00:00
|
|
|
path = SkOSPath::Join(dir.c_str(), md5.c_str());
|
2014-09-08 18:33:48 +00:00
|
|
|
path.append(fExtension);
|
|
|
|
// We're content-addressed, so it's possible two threads race to write
|
|
|
|
// this file. We let the first one win. This also means we won't
|
|
|
|
// overwrite identical files from previous runs.
|
|
|
|
if (sk_exists(path.c_str())) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
// Nested by mode, config, etc.
|
|
|
|
for (int i = 0; i < fSuffixes.count(); i++) {
|
|
|
|
dir = SkOSPath::Join(dir.c_str(), fSuffixes[i].c_str());
|
|
|
|
this->makeDirOrFail(dir);
|
|
|
|
}
|
|
|
|
path = SkOSPath::Join(dir.c_str(), fBaseName.c_str());
|
|
|
|
path.append(fExtension);
|
|
|
|
// The path is unique, so two threads can't both write to the same file.
|
|
|
|
// If already present we overwrite here, since the content may have changed.
|
|
|
|
}
|
2014-06-03 20:57:14 +00:00
|
|
|
|
2014-09-08 19:42:23 +00:00
|
|
|
SkFILEWStream file(path.c_str());
|
|
|
|
if (!file.isValid()) {
|
|
|
|
return this->fail("Can't open file.");
|
|
|
|
}
|
|
|
|
|
2014-09-09 15:42:04 +00:00
|
|
|
bool ok = fData ? file.writeStream(fData, fData->getLength())
|
|
|
|
: SkImageEncoder::EncodeStream(&file, fBitmap, SkImageEncoder::kPNG_Type, 100);
|
|
|
|
if (!ok) {
|
2014-09-08 19:42:23 +00:00
|
|
|
return this->fail("Can't write to file.");
|
2013-10-18 20:52:44 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
SkString WriteTask::name() const {
|
2013-12-02 13:50:38 +00:00
|
|
|
SkString name("writing ");
|
|
|
|
for (int i = 0; i < fSuffixes.count(); i++) {
|
|
|
|
name.appendf("%s/", fSuffixes[i].c_str());
|
|
|
|
}
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
name.append(fBaseName.c_str());
|
2013-12-02 13:50:38 +00:00
|
|
|
return name;
|
2013-10-18 20:52:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
bool WriteTask::shouldSkip() const {
|
|
|
|
return FLAGS_writePath.isEmpty();
|
|
|
|
}
|
|
|
|
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
void WriteTask::DumpJson() {
|
|
|
|
if (FLAGS_writePath.isEmpty()) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
Json::Value root;
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
|
|
|
|
for (int i = 1; i < FLAGS_properties.count(); i += 2) {
|
|
|
|
root[FLAGS_properties[i-1]] = FLAGS_properties[i];
|
|
|
|
}
|
|
|
|
for (int i = 1; i < FLAGS_key.count(); i += 2) {
|
|
|
|
root["key"][FLAGS_key[i-1]] = FLAGS_key[i];
|
|
|
|
}
|
|
|
|
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
{
|
|
|
|
SkAutoMutexAcquire lock(&gJsonDataLock);
|
|
|
|
for (int i = 0; i < gJsonData.count(); i++) {
|
Update DM JSON format.
Ex. dm --match patch -w bad --key arch x86 gpu nvidia model z620 --properties git_hash abcd build_number 20 ->
{
"build_number" : "20",
"git_hash" : "abcd",
"key" : {
"arch" : "x86",
"gpu" : "nvidia",
"model" : "z620"
},
"results" : [
{
"key" : {
"config" : "565",
"name" : "ninepatch-stretch"
},
"md5" : "f78cfafcbabaf815f3dfcf61fb59acc7",
"options" : {
"source_type" : "GM"
}
},
{
"key" : {
"config" : "8888",
"name" : "ninepatch-stretch"
},
"md5" : "3e8a42f35a1e76f00caa191e6310d789",
"options" : {
"source_type" : "GM"
}
},
...
This breaks -r, but that's okay. Going to follow up this CL with one that removes that entirely.
BUG=skia:
R=stephana@google.com, jcgregorio@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/551873003
2014-09-09 14:59:46 +00:00
|
|
|
Json::Value result;
|
|
|
|
result["key"]["name"] = gJsonData[i].name.c_str();
|
|
|
|
result["key"]["config"] = gJsonData[i].config.c_str();
|
|
|
|
result["options"]["source_type"] = gJsonData[i].sourceType.c_str();
|
|
|
|
result["md5"] = gJsonData[i].md5.c_str();
|
|
|
|
|
|
|
|
root["results"].append(result);
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
}
|
2014-01-06 20:24:21 +00:00
|
|
|
}
|
|
|
|
|
Start to rework DM JSON handling.
DM's striking off into its own JSON world. This gets strawman implementations
in place for writing and reading a JSON file mapping test name to hashes.
For what it's worth, I basically want to change _all_ these pieces,
- MD5 is slow and we can replace it with something faster,
- JSON schema needs room to grow more data,
- it'd be nice to hash once instead of twice when reading and writing,
- this code wants lots of refactoring,
but this gives us a starting platform to work on these bits at our leisure.
E.x. file for now:
mtklein@mtklein ~/skia (dm)> cat good/dm.json
{
"3x3bitmaprect_565" : "fc70d985fbfbe70e3a3c9dc626d4f5bc",
"3x3bitmaprect_8888" : "df1591dde35907399734ea19feb76663",
"3x3bitmaprect_gpu" : "df1591dde35907399734ea19feb76663",
"aaclip_565" : "1862798689b838a7ab0dc0652b9ace3a",
"aaclip_8888" : "47bb314329f0ce243f1d83fd583decb7",
"aaclip_gpu" : "75f72412d0ef4815770202d297246e7d",
...
BUG=skia:
R=jcgregorio@google.com, stephana@google.com, mtklein@google.com
Author: mtklein@chromium.org
Review URL: https://codereview.chromium.org/546873002
2014-09-08 15:05:18 +00:00
|
|
|
SkString path = SkOSPath::Join(FLAGS_writePath[0], "dm.json");
|
|
|
|
SkFILEWStream stream(path.c_str());
|
|
|
|
stream.writeText(Json::StyledWriter().write(root).c_str());
|
|
|
|
stream.flush();
|
Add support for reading a directory of images with --expectations (-r).
DM writes out its images in a hierarchy that's a little different than GM,
so this can't read GM's output. But it can read its own, written with -w.
Example usage:
$ out/Release/dm -w /tmp/baseline
$ out/Release/dm -r /tmp/baseline -w /tmp/new
(and optionally)
$ mkdir /tmp/diff; out/Release/skdiff /tmp/baseline /tmp/new /tmp/diff
GM's IndividualImageExpectationsSource and Expectations are a little too eager
about decoding and hashing the expected images, so I took the opportunity to
add DM::Expectations that mostly replaces skiagm::ExpectationsSource and
skiagm::Expectations in DM. It mainly exists to move the image decoding and
comparison off the main thread, which would otherwise be a major speed
bottleneck.
I tried to use skiagm code where possible. One notable place where I differed
is in this new feature. When -r is a directory of images, DM does no hashing.
It considerably faster to read the expected file into an SkBitmap and do a
byte-for-byte comparison than to hash the two bitmaps and check those.
The example usage above isn't quite working 100% yet. Expectations on some GMs
fail, even with no binary change. I haven't pinned down whether this is due to
- a bug in DM
- flaky GMs
- unthreadsafe GMs
- flaky image decoding
- unthreadsafe image decoding
- something else
but I intend to. Leon, Derek and I have suspected PNG decoding isn't
threadsafe, but are as yet unable to prove it.
I also seem to be able to cause malloc to fail on my laptop if I run too many
configs at once, though I never seem to be using more than ~1G of RAM. Will
track that down too.
BUG=
R=reed@google.com, bsalomon@google.com
Author: mtklein@google.com
Review URL: https://codereview.chromium.org/108963002
git-svn-id: http://skia.googlecode.com/svn/trunk@12596 2bbb7eff-a529-9590-31e7-b0007b416f81
2013-12-10 14:53:16 +00:00
|
|
|
}
|
|
|
|
|
2013-10-18 20:52:44 +00:00
|
|
|
} // namespace DM
|