19b91531e9
Add an interface to decode frames beyond the first in SkCodec, and add an implementation for SkGifCodec. Add getFrameData to SkCodec. This method reads ahead in the stream to return a vector containing meta data about each frame in the image. This is not required in order to decode frames beyond the first, but it allows a client to learn extra information: - how long the frame should be displayed - whether a frame should be blended with a prior frame, allowing the client to provide the prior frame to speed up decoding Add a new fields to SkCodec::Options: - fFrameIndex - fHasPriorFrame The API is designed so that SkCodec never caches frames. If a client wants a frame beyond the first, they specify the frame in Options.fFrameIndex. If the client does not have the frame's required frame (the frame that this frame must be blended on top of) cached, they pass false for Options.fHasPriorFrame. Unless the frame is independent, the codec will then recursively decode all frames necessary to decode fFrameIndex. If the client has the required frame cached, they can put it in the dst they pass to the codec, and the codec will only draw fFrameIndex onto it. Replace SkGifCodec's scanline decoding support with progressive decoding, and update the tests accordingly. Implement new APIs in SkGifCodec. Instead of using gif_lib, use GIFImageReader, imported from Chromium (along with its copyright headers) with the following changes: - SkGifCodec is now the client - Replace blink types - Combine GIFColorMap::buildTable and ::getTable into a method that creates and returns an SkColorTable - Input comes from an SkStream, instead of a SegmentReader. Add SkStreamBuffer, which buffers the (potentially partial) stream in order to decode progressively. (FIXME: This requires copying data that previously was read directly from the SegmentReader. Does this hurt performance? If so, can we fix it?) - Remove UMA code - Instead of reporting screen width and height to the client, allow the client to query for it - Fail earlier if the first frame AND screen have size of zero - Compute required previous frame when adding a new one - Move GIFParseQuery from GIFImageDecoder to GIFImageReader - Allow parsing up to a specific frame (to skip parsing the rest of the stream if a client only wants the first frame) - Compute whether the first frame has alpha and supports index 8, to create the SkImageInfo. This happens before reporting that the size has been decoded. Add GIFImageDecoder::haveDecodedRow to SkGifCodec, imported from Chromium (along with its copyright header), with the following changes: - Add support for sampling - Use the swizzler - Keep track of the rows decoded - Do *not* keep track of whether we've seen alpha Remove SkCodec::kOutOfOrder_SkScanlineOrder, which was only used by GIF scanline decoding. Call onRewind even if there is no stream (SkGifCodec needs to clear its decoded state so it will decode from the beginning). Add a method to SkSwizzler to access the offset into the dst, taking subsetting into account. Add a GM that animates a GIF. Add tests for the new APIs. *** Behavior changes: * Previously, we reported that an image with a subset frame and no transparent index was opaque and used the background index (if present) to fill the background. This is necessary in order to support index 8, but it does not match viewers/browsers I have seen. Examples: - Chromium and Gimp render the background transparent - Firefox, Safari, Linux Image Viewer, Safari Preview clip to the frame (for a single frame image) This CL matches Chromium's behavior and renders the background transparent. This allows us to have consistent behavior across products and simplifies the code (relative to what we would have to do to continue the old behavior on Android). It also means that we will no longer support index 8 for some GIFs. * Stop checking for GIFSTAMP - all GIFs should be either 89a or 87a. This matches Chromium. I suspect that bugs would have been reported if valid GIFs started with "GIFVER" instead of "GIF89a" or "GIF87a" (but did not decode in Chromium). *** Future work not included in this CL: * Move some checks out of haveDecodedRow, since they are the same for the entire frame e.g. - intersecting the frameRect with the full image size - whether there is a color table * Change when we write transparent pixels - In some cases, Chromium deemed this unnecessary, but I suspect it is slower than the fallback case. There will continue to be cases where we should *not* write them, but for e.g. the first pass where we have already cleared to transparent (which we may also be able to skip) writing the transparent pixels will not make anything incorrect. * Report color type and alpha type per frame - Depending on alpha values, disposal methods, frame rects, etc, subsequent frames may have different properties than the first. * Skip copies of the encoded data - We copy the encoded data in case the stream is one that cannot be rewound, so we can parse and then decode (possibly not immediately). For some input streams, this is unnecessary. - I was concerned this cause a performance regression, but on average the new code is faster than the old for the images I tested [1]. - It may cause a performance regression for Chromium, though, where we can always move back in the stream, so this should be addressed. Design doc: https://docs.google.com/a/google.com/document/d/12Qhf9T92MWfdWujQwCIjhCO3sw6pTJB5pJBwDM1T7Kc/ [1] https://docs.google.com/a/google.com/spreadsheets/d/19V-t9BfbFw5eiwBTKA1qOBkZbchjlTC5EIz6HFy-6RI/ GOLD_TRYBOT_URL= https://gold.skia.org/search2?unt=true&query=source_type%3Dgm&master=false&issue=2045293002 Review-Url: https://codereview.chromium.org/2045293002
323 lines
10 KiB
C++
323 lines
10 KiB
C++
/*
|
|
* Copyright 2016 Google Inc.
|
|
*
|
|
* Use of this source code is governed by a BSD-style license that can be
|
|
* found in the LICENSE file.
|
|
*/
|
|
|
|
#include "SkBitmap.h"
|
|
#include "SkCodec.h"
|
|
#include "SkData.h"
|
|
#include "SkImageInfo.h"
|
|
#include "SkRWBuffer.h"
|
|
#include "SkString.h"
|
|
|
|
#include "Resources.h"
|
|
#include "Test.h"
|
|
|
|
static sk_sp<SkData> make_from_resource(const char* name) {
|
|
SkString fullPath = GetResourcePath(name);
|
|
return SkData::MakeFromFileName(fullPath.c_str());
|
|
}
|
|
|
|
static SkImageInfo standardize_info(SkCodec* codec) {
|
|
SkImageInfo defaultInfo = codec->getInfo();
|
|
// Note: This drops the SkColorSpace, allowing the equality check between two
|
|
// different codecs created from the same file to have the same SkImageInfo.
|
|
return SkImageInfo::MakeN32Premul(defaultInfo.width(), defaultInfo.height());
|
|
}
|
|
|
|
static bool create_truth(sk_sp<SkData> data, SkBitmap* dst) {
|
|
SkAutoTDelete<SkCodec> codec(SkCodec::NewFromData(std::move(data)));
|
|
if (!codec) {
|
|
return false;
|
|
}
|
|
|
|
const SkImageInfo info = standardize_info(codec);
|
|
dst->allocPixels(info);
|
|
return SkCodec::kSuccess == codec->getPixels(info, dst->getPixels(), dst->rowBytes());
|
|
}
|
|
|
|
static void compare_bitmaps(skiatest::Reporter* r, const SkBitmap& bm1, const SkBitmap& bm2) {
|
|
const SkImageInfo& info = bm1.info();
|
|
if (info != bm2.info()) {
|
|
ERRORF(r, "Bitmaps have different image infos!");
|
|
return;
|
|
}
|
|
const size_t rowBytes = info.minRowBytes();
|
|
for (int i = 0; i < info.height(); i++) {
|
|
REPORTER_ASSERT(r, !memcmp(bm1.getAddr(0, 0), bm2.getAddr(0, 0), rowBytes));
|
|
}
|
|
}
|
|
|
|
/*
|
|
* Represents a stream without all of its data.
|
|
*/
|
|
class HaltingStream : public SkStream {
|
|
public:
|
|
HaltingStream(sk_sp<SkData> data, size_t initialLimit)
|
|
: fTotalSize(data->size())
|
|
, fLimit(initialLimit)
|
|
, fStream(std::move(data))
|
|
{}
|
|
|
|
void addNewData(size_t extra) {
|
|
fLimit = SkTMin(fTotalSize, fLimit + extra);
|
|
}
|
|
|
|
size_t read(void* buffer, size_t size) override {
|
|
if (fStream.getPosition() + size > fLimit) {
|
|
size = fLimit - fStream.getPosition();
|
|
}
|
|
|
|
return fStream.read(buffer, size);
|
|
}
|
|
|
|
bool isAtEnd() const override {
|
|
return fStream.isAtEnd();
|
|
}
|
|
|
|
bool hasPosition() const override { return true; }
|
|
size_t getPosition() const override { return fStream.getPosition(); }
|
|
bool rewind() override { return fStream.rewind(); }
|
|
bool move(long offset) override { return fStream.move(offset); }
|
|
|
|
bool isAllDataReceived() const { return fLimit == fTotalSize; }
|
|
|
|
private:
|
|
const size_t fTotalSize;
|
|
size_t fLimit;
|
|
SkMemoryStream fStream;
|
|
};
|
|
|
|
static void test_partial(skiatest::Reporter* r, const char* name) {
|
|
sk_sp<SkData> file = make_from_resource(name);
|
|
if (!file) {
|
|
SkDebugf("missing resource %s\n", name);
|
|
return;
|
|
}
|
|
|
|
SkBitmap truth;
|
|
if (!create_truth(file, &truth)) {
|
|
ERRORF(r, "Failed to decode %s\n", name);
|
|
return;
|
|
}
|
|
|
|
// Now decode part of the file
|
|
HaltingStream* stream = new HaltingStream(file, file->size() / 2);
|
|
|
|
// Note that we cheat and hold on to a pointer to stream, though it is owned by
|
|
// partialCodec.
|
|
SkAutoTDelete<SkCodec> partialCodec(SkCodec::NewFromStream(stream));
|
|
if (!partialCodec) {
|
|
// Technically, this could be a small file where half the file is not
|
|
// enough.
|
|
ERRORF(r, "Failed to create codec for %s", name);
|
|
return;
|
|
}
|
|
|
|
const SkImageInfo info = standardize_info(partialCodec);
|
|
SkASSERT(info == truth.info());
|
|
SkBitmap incremental;
|
|
incremental.allocPixels(info);
|
|
|
|
while (true) {
|
|
const SkCodec::Result startResult = partialCodec->startIncrementalDecode(info,
|
|
incremental.getPixels(), incremental.rowBytes());
|
|
if (startResult == SkCodec::kSuccess) {
|
|
break;
|
|
}
|
|
|
|
if (stream->isAllDataReceived()) {
|
|
ERRORF(r, "Failed to start incremental decode\n");
|
|
return;
|
|
}
|
|
|
|
// Append some data. The size is arbitrary, but deliberately different from
|
|
// the buffer size used by SkPngCodec.
|
|
stream->addNewData(1000);
|
|
}
|
|
|
|
while (true) {
|
|
const SkCodec::Result result = partialCodec->incrementalDecode();
|
|
|
|
if (result == SkCodec::kSuccess) {
|
|
break;
|
|
}
|
|
|
|
REPORTER_ASSERT(r, result == SkCodec::kIncompleteInput);
|
|
|
|
if (stream->isAllDataReceived()) {
|
|
ERRORF(r, "Failed to completely decode %s", name);
|
|
return;
|
|
}
|
|
|
|
// Append some data. The size is arbitrary, but deliberately different from
|
|
// the buffer size used by SkPngCodec.
|
|
stream->addNewData(1000);
|
|
}
|
|
|
|
// compare to original
|
|
compare_bitmaps(r, truth, incremental);
|
|
}
|
|
|
|
DEF_TEST(Codec_partial, r) {
|
|
test_partial(r, "plane.png");
|
|
test_partial(r, "plane_interlaced.png");
|
|
test_partial(r, "yellow_rose.png");
|
|
test_partial(r, "index8.png");
|
|
test_partial(r, "color_wheel.png");
|
|
test_partial(r, "mandrill_256.png");
|
|
test_partial(r, "mandrill_32.png");
|
|
test_partial(r, "arrow.png");
|
|
test_partial(r, "randPixels.png");
|
|
test_partial(r, "baby_tux.png");
|
|
|
|
test_partial(r, "box.gif");
|
|
test_partial(r, "randPixels.gif");
|
|
test_partial(r, "color_wheel.gif");
|
|
}
|
|
|
|
DEF_TEST(Codec_partialAnim, r) {
|
|
auto path = "test640x479.gif";
|
|
sk_sp<SkData> file = make_from_resource(path);
|
|
if (!file) {
|
|
return;
|
|
}
|
|
|
|
// This stream will be owned by fullCodec, but we hang on to the pointer
|
|
// to determine frame offsets.
|
|
SkStream* stream = new SkMemoryStream(file);
|
|
std::unique_ptr<SkCodec> fullCodec(SkCodec::NewFromStream(stream));
|
|
const auto info = standardize_info(fullCodec.get());
|
|
|
|
// frameByteCounts stores the number of bytes to decode a particular frame.
|
|
// - [0] is the number of bytes for the header
|
|
// - frames[i] requires frameByteCounts[i+1] bytes to decode
|
|
std::vector<size_t> frameByteCounts;
|
|
std::vector<SkBitmap> frames;
|
|
size_t lastOffset = 0;
|
|
for (size_t i = 0; true; i++) {
|
|
frameByteCounts.push_back(stream->getPosition() - lastOffset);
|
|
lastOffset = stream->getPosition();
|
|
|
|
SkBitmap frame;
|
|
frame.allocPixels(info);
|
|
|
|
SkCodec::Options opts;
|
|
opts.fFrameIndex = i;
|
|
const SkCodec::Result result = fullCodec->getPixels(info, frame.getPixels(),
|
|
frame.rowBytes(), &opts, nullptr, nullptr);
|
|
|
|
if (result == SkCodec::kIncompleteInput) {
|
|
frameByteCounts.push_back(stream->getPosition() - lastOffset);
|
|
|
|
// We need to distinguish between a partial frame and no more frames.
|
|
// getFrameInfo lets us do this, since it tells the number of frames
|
|
// not considering whether they are complete.
|
|
// FIXME: Should we use a different Result?
|
|
if (fullCodec->getFrameInfo().size() > i) {
|
|
// This is a partial frame.
|
|
frames.push_back(frame);
|
|
}
|
|
break;
|
|
}
|
|
|
|
if (result != SkCodec::kSuccess) {
|
|
ERRORF(r, "Failed to decode frame %i from %s", i, path);
|
|
return;
|
|
}
|
|
|
|
frames.push_back(frame);
|
|
}
|
|
|
|
// Now decode frames partially, then completely, and compare to the original.
|
|
HaltingStream* haltingStream = new HaltingStream(file, frameByteCounts[0]);
|
|
std::unique_ptr<SkCodec> partialCodec(SkCodec::NewFromStream(haltingStream));
|
|
if (!partialCodec) {
|
|
ERRORF(r, "Failed to create a partial codec from %s with %i bytes out of %i",
|
|
path, frameByteCounts[0], file->size());
|
|
return;
|
|
}
|
|
|
|
SkASSERT(frameByteCounts.size() > frames.size());
|
|
for (size_t i = 0; i < frames.size(); i++) {
|
|
const size_t fullFrameBytes = frameByteCounts[i + 1];
|
|
const size_t firstHalf = fullFrameBytes / 2;
|
|
const size_t secondHalf = fullFrameBytes - firstHalf;
|
|
|
|
haltingStream->addNewData(firstHalf);
|
|
|
|
SkBitmap frame;
|
|
frame.allocPixels(info);
|
|
|
|
SkCodec::Options opts;
|
|
opts.fFrameIndex = i;
|
|
SkCodec::Result result = partialCodec->startIncrementalDecode(info,
|
|
frame.getPixels(), frame.rowBytes(), &opts);
|
|
if (result != SkCodec::kSuccess) {
|
|
ERRORF(r, "Failed to start incremental decode for %s on frame %i",
|
|
path, i);
|
|
return;
|
|
}
|
|
|
|
result = partialCodec->incrementalDecode();
|
|
REPORTER_ASSERT(r, SkCodec::kIncompleteInput == result);
|
|
|
|
haltingStream->addNewData(secondHalf);
|
|
result = partialCodec->incrementalDecode();
|
|
REPORTER_ASSERT(r, SkCodec::kSuccess == result);
|
|
|
|
// allocPixels locked the pixels for frame, but frames[i] was copied
|
|
// from another bitmap, and did not retain the locked status.
|
|
SkAutoLockPixels alp(frames[i]);
|
|
compare_bitmaps(r, frames[i], frame);
|
|
}
|
|
}
|
|
|
|
// Test that calling getPixels when an incremental decode has been
|
|
// started (but not finished) makes the next call to incrementalDecode
|
|
// require a call to startIncrementalDecode.
|
|
static void test_interleaved(skiatest::Reporter* r, const char* name) {
|
|
sk_sp<SkData> file = make_from_resource(name);
|
|
if (!file) {
|
|
return;
|
|
}
|
|
const size_t halfSize = file->size() / 2;
|
|
SkAutoTDelete<SkCodec> partialCodec(SkCodec::NewFromStream(
|
|
new HaltingStream(std::move(file), halfSize)));
|
|
if (!partialCodec) {
|
|
ERRORF(r, "Failed to create codec for %s", name);
|
|
return;
|
|
}
|
|
|
|
const SkImageInfo info = standardize_info(partialCodec);
|
|
SkBitmap incremental;
|
|
incremental.allocPixels(info);
|
|
|
|
const SkCodec::Result startResult = partialCodec->startIncrementalDecode(info,
|
|
incremental.getPixels(), incremental.rowBytes());
|
|
if (startResult != SkCodec::kSuccess) {
|
|
ERRORF(r, "Failed to start incremental decode\n");
|
|
return;
|
|
}
|
|
|
|
SkCodec::Result result = partialCodec->incrementalDecode();
|
|
REPORTER_ASSERT(r, result == SkCodec::kIncompleteInput);
|
|
|
|
SkBitmap full;
|
|
full.allocPixels(info);
|
|
result = partialCodec->getPixels(info, full.getPixels(), full.rowBytes());
|
|
REPORTER_ASSERT(r, result == SkCodec::kIncompleteInput);
|
|
|
|
// Now incremental decode will fail
|
|
result = partialCodec->incrementalDecode();
|
|
REPORTER_ASSERT(r, result == SkCodec::kInvalidParameters);
|
|
}
|
|
|
|
DEF_TEST(Codec_rewind, r) {
|
|
test_interleaved(r, "plane.png");
|
|
test_interleaved(r, "plane_interlaced.png");
|
|
test_interleaved(r, "box.gif");
|
|
}
|