v8/test/unittests/api/deserialize-unittest.cc

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

242 lines
8.0 KiB
C++
Raw Normal View History

[api] Add API for off-thread code cache deserialization To consume a code cache off-thread 1. The embedder creates a CachedData object wrapping the data blob. 2. The embedder calls ScriptCompiler::StartConsumingCodeCache with the CachedData, and receives a ScriptCompiler::CodeCacheConsumeTask which takes ownership of the CachedData. 3. The embedder calls ScriptCompiler::CodeCacheConsumeTask::Run on a different thread. 4. Once this completes, the embedded passes the completed task as an optional argument into Source constructor, and calls Compile as before. This is roughly similar to how streaming compilation works, with the QoL improvement that Source owns the CodeCacheConsumeTask and therefore we can reuse the same Compile method and do the off-thread finalization behind the scenes inside Compile. On the v8::internal side, ScriptCompiler::CodeCacheConsumeTask wraps a v8::internal::BackgroundDeserializeTask, which has a Run and a Finish method. The Run creates a LocalIsolate (again, similar to BackgroundCompileTask), calls some helpers on CodeSerializer, and stores the pre-finalization result in a OffThreadDeserializeData structure. This stores Persistent Handles to the off-thread initialized SFI and a vector of Scripts needing fixing up, and it owns the PersistentHandles object which owns those Handles. Finally, the Finish method consumes this OffThreadDeserializeData structure, fixes up Scripts, moves the SFI Handle into the caller HandleScope, and that's it. Since we don't yet have the source at off-thread deserialization time, the various code cache sanity checks are done without the source hash when deserializing, and the Finish method re-does them now that the source is available. Bug: chromium:1075999 Change-Id: If1faf35ba3ef840fa4e735581d0b29c96c1d5fc8 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3067322 Commit-Queue: Leszek Swirski <leszeks@chromium.org> Reviewed-by: Omer Katz <omerkatz@chromium.org> Reviewed-by: Jakob Gruber <jgruber@chromium.org> Reviewed-by: Camillo Bruni <cbruni@chromium.org> Cr-Commit-Position: refs/heads/master@{#76155}
2021-08-06 13:53:48 +00:00
// Copyright 2021 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include "include/libplatform/libplatform.h"
Reland "[include] Split out v8.h" This is a reland of d1b27019d3bf86360ea838c317f8505fac6d3a7e Fixes include: Adding missing file to bazel build Forward-declaring classing before friend-classing them to fix win/gcc Add missing v8-isolate.h include for vtune builds Original change's description: > [include] Split out v8.h > > This moves every single class/function out of include/v8.h into a > separate header in include/, which v8.h then includes so that > externally nothing appears to have changed. > > Every include of v8.h from inside v8 has been changed to a more > fine-grained include. > > Previously inline functions defined at the bottom of v8.h would call > private non-inline functions in the V8 class. Since that class is now > in v8-initialization.h and is rarely included (as that would create > dependency cycles), this is not possible and so those methods have been > moved out of the V8 class into the namespace v8::api_internal. > > None of the previous files in include/ now #include v8.h, which means > if embedders were relying on this transitive dependency then it will > give compile failures. > > v8-inspector.h does depend on v8-scripts.h for the time being to ensure > that Chrome continue to compile but that change will be reverted once > those transitive #includes in chrome are changed to include it directly. > > Full design: > https://docs.google.com/document/d/1rTD--I8hCAr-Rho1WTumZzFKaDpEp0IJ8ejZtk4nJdA/edit?usp=sharing > > Bug: v8:11965 > Change-Id: I53b84b29581632710edc80eb11f819c2097a2877 > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3097448 > Reviewed-by: Yang Guo <yangguo@chromium.org> > Reviewed-by: Camillo Bruni <cbruni@chromium.org> > Reviewed-by: Jakob Kummerow <jkummerow@chromium.org> > Reviewed-by: Leszek Swirski <leszeks@chromium.org> > Reviewed-by: Michael Lippautz <mlippautz@chromium.org> > Commit-Queue: Dan Elphick <delphick@chromium.org> > Cr-Commit-Position: refs/heads/main@{#76424} Cq-Include-Trybots: luci.v8.try:v8_linux_vtunejit Bug: v8:11965 Change-Id: I99f5d3a73bf8fe25b650adfaf9567dc4e44a09e6 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3113629 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Reviewed-by: Camillo Bruni <cbruni@chromium.org> Reviewed-by: Michael Lippautz <mlippautz@chromium.org> Reviewed-by: Jakob Kummerow <jkummerow@chromium.org> Reviewed-by: Simon Zünd <szuend@chromium.org> Commit-Queue: Dan Elphick <delphick@chromium.org> Cr-Commit-Position: refs/heads/main@{#76460}
2021-08-23 13:01:06 +00:00
#include "include/v8-context.h"
#include "include/v8-function.h"
#include "include/v8-isolate.h"
#include "include/v8-local-handle.h"
[api] Add API for off-thread code cache deserialization To consume a code cache off-thread 1. The embedder creates a CachedData object wrapping the data blob. 2. The embedder calls ScriptCompiler::StartConsumingCodeCache with the CachedData, and receives a ScriptCompiler::CodeCacheConsumeTask which takes ownership of the CachedData. 3. The embedder calls ScriptCompiler::CodeCacheConsumeTask::Run on a different thread. 4. Once this completes, the embedded passes the completed task as an optional argument into Source constructor, and calls Compile as before. This is roughly similar to how streaming compilation works, with the QoL improvement that Source owns the CodeCacheConsumeTask and therefore we can reuse the same Compile method and do the off-thread finalization behind the scenes inside Compile. On the v8::internal side, ScriptCompiler::CodeCacheConsumeTask wraps a v8::internal::BackgroundDeserializeTask, which has a Run and a Finish method. The Run creates a LocalIsolate (again, similar to BackgroundCompileTask), calls some helpers on CodeSerializer, and stores the pre-finalization result in a OffThreadDeserializeData structure. This stores Persistent Handles to the off-thread initialized SFI and a vector of Scripts needing fixing up, and it owns the PersistentHandles object which owns those Handles. Finally, the Finish method consumes this OffThreadDeserializeData structure, fixes up Scripts, moves the SFI Handle into the caller HandleScope, and that's it. Since we don't yet have the source at off-thread deserialization time, the various code cache sanity checks are done without the source hash when deserializing, and the Finish method re-does them now that the source is available. Bug: chromium:1075999 Change-Id: If1faf35ba3ef840fa4e735581d0b29c96c1d5fc8 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3067322 Commit-Queue: Leszek Swirski <leszeks@chromium.org> Reviewed-by: Omer Katz <omerkatz@chromium.org> Reviewed-by: Jakob Gruber <jgruber@chromium.org> Reviewed-by: Camillo Bruni <cbruni@chromium.org> Cr-Commit-Position: refs/heads/master@{#76155}
2021-08-06 13:53:48 +00:00
#include "include/v8-platform.h"
Reland "[include] Split out v8.h" This is a reland of d1b27019d3bf86360ea838c317f8505fac6d3a7e Fixes include: Adding missing file to bazel build Forward-declaring classing before friend-classing them to fix win/gcc Add missing v8-isolate.h include for vtune builds Original change's description: > [include] Split out v8.h > > This moves every single class/function out of include/v8.h into a > separate header in include/, which v8.h then includes so that > externally nothing appears to have changed. > > Every include of v8.h from inside v8 has been changed to a more > fine-grained include. > > Previously inline functions defined at the bottom of v8.h would call > private non-inline functions in the V8 class. Since that class is now > in v8-initialization.h and is rarely included (as that would create > dependency cycles), this is not possible and so those methods have been > moved out of the V8 class into the namespace v8::api_internal. > > None of the previous files in include/ now #include v8.h, which means > if embedders were relying on this transitive dependency then it will > give compile failures. > > v8-inspector.h does depend on v8-scripts.h for the time being to ensure > that Chrome continue to compile but that change will be reverted once > those transitive #includes in chrome are changed to include it directly. > > Full design: > https://docs.google.com/document/d/1rTD--I8hCAr-Rho1WTumZzFKaDpEp0IJ8ejZtk4nJdA/edit?usp=sharing > > Bug: v8:11965 > Change-Id: I53b84b29581632710edc80eb11f819c2097a2877 > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3097448 > Reviewed-by: Yang Guo <yangguo@chromium.org> > Reviewed-by: Camillo Bruni <cbruni@chromium.org> > Reviewed-by: Jakob Kummerow <jkummerow@chromium.org> > Reviewed-by: Leszek Swirski <leszeks@chromium.org> > Reviewed-by: Michael Lippautz <mlippautz@chromium.org> > Commit-Queue: Dan Elphick <delphick@chromium.org> > Cr-Commit-Position: refs/heads/main@{#76424} Cq-Include-Trybots: luci.v8.try:v8_linux_vtunejit Bug: v8:11965 Change-Id: I99f5d3a73bf8fe25b650adfaf9567dc4e44a09e6 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3113629 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Reviewed-by: Camillo Bruni <cbruni@chromium.org> Reviewed-by: Michael Lippautz <mlippautz@chromium.org> Reviewed-by: Jakob Kummerow <jkummerow@chromium.org> Reviewed-by: Simon Zünd <szuend@chromium.org> Commit-Queue: Dan Elphick <delphick@chromium.org> Cr-Commit-Position: refs/heads/main@{#76460}
2021-08-23 13:01:06 +00:00
#include "include/v8-primitive.h"
#include "include/v8-script.h"
[api] Add API for off-thread code cache deserialization To consume a code cache off-thread 1. The embedder creates a CachedData object wrapping the data blob. 2. The embedder calls ScriptCompiler::StartConsumingCodeCache with the CachedData, and receives a ScriptCompiler::CodeCacheConsumeTask which takes ownership of the CachedData. 3. The embedder calls ScriptCompiler::CodeCacheConsumeTask::Run on a different thread. 4. Once this completes, the embedded passes the completed task as an optional argument into Source constructor, and calls Compile as before. This is roughly similar to how streaming compilation works, with the QoL improvement that Source owns the CodeCacheConsumeTask and therefore we can reuse the same Compile method and do the off-thread finalization behind the scenes inside Compile. On the v8::internal side, ScriptCompiler::CodeCacheConsumeTask wraps a v8::internal::BackgroundDeserializeTask, which has a Run and a Finish method. The Run creates a LocalIsolate (again, similar to BackgroundCompileTask), calls some helpers on CodeSerializer, and stores the pre-finalization result in a OffThreadDeserializeData structure. This stores Persistent Handles to the off-thread initialized SFI and a vector of Scripts needing fixing up, and it owns the PersistentHandles object which owns those Handles. Finally, the Finish method consumes this OffThreadDeserializeData structure, fixes up Scripts, moves the SFI Handle into the caller HandleScope, and that's it. Since we don't yet have the source at off-thread deserialization time, the various code cache sanity checks are done without the source hash when deserializing, and the Finish method re-does them now that the source is available. Bug: chromium:1075999 Change-Id: If1faf35ba3ef840fa4e735581d0b29c96c1d5fc8 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3067322 Commit-Queue: Leszek Swirski <leszeks@chromium.org> Reviewed-by: Omer Katz <omerkatz@chromium.org> Reviewed-by: Jakob Gruber <jgruber@chromium.org> Reviewed-by: Camillo Bruni <cbruni@chromium.org> Cr-Commit-Position: refs/heads/master@{#76155}
2021-08-06 13:53:48 +00:00
#include "test/unittests/test-utils.h"
#include "testing/gtest/include/gtest/gtest.h"
namespace v8 {
class DeserializeTest : public testing::Test {
public:
class IsolateAndContextScope {
public:
explicit IsolateAndContextScope(DeserializeTest* test)
: test_(test),
isolate_wrapper_(kNoCounters),
isolate_scope_(isolate_wrapper_.isolate()),
handle_scope_(isolate_wrapper_.isolate()),
context_(Context::New(isolate_wrapper_.isolate())),
context_scope_(context_) {
CHECK_NULL(test->isolate_);
CHECK(test->context_.IsEmpty());
test->isolate_ = isolate_wrapper_.isolate();
test->context_ = context_;
}
~IsolateAndContextScope() {
test_->isolate_ = nullptr;
test_->context_ = {};
}
private:
DeserializeTest* test_;
v8::IsolateWrapper isolate_wrapper_;
v8::Isolate::Scope isolate_scope_;
v8::HandleScope handle_scope_;
v8::Local<v8::Context> context_;
v8::Context::Scope context_scope_;
};
Local<String> NewString(const char* val) {
return String::NewFromUtf8(isolate(), val).ToLocalChecked();
}
Local<Value> RunGlobalFunc(const char* name) {
Local<Value> func_val =
context()->Global()->Get(context(), NewString(name)).ToLocalChecked();
CHECK(func_val->IsFunction());
Local<Function> func = Local<Function>::Cast(func_val);
return func->Call(context(), Undefined(isolate()), 0, nullptr)
.ToLocalChecked();
}
Isolate* isolate() { return isolate_; }
v8::Local<v8::Context> context() { return context_.ToLocalChecked(); }
private:
Isolate* isolate_ = nullptr;
v8::MaybeLocal<v8::Context> context_;
};
// Check that deserialization works.
TEST_F(DeserializeTest, Deserialize) {
std::unique_ptr<v8::ScriptCompiler::CachedData> cached_data;
{
IsolateAndContextScope scope(this);
Local<String> source_code = NewString("function foo() { return 42; }");
Local<Script> script =
Script::Compile(context(), source_code).ToLocalChecked();
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("foo"), Integer::New(isolate(), 42));
cached_data.reset(
ScriptCompiler::CreateCodeCache(script->GetUnboundScript()));
}
{
IsolateAndContextScope scope(this);
Local<String> source_code = NewString("function foo() { return 42; }");
ScriptCompiler::Source source(source_code, cached_data.release());
Local<Script> script =
ScriptCompiler::Compile(context(), &source,
ScriptCompiler::kConsumeCodeCache)
.ToLocalChecked();
CHECK(!source.GetCachedData()->rejected);
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("foo"), v8::Integer::New(isolate(), 42));
}
}
// Check that deserialization with a different script rejects the cache but
// still works via standard compilation.
TEST_F(DeserializeTest, DeserializeRejectsDifferentSource) {
std::unique_ptr<v8::ScriptCompiler::CachedData> cached_data;
{
IsolateAndContextScope scope(this);
Local<String> source_code = NewString("function foo() { return 42; }");
Local<Script> script =
Script::Compile(context(), source_code).ToLocalChecked();
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("foo"), Integer::New(isolate(), 42));
cached_data.reset(
ScriptCompiler::CreateCodeCache(script->GetUnboundScript()));
}
{
IsolateAndContextScope scope(this);
// The source hash is based on the source length, so have to make sure that
// this is different here.
Local<String> source_code = NewString("function bar() { return 142; }");
ScriptCompiler::Source source(source_code, cached_data.release());
Local<Script> script =
ScriptCompiler::Compile(context(), &source,
ScriptCompiler::kConsumeCodeCache)
.ToLocalChecked();
CHECK(source.GetCachedData()->rejected);
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("bar"), v8::Integer::New(isolate(), 142));
}
}
class DeserializeThread : public base::Thread {
public:
explicit DeserializeThread(ScriptCompiler::ConsumeCodeCacheTask* task)
: Thread(base::Thread::Options("DeserializeThread")), task_(task) {}
void Run() override { task_->Run(); }
std::unique_ptr<ScriptCompiler::ConsumeCodeCacheTask> TakeTask() {
return std::move(task_);
}
private:
std::unique_ptr<ScriptCompiler::ConsumeCodeCacheTask> task_;
};
// Check that off-thread deserialization works.
TEST_F(DeserializeTest, OffThreadDeserialize) {
std::unique_ptr<v8::ScriptCompiler::CachedData> cached_data;
{
IsolateAndContextScope scope(this);
Local<String> source_code = NewString("function foo() { return 42; }");
Local<Script> script =
Script::Compile(context(), source_code).ToLocalChecked();
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("foo"), Integer::New(isolate(), 42));
cached_data.reset(
ScriptCompiler::CreateCodeCache(script->GetUnboundScript()));
}
{
IsolateAndContextScope scope(this);
DeserializeThread deserialize_thread(
ScriptCompiler::StartConsumingCodeCache(
isolate(), std::make_unique<ScriptCompiler::CachedData>(
cached_data->data, cached_data->length,
ScriptCompiler::CachedData::BufferNotOwned)));
CHECK(deserialize_thread.Start());
deserialize_thread.Join();
Local<String> source_code = NewString("function foo() { return 42; }");
ScriptCompiler::Source source(source_code, cached_data.release(),
deserialize_thread.TakeTask().release());
Local<Script> script =
ScriptCompiler::Compile(context(), &source,
ScriptCompiler::kConsumeCodeCache)
.ToLocalChecked();
CHECK(!source.GetCachedData()->rejected);
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("foo"), v8::Integer::New(isolate(), 42));
}
}
// Check that off-thread deserialization works.
TEST_F(DeserializeTest, OffThreadDeserializeRejectsDifferentSource) {
std::unique_ptr<v8::ScriptCompiler::CachedData> cached_data;
{
IsolateAndContextScope scope(this);
Local<String> source_code = NewString("function foo() { return 42; }");
Local<Script> script =
Script::Compile(context(), source_code).ToLocalChecked();
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("foo"), Integer::New(isolate(), 42));
cached_data.reset(
ScriptCompiler::CreateCodeCache(script->GetUnboundScript()));
}
{
IsolateAndContextScope scope(this);
DeserializeThread deserialize_thread(
ScriptCompiler::StartConsumingCodeCache(
isolate(), std::make_unique<ScriptCompiler::CachedData>(
cached_data->data, cached_data->length,
ScriptCompiler::CachedData::BufferNotOwned)));
CHECK(deserialize_thread.Start());
deserialize_thread.Join();
Local<String> source_code = NewString("function bar() { return 142; }");
ScriptCompiler::Source source(source_code, cached_data.release(),
deserialize_thread.TakeTask().release());
Local<Script> script =
ScriptCompiler::Compile(context(), &source,
ScriptCompiler::kConsumeCodeCache)
.ToLocalChecked();
CHECK(source.GetCachedData()->rejected);
CHECK(!script->Run(context()).IsEmpty());
CHECK_EQ(RunGlobalFunc("bar"), v8::Integer::New(isolate(), 142));
}
}
} // namespace v8