Step 1 (of 3-ish): Basic ETW Instrumentation in V8

Design doc:
https://docs.google.com/document/d/1xkXj94iExFgLWc_OszTNyNGi523ARaKMWPZTeomhI4U
A lot has changed since the last patchset! I recommend revisiting this
design doc and reading the parts in green. I explain the roadmap for
what changes to expect from ETW instrumentation as well as the
instrumentation of this particular CL.

I'll do my best to answer any further questions anyone has about my
particular instrumentation or ETW in general :)

---

This is the first of a series of changelists to round out ETW
instrumentation for V8.

This changelist represents the most minimal change needed to instrument
ETW in V8. In particular, it:
  - defines and registers the ETW provider,
  - interacts minimally with the rest of V8, by hooking into the
    existing TracingController::AddTraceEvent function,
  - is designed with a platform-agnostic layer, so that event tracers
    for other platforms can be instrumented in teh future.

Some notes on instrumentation (aka I copied stuff from the design doc):

We make heavy use of the TraceLogging API to log events. It differs from
previous methods of emitting ETW events in that it doesn<E2><80><99>t
require the overhead of a separate manifest file to keep track of
metadata; rather, events using this API are self-descriptive.

Here are the five major steps to instrument the TraceLogging API:
  - Forward declare the provider (from provider-win.h)
  - Define the provider in a .cc file (from provider-win.cc)
  - Register the provider (called from v8.cc).
  - Write events (called from libplatform/tracing-controller.cc)
  - Unregister the provider (called from v8.cc)

At the base, we have an abstract provider class that encapsulates the
functionality of an event provider. These are things like registering
and unregistering the provider, and the actual event-logging.

The provider class is split into provider-win and provider-mac
(currently not instantiated) classes, with OS-dependent implementations
of the above functions.

In particular, the TraceLogging API is used only in provider-win. It is
here that we forward declare and define the provider, as well as write
ETW events.

Finally, there is a v8-provider class that serves as a top-level API and
is exposed to the rest of V8. It acts as a wrapper for the
platform-specific providers.

The .wprp file is needed so that Windows Performance Recorder knows how
to capture our events.

Some considerations:
  - Is TracingController::AddTraceEvent the best place from which to
    write my events?
  - Is src/libplatform/tracing the best place to put my instrumentation?
  - Right now, I fail the preupload because of this, which tells me my
    files are probably not in the best location:

You added one or more #includes that violate checkdeps rules.
src\init\v8.cc Illegal include: "src/libplatform/tracing/v8-provider.h"
Because of "-src/libplatform" from src's include_rules.

Change-Id: Id53e4a034c9e526524a17000da0a647a95d93edf
Bug: v8:11043
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2233407
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Commit-Queue: Sara Tang <sartang@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#71918}

BUILD.gn

@@ -45,6 +45,12 @@ declare_args() {
   # Sets --DV8_LITE_MODE.
   v8_enable_lite_mode = false
 
+  # Sets -DSYSTEM_INSTRUMENTATION. Enables OS-dependent event tracing
+  v8_enable_system_instrumentation = false
+
+  # Sets the GUID for the ETW provider
+  v8_etw_guid = ""
+
   # Sets -DVERIFY_HEAP.
   v8_enable_verify_heap = ""
 
@@ -695,6 +701,12 @@ config("features") {
   if (v8_dict_mode_prototypes) {
     defines += [ "V8_DICT_MODE_PROTOTYPES" ]
   }
+  if (v8_enable_system_instrumentation) {
+    defines += [ "V8_ENABLE_SYSTEM_INSTRUMENTATION" ]
+  }
+  if (v8_etw_guid != "") {
+    defines += [ "V8_ETW_GUID=\"$v8_etw_guid\"" ]
+  }
 }
 
 config("toolchain") {
@@ -4262,6 +4274,10 @@ v8_component("v8_libbase") {
       "ws2_32.lib",
     ]
 
+    if (v8_enable_system_instrumentation) {
+      libs += [ "advapi32.lib" ]  # Needed for TraceLoggingProvider.h
+    }
+
     data_deps += [ "//build/win:runtime_libs" ]
   }
 
@@ -4304,6 +4320,8 @@ v8_component("v8_libplatform") {
     "src/libplatform/delayed-task-queue.h",
     "src/libplatform/task-queue.cc",
     "src/libplatform/task-queue.h",
+    "src/libplatform/tracing/recorder-default.cc",
+    "src/libplatform/tracing/recorder.h",
     "src/libplatform/tracing/trace-buffer.cc",
     "src/libplatform/tracing/trace-buffer.h",
     "src/libplatform/tracing/trace-config.cc",
@@ -4335,6 +4353,7 @@ v8_component("v8_libplatform") {
   if (v8_use_perfetto) {
     sources -= [
       "//base/trace_event/common/trace_event_common.h",
+      "src/libplatform/tracing/recorder-default.cc",
       "src/libplatform/tracing/trace-buffer.cc",
       "src/libplatform/tracing/trace-buffer.h",
       "src/libplatform/tracing/trace-object.cc",
@@ -4349,6 +4368,9 @@ v8_component("v8_libplatform") {
       # TODO(skyostil): Switch TraceEventListener to protozero.
       "//third_party/perfetto/protos/perfetto/trace:lite",
     ]
+  } else if (is_win) {
+    sources -= [ "src/libplatform/tracing/recorder-default.cc" ]
+    sources += [ "src/libplatform/tracing/recorder-win.cc" ]
   }
 }
 

include/libplatform/v8-tracing.h

@@ -125,6 +125,8 @@ class V8_PLATFORM_EXPORT TraceWriter {
   static TraceWriter* CreateJSONTraceWriter(std::ostream& stream,
                                             const std::string& tag);
 
+  static TraceWriter* CreateSystemInstrumentationTraceWriter();
+
  private:
   // Disallow copy and assign
   TraceWriter(const TraceWriter&) = delete;

src/d8/d8.cc

@@ -3645,6 +3645,12 @@ bool Shell::SetOptions(int argc, char* argv[]) {
     } else if (strcmp(argv[i], "--fuzzy-module-file-extensions") == 0) {
       options.fuzzy_module_file_extensions = true;
       argv[i] = nullptr;
+#ifdef V8_ENABLE_SYSTEM_INSTRUMENTATION
+    } else if (strcmp(argv[i], "--enable-system-instrumentation") == 0) {
+      options.enable_system_instrumentation = true;
+      options.trace_enabled = true;
+      argv[i] = nullptr;
+#endif
     }
   }
 
@@ -4216,8 +4222,8 @@ int Shell::Main(int argc, char* argv[]) {
           ? v8::platform::InProcessStackDumping::kDisabled
           : v8::platform::InProcessStackDumping::kEnabled;
 
-  std::unique_ptr<platform::tracing::TracingController> tracing;
   std::ofstream trace_file;
+  std::unique_ptr<platform::tracing::TracingController> tracing;
   if (options.trace_enabled && !i::FLAG_verify_predictable) {
     tracing = std::make_unique<platform::tracing::TracingController>();
     const char* trace_path =
@@ -4238,10 +4244,23 @@ int Shell::Main(int argc, char* argv[]) {
 
     tracing->InitializeForPerfetto(&trace_file);
 #else
-    platform::tracing::TraceBuffer* trace_buffer =
-        platform::tracing::TraceBuffer::CreateTraceBufferRingBuffer(
-            platform::tracing::TraceBuffer::kRingBufferChunks,
-            platform::tracing::TraceWriter::CreateJSONTraceWriter(trace_file));
+    platform::tracing::TraceBuffer* trace_buffer = nullptr;
+#if defined(V8_ENABLE_SYSTEM_INSTRUMENTATION)
+    if (options.enable_system_instrumentation) {
+      trace_buffer =
+          platform::tracing::TraceBuffer::CreateTraceBufferRingBuffer(
+              platform::tracing::TraceBuffer::kRingBufferChunks,
+              platform::tracing::TraceWriter::
+                  CreateSystemInstrumentationTraceWriter());
+    }
+#endif  // V8_ENABLE_SYSTEM_INSTRUMENTATION
+    if (!trace_buffer) {
+      trace_buffer =
+          platform::tracing::TraceBuffer::CreateTraceBufferRingBuffer(
+              platform::tracing::TraceBuffer::kRingBufferChunks,
+              platform::tracing::TraceWriter::CreateJSONTraceWriter(
+                  trace_file));
+    }
     tracing->Initialize(trace_buffer);
 #endif  // V8_USE_PERFETTO
   }
@@ -4358,6 +4377,9 @@ int Shell::Main(int argc, char* argv[]) {
         } else {
           trace_config =
               platform::tracing::TraceConfig::CreateDefaultTraceConfig();
+          if (options.enable_system_instrumentation) {
+            trace_config->AddIncludedCategory("disabled-by-default-v8.compile");
+          }
         }
         tracing_controller->StartTracing(trace_config);
       }

src/d8/d8.h

@@ -387,6 +387,8 @@ class ShellOptions {
   DisallowReassignment<bool> cpu_profiler_print = {"cpu-profiler-print", false};
   DisallowReassignment<bool> fuzzy_module_file_extensions = {
       "fuzzy-module-file-extensions", true};
+  DisallowReassignment<bool> enable_system_instrumentation = {
+      "enable-system-instrumentation", false};
 };
 
 class Shell : public i::AllStatic {

src/libplatform/tracing/recorder-default.cc

@@ -0,0 +1,25 @@
+// Copyright 2020 the V8 project authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+#ifndef V8_LIBPLATFORM_TRACING_RECORDER_DEFAULT_H_
+#define V8_LIBPLATFORM_TRACING_RECORDER_DEFAULT_H_
+
+#include "src/libplatform/tracing/recorder.h"
+
+namespace v8 {
+namespace platform {
+namespace tracing {
+
+Recorder::Recorder() {}
+Recorder::~Recorder() {}
+
+bool Recorder::IsEnabled() { return false; }
+bool Recorder::IsEnabled(const uint8_t level) { return false; }
+
+void Recorder::AddEvent(TraceObject* trace_event) {}
+
+}  // namespace tracing
+}  // namespace platform
+}  // namespace v8
+
+#endif  // V8_LIBPLATFORM_TRACING_RECORDER_DEFAULT_H_

src/libplatform/tracing/recorder-win.cc

@@ -0,0 +1,72 @@
+// Copyright 2020 the V8 project authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+#ifndef V8_LIBPLATFORM_TRACING_RECORDER_WIN_H_
+#define V8_LIBPLATFORM_TRACING_RECORDER_WIN_H_
+
+#include <windows.h>
+#include <TraceLoggingProvider.h>
+
+#include "src/libplatform/tracing/recorder.h"
+
+#if defined(__clang__)
+#pragma clang diagnostic ignored "-Wc++98-compat-extra-semi"
+#endif
+
+#ifndef V8_ETW_GUID
+#define V8_ETW_GUID \
+  0x57277741, 0x3638, 0x4A4B, 0xBD, 0xBA, 0x0A, 0xC6, 0xE4, 0x5D, 0xA5, 0x6C
+#endif
+
+namespace v8 {
+namespace platform {
+namespace tracing {
+
+TRACELOGGING_DECLARE_PROVIDER(g_v8Provider);
+
+TRACELOGGING_DEFINE_PROVIDER(g_v8Provider, "V8.js", (V8_ETW_GUID));
+
+Recorder::Recorder() { TraceLoggingRegister(g_v8Provider); }
+
+Recorder::~Recorder() {
+  if (g_v8Provider) {
+    TraceLoggingUnregister(g_v8Provider);
+  }
+}
+
+bool Recorder::IsEnabled() {
+  return TraceLoggingProviderEnabled(g_v8Provider, 0, 0);
+}
+
+bool Recorder::IsEnabled(const uint8_t level) {
+  return TraceLoggingProviderEnabled(g_v8Provider, level, 0);
+}
+
+void Recorder::AddEvent(TraceObject* trace_event) {
+  // TODO(sartang@microsoft.com): Figure out how to write the conditional
+  // arguments
+  wchar_t* wName = new wchar_t[4096];
+  MultiByteToWideChar(CP_ACP, 0, trace_event->name(), -1, wName, 4096);
+
+  wchar_t* wCategoryGroupName = new wchar_t[4096];
+  MultiByteToWideChar(CP_ACP, 0,
+                      TracingController::GetCategoryGroupName(
+                          trace_event->category_enabled_flag()),
+                      -1, wCategoryGroupName, 4096);
+
+  TraceLoggingWrite(g_v8Provider, "", TraceLoggingValue(wName, "Event Name"),
+                    TraceLoggingValue(trace_event->pid(), "pid"),
+                    TraceLoggingValue(trace_event->tid(), "tid"),
+                    TraceLoggingValue(trace_event->ts(), "ts"),
+                    TraceLoggingValue(trace_event->tts(), "tts"),
+                    TraceLoggingValue(trace_event->phase(), "phase"),
+                    TraceLoggingValue(wCategoryGroupName, "category"),
+                    TraceLoggingValue(trace_event->duration(), "dur"),
+                    TraceLoggingValue(trace_event->cpu_duration(), "tdur"));
+}
+
+}  // namespace tracing
+}  // namespace platform
+}  // namespace v8
+
+#endif  // V8_LIBPLATFORM_TRACING_RECORDER_WIN_H_

src/libplatform/tracing/recorder.h

@@ -0,0 +1,37 @@
+// Copyright 2020 the V8 project authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef V8_LIBPLATFORM_TRACING_RECORDER_H_
+#define V8_LIBPLATFORM_TRACING_RECORDER_H_
+
+#include <stdint.h>
+
+#include "include/libplatform/v8-tracing.h"
+
+namespace v8 {
+namespace platform {
+namespace tracing {
+
+// This class serves as a base class for emitting events to system event
+// controllers: ETW for Windows, Signposts on Mac (to be implemented). It is
+// enabled by turning on both the ENABLE_SYSTEM_INSTRUMENTATION build flag and
+// the --enable-system-instrumentation command line flag. When enabled, it is
+// called from within SystemInstrumentationTraceWriter and replaces the
+// JSONTraceWriter for event-tracing.
+class Recorder {
+ public:
+  Recorder();
+  ~Recorder();
+
+  bool IsEnabled();
+  bool IsEnabled(const uint8_t level);
+
+  void AddEvent(TraceObject* trace_event);
+};
+
+}  // namespace tracing
+}  // namespace platform
+}  // namespace v8
+
+#endif  // V8_LIBPLATFORM_TRACING_RECORDER_H_

src/libplatform/tracing/trace-writer.cc

@@ -9,6 +9,7 @@
 #include "base/trace_event/common/trace_event_common.h"
 #include "include/v8-platform.h"
 #include "src/base/platform/platform.h"
+#include "src/libplatform/tracing/recorder.h"
 
 namespace v8 {
 namespace platform {
@@ -190,6 +191,27 @@ TraceWriter* TraceWriter::CreateJSONTraceWriter(std::ostream& stream,
   return new JSONTraceWriter(stream, tag);
 }
 
+SystemInstrumentationTraceWriter::SystemInstrumentationTraceWriter() {
+  recorder_ = std::make_unique<Recorder>();
+}
+
+SystemInstrumentationTraceWriter::~SystemInstrumentationTraceWriter() {
+  recorder_.reset(nullptr);
+}
+
+void SystemInstrumentationTraceWriter::AppendTraceEvent(
+    TraceObject* trace_event) {
+  if (recorder_->IsEnabled()) {
+    recorder_->AddEvent(trace_event);
+  }
+}
+
+void SystemInstrumentationTraceWriter::Flush() {}
+
+TraceWriter* TraceWriter::CreateSystemInstrumentationTraceWriter() {
+  return new SystemInstrumentationTraceWriter();
+}
+
 }  // namespace tracing
 }  // namespace platform
 }  // namespace v8

src/libplatform/tracing/trace-writer.h

@@ -11,6 +11,8 @@ namespace v8 {
 namespace platform {
 namespace tracing {
 
+class Recorder;
+
 class JSONTraceWriter : public TraceWriter {
  public:
   explicit JSONTraceWriter(std::ostream& stream);
@@ -27,6 +29,17 @@ class JSONTraceWriter : public TraceWriter {
   bool append_comma_ = false;
 };
 
+class SystemInstrumentationTraceWriter : public TraceWriter {
+ public:
+  SystemInstrumentationTraceWriter();
+  ~SystemInstrumentationTraceWriter() override;
+  void AppendTraceEvent(TraceObject* trace_event) override;
+  void Flush() override;
+
+ private:
+  std::unique_ptr<Recorder> recorder_;
+};
+
 }  // namespace tracing
 }  // namespace platform
 }  // namespace v8

tools/wpr.wprp

@@ -0,0 +1,68 @@
+<?xml version="1.0" encoding="utf-8"?>
+<WindowsPerformanceRecorder Version="1.0">
+  <!--
+Note: The following utilities are usually installed to: "C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit"
+See https://docs.microsoft.com/en-us/windows-hardware/test/wpt/ for an overview of the tools.
+
+Start and stop the trace:
+
+  wpr -start wpr.wprp!V8js
+...run scenario...
+  wpr -stop v8js.etl
+
+You can also run "wpr -status collectors details" while recording to check on status.
+
+Note: If you have issues with the command line, run WRPUI, and load this profile via the dialog.
+
+Run the below to open the trace:
+
+  wpa v8js.etl
+
+Set _NT_SYMBOL_PATH to a value such as "C:\src\v8\v8\out.gn\x64.debug;srv*c:\symbols*https://msdl.microsoft.com/download/symbols" first.
+Append "-symcacheonly" on the WPA command to save it trying to reload prior failed symbols on subsequent runs.
+
+For details on editing this file, see https://docs.microsoft.com/en-us/windows-hardware/test/wpt/authoring-recording-profiles
+  -->
+  <Profiles>
+    <SystemCollector Id="SystemCollector" Name="NT Kernel Logger">
+      <BufferSize Value="1024"/>
+      <Buffers Value="384"/>
+    </SystemCollector>
+    <EventCollector Id="EventCollector_V8js" Name="V8js Event Collector">
+      <BufferSize Value="1024"/>
+      <Buffers Value="256"/>
+    </EventCollector>
+    <SystemProvider Id="SystemProvider">
+      <Keywords>
+        <Keyword Value="ProcessThread"/>
+        <Keyword Value="Loader"/>
+        <Keyword Value="SampledProfile"/>
+        <Keyword Value="ReadyThread"/>
+        <Keyword Value="CSwitch"/>
+        <Keyword Value="DiskIOInit"/>
+        <Keyword Value="FileIOInit"/>
+        <Keyword Value="HardFaults"/>
+      </Keywords>
+      <Stacks>
+        <!-- See https://docs.microsoft.com/en-us/windows-hardware/test/wpt/stack-wpa for options -->
+        <Stack Value="SampledProfile"/>
+        <Stack Value="ReadyThread"/>
+        <Stack Value="CSwitch"/>
+      </Stacks>
+    </SystemProvider>
+    <EventProvider Id="Provider_V8js" Name="57277741-3638-4A4B-BDBA-0AC6E45DA56C" Level="5" Stack="true"></EventProvider>
+    <Profile Id="V8js.Verbose.File" Name="V8js" DetailLevel="Verbose" LoggingMode="File" Description="V8.js profile">
+      <Collectors>
+        <SystemCollectorId Value="SystemCollector">
+          <SystemProviderId Value="SystemProvider"></SystemProviderId>
+        </SystemCollectorId>
+        <EventCollectorId Value="EventCollector_V8js">
+          <EventProviders>
+            <EventProviderId Value="Provider_V8js"></EventProviderId>
+          </EventProviders>
+        </EventCollectorId>
+      </Collectors>
+    </Profile>
+    <Profile Id="V8js.Verbose.Memory" Base="V8js.Verbose.File" Name="V8js" DetailLevel="Verbose" LoggingMode="Memory" Description="V8.js profile"></Profile>
+  </Profiles>
+</WindowsPerformanceRecorder>