v8/tools/gcmole
Seth Brenith 0921e8f28b Reland "Add postmortem debugging helper library"
This is a reland of 517ab73fd7

Updates since original: now compressed pointers passed to the function
GetObjectProperties are required to be sign-extended. Previously, the
function allowed zero-extended values, but that led to ambiguity on
pointers like 0x88044919: is it compressed or is the heap range actually
centered on 0x100000000?

Original change's description:
> Add postmortem debugging helper library
>
> This change begins to implement the functionality described in
> https://docs.google.com/document/d/1evHnb1uLlSbvHAAsmOXyc25x3uh1DjgNa8u1RHvwVhk/edit#
> for investigating V8 state in crash dumps.
>
> This change adds a new library, v8_debug_helper, for providing platform-
> agnostic assistance with postmortem debugging. This library can be used
> by extensions built for debuggers such as WinDbg or lldb. Its public API
> is described by debug-helper.h; currently the only method it exposes is
> GetObjectProperties, but we'd like to add more functionality over time.
> The API surface is restricted to plain C-style structs and pointers, so
> that it's easy to link from a debugger extension built with a different
> toolchain.
>
> This change also adds a new cctest file to exercise some basic
> interaction with the new library.
>
> The API function GetObjectProperties takes an object pointer (which
> could be compressed, or weak, or a SMI), and returns a string
> description of the object and a list of properties the object contains.
> For now, the list of properties is entirely based on Torque object
> definitions, but we expect to add custom properties in future updates so
> that it can be easier to make sense of complex data structures such as
> dictionaries.
>
> GetObjectProperties does several things that are intended to generate
> somewhat useful results even in cases where memory may be corrupt or
> unavailable:
> - The caller may optionally provide a type string which will be used if
>   the memory for the object's Map is inaccessible.
> - All object pointers are compared against the list of known objects
>   generated by mkgrokdump. The caller may optionally provide the
>   pointers for the first pages of various heap spaces, to avoid spurious
>   matches. If those pointers are not provided, then any matches are
>   prefixed with "maybe" in the resulting description string, such as
>   "maybe UndefinedValue (0x4288000341 <Oddball>)".
>
> Bug: v8:9376
>
> Change-Id: Iebf3cc2dea3133c7811bcefcdf38d9458b02fded
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1628012
> Commit-Queue: Seth Brenith <seth.brenith@microsoft.com>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Michael Stanton <mvstanton@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#62882}

Bug: v8:9376
Change-Id: I866a1cc9d4c34bfe10c7b98462451fe69763cf3f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1717090
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Commit-Queue: Seth Brenith <seth.brenith@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#63008}
2019-07-31 14:30:19 +00:00
..
bootstrap.sh [gcmole] Update bootstrap script for static linkage. 2019-03-15 12:53:51 +00:00
BUILD.gn Reland "Add postmortem debugging helper library" 2019-07-31 14:30:19 +00:00
gccause.lua Adjust contents of kAheadOfTime to match write-barrier stub called from CompileArrayPushCall. 2011-10-04 11:38:12 +00:00
gcmole-test.cc Move handles-related files to src/handles 2019-05-23 06:00:15 +00:00
gcmole-tools.tar.gz.sha1 [tools] Update gcmole 2019-03-15 10:24:49 +00:00
gcmole.cc [gcmole] Switch assignment operator special case. 2019-03-13 11:09:45 +00:00
gcmole.lua Revert GC scheduling for external backing stores 2018-08-30 14:24:38 +00:00
Makefile [gcmole] Update bootstrap script for static linkage. 2019-03-15 12:53:51 +00:00
package.sh [gcmole] Provide packaging script for gcmole deployment. 2019-03-14 13:50:30 +00:00
parallel.py Preparing v8 to use with python3 /tools 2019-02-19 09:12:07 +00:00
README [gcmole] Provide packaging script for gcmole deployment. 2019-03-14 13:50:30 +00:00
run-gcmole.py [torque] Rename ...from-dsl... to ...tq... 2019-05-14 14:49:23 +00:00

DESCRIPTION -------------------------------------------------------------------

gcmole is a simple static analysis tool used to find possible evaluation order 
dependent GC-unsafe places in the V8 codebase.

For example the following code is GC-unsafe:

    Handle<Object> Foo();  // Assume Foo can trigger a GC.
    void Bar(Object*, Object*);

    Handle<Object> baz;
    baz->Qux(*Foo());  // (a)
    Bar(*Foo(), *baz);  // (b)

Both in cases (a) and (b) compiler is free to evaluate call arguments (that 
includes receiver) in any order. That means it can dereference baz before 
calling to Foo and save a raw pointer to a heap object in the register or 
on the stack.  

PREREQUISITES -----------------------------------------------------------------

(1) Install Lua 5.1

    $ sudo apt-get install lua5.1

(2) Get LLVM 8.0 and Clang 8.0 sources and build them.

    Follow the instructions on http://clang.llvm.org/get_started.html.

    Make sure to pass -DCMAKE_BUILD_TYPE=Release to cmake to get Release build 
    instead of a Debug one.

(3) Build gcmole Clang plugin (libgcmole.so)

    In the tools/gcmole directory execute the following command:

    $ BUILD_ROOT=<path> LLVM_SRC_ROOT=<path> CLANG_SRC_ROOT=<path> make

(*) Note that steps (2) and (3) can also be achieved by just using the included
    bootstrapping script in this directory:

    $ ./tools/gcmole/bootstrap.sh

    This will use "third_party/llvm+clang-build" as a build directory and checkout
    required sources in the "third_party" directory.

USING GCMOLE ------------------------------------------------------------------

gcmole consists of driver script written in Lua and Clang plugin that does
C++ AST processing. Plugin (libgcmole.so) is expected to be in the same
folder as driver (gcmole.lua).

To start analysis cd into the root of v8 checkout and execute the following
command:

CLANG_BIN=<path-to-clang-bin-folder> lua tools/gcmole/gcmole.lua [<arch>]

where arch should be one of architectures supported by V8 (arm, ia32, x64).

Analysis will be performed in 2 stages: 

- on the first stage driver will parse all files and build a global callgraph 
approximation to find all functions that might potentially cause GC, list
of this functions will be written into gcsuspects file.

- on the second stage driver will parse all files again and will locate all 
callsites that might be GC-unsafe based on the list of functions causing GC. 
Such places are marked with a "Possible problem with evaluation order." 
warning. Messages "Failed to resolve v8::internal::Object" are benign and 
can be ignored.

If any errors were found driver exits with non-zero status.

PACKAGING ---------------------------------------------------------------------

gcmole is deployed on V8's buildbot infrastructure to run it as part of the
continuous integration. A pre-built package of gcmole together with Clang is
hosted on Google Cloud Storage for this purpose. To update this package to a
newer version, use the provided packaging script:

    $ ./tools/gcmole/package.sh

This will create a new "tools/gcmole/gcmole-tools.tar.gz" package with the
corresponding SHA1 sum suitable to be used for this purpose. It assumes that
Clang was built in "third_party/llvm+clang-build" (e.g. by the bootstrapping
script "bootstrap.sh" mentioned above).

TROUBLESHOOTING ---------------------------------------------------------------

gcmole is tighly coupled with the AST structure that Clang produces. Therefore
when upgrading to a newer Clang version, it might start producing bogus output
or completely stop outputting warnings. In such occasion, one might start the
debugging process by checking weather a new AST node type is introduced which
is currently not supported by gcmole. Insert the following code at the end of
the FunctionAnalyzer::VisitExpr method to see the unsupported AST class(es)
and the source position which generates them:

    if (expr) {
      clang::Stmt::StmtClass stmtClass = expr->getStmtClass();
      d_.Report(clang::FullSourceLoc(expr->getExprLoc(), sm_),
        d_.getCustomDiagID(clang::DiagnosticsEngine::Remark, "%0")) << stmtClass;
    }

For instance, gcmole currently doesn't support AtomicExprClass statements
introduced for atomic operations.

A convenient way to observe the AST generated by Clang is to pass the following
flags when invoking clang++

    -Xclang -ast-dump -fsyntax-only