v8/tools
Michael Achenbach 2d51120a9d [gcmole] Add API for running on multiple hosts
This adds new modes to run gcmole on multiple hosts:
- collect: Parse source files and store partial raw call graph in
a binary output file.
- merge: Read multiple partial call graphs stored by 'collect', merge
them and store the gc-suspect information.
- check: Use the stored suspect information to check source files for
problems.

The modes 'collect' and 'check' support sharding parameters, so that
each shard runs on a deterministic fraction of the source files.

The previous way of running gcmole locally is unchanged as 'full'
run. All modes also support the --test-run parameter, running against
a test double.

The parameter to reuse suspects is removed as the new 'check' mode
makes it redundant.

This also adds python tests to verify most additions for merging
call graphs and for distributing source files to shards.

The new API is used on the infra side via the gcmole_v3 config, which
we configure now in parallel to the previous version to test for
equality.

Based on the builders.pyl change, these additions also run on the
v8_linux_rel trybot on this CL.

Bug: v8:12660
Change-Id: Ibe66292cb00830fa1c23451081a8de4cbc4766a1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4075907
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84674}
2022-12-06 07:08:43 +00:00
..
bazel [bazel] Add optional deps for prebuilt bazel for bots 2021-06-22 08:42:15 +00:00
builtins-pgo [builtins-pgo] Make builtins profiles architecture independent 2022-11-30 10:27:24 +00:00
cfi [cfi] Use inclusive terminology 2020-07-28 05:57:41 +00:00
clusterfuzz [heap] Enable --shared-string-table on more fuzzers again 2022-11-28 14:38:29 +00:00
cppgc [mips32] Delete mips32 from v8 2022-09-13 07:54:54 +00:00
debug_helper [v8windbg] Fix crash when reading external strings 2022-11-28 16:13:09 +00:00
dev [tools] Fix update-vscode.sh again 2022-12-05 15:22:42 +00:00
gcmole [gcmole] Add API for running on multiple hosts 2022-12-06 07:08:43 +00:00
generate_shim_headers [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
heap-layout [tools] Add placeholder comment for analytics code 2022-08-22 12:41:18 +00:00
heap-stats Reland "[intl] Implement DurationFormat" 2022-10-05 07:42:28 +00:00
js [tools] Fix parse-processor 2022-08-04 19:00:33 +00:00
jsfunfuzz
link_clicker.extension
mb [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
memory [tools] Add tools/memory/rss.py to sample RSS usage 2022-10-21 16:38:40 +00:00
msan Rename msan blacklist.txt to ignorelist.txt 2021-05-27 08:24:50 +00:00
perf
profiling [tools] linux-perf-d8.py: Add option to skip pprof 2022-10-19 13:13:57 +00:00
profview [tools] Profview fixes 2022-08-30 13:55:08 +00:00
release [tools] Fix tools/mergeinfo.py python3 issues 2022-10-11 13:09:33 +00:00
sanitizers [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
snapshot [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
system-analyzer [tools] Fix helper.mjs 2022-11-21 09:44:00 +00:00
testrunner [resultdb] Add framework_name tag to ResultDB 2022-12-05 19:29:27 +00:00
toolchain [mips32] Delete mips32 from v8 2022-09-13 07:54:54 +00:00
torque [tools] Clean up Py2 code 2022-03-17 09:05:27 +00:00
tracing/proto-converter [tools] Refactor, use built-in Array.flat() instead of custom flatten method 2021-05-31 07:46:15 +00:00
turbolizer Reland "[turbolizer] Show basic block id in schedule phase" 2022-12-05 13:39:11 +00:00
ubsan Reland "Update V8 DEPS (trusted)" 2022-11-30 14:46:31 +00:00
unittests [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
v8.xcodeproj
v8windbg [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
valgrind/asan
vim [vim] Update vim scripts 2021-04-01 16:14:04 +00:00
visual_studio
wasm [wasm] Always use the engine allocator for decoded modules 2022-12-02 12:22:06 +00:00
wasm-compilation-hints [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
zone-stats [CSA][codegen] Add BitcastElision Phase 2022-10-20 02:29:57 +00:00
__init__.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
adb-d8.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
android-build.sh
android-run.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
android-sync.sh [mjsunit][tools] Sync module files for tests 2020-10-07 12:45:19 +00:00
arguments.mjs [tools] Cleanup Arguments processing in tools 2021-06-01 09:43:46 +00:00
avg.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
bash-completion.sh [bash-completion] Support some gm.py completion 2021-03-24 16:24:25 +00:00
bigint-tester.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
BUILD.gn [gcmole] Optimize data dependencies 2022-12-05 14:03:39 +00:00
callstats_groups.py [web snapshot] web snapshot <3 RuntimeCallStats 2021-05-19 12:16:28 +00:00
callstats-from-telemetry.sh [tools] Fix callstats 2022-02-17 19:12:24 +00:00
callstats.html [tools] Skip over group entries in RCS input file 2022-09-12 15:40:41 +00:00
callstats.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
check-inline-includes.sh
check-static-initializers.sh Bump expected static initializers count from 2 to 3 2021-02-12 09:47:44 +00:00
check-unused-bailouts.sh
check-unused-symbols.sh
codemap.mjs [tools][system-analyzer] Add profiler-panel 2022-04-26 17:57:03 +00:00
collect_deprecation_stats.sh
compare_torque_output.py [v8] Migrate compare_torque_output to py3 2022-02-08 15:46:11 +00:00
compare-table-gen.js
consarray.mjs [tools] Migrate more tools to ES6 classes 2021-01-11 16:40:39 +00:00
cpu.sh
cross_build_gcc.sh
csvparser.mjs [tools][system-analyzer] Various improvements 2022-04-08 19:46:22 +00:00
DEPS
detect-builtins.js
disasm.py [mips32] Delete mips32 from v8 2022-09-13 07:54:54 +00:00
draw_instruction_graph.sh
dump-cpp.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
dumpcpp-driver.mjs [tools][system-analyzer] Add local symbol server 2021-07-01 09:59:17 +00:00
dumpcpp.mjs [tools][system-analyzer] Speed up log parsing 2022-04-06 11:31:03 +00:00
eval_gc_nvp.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
eval_gc_time.sh Remove all leftovers of store buffer 2021-11-10 09:29:41 +00:00
find_depot_tools.py
find-builtin [tools] Add a tool for finding builtins 2021-04-07 12:51:38 +00:00
find-commit-for-patch.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
freebsd-tick-processor
fuzz-harness.sh
gc_nvp_common.py
gc-nvp-to-csv.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
gc-nvp-trace-processor.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
gdb-v8-support.py [tools] Fix gdb redirect helper in gdb-v8-support.py 2021-10-21 12:27:45 +00:00
gdbinit [tools][gdbinit] Increase search depth for V8_Dcheck frame 2022-12-02 12:59:48 +00:00
gen-inlining-tests.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
gen-keywords-gen-h.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
gen-postmortem-metadata.py [build] Make build deterministic 2022-11-30 11:37:43 +00:00
gen-v8-gn.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
generate-builtins-tests.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
generate-header-include-checks.py [mips32] Delete mips32 from v8 2022-09-13 07:54:54 +00:00
generate-runtime-call-stats.py [tools] Fix retention of json files in RCS script 2020-03-26 12:07:56 +00:00
generate-ten-powers.scm
get_landmines.py [py3] Bump get_landmines to py3 2022-04-04 11:03:36 +00:00
grokdump.py [tools] grokdump: Fix objdump detection for Python3 2022-06-23 14:28:54 +00:00
ic-processor [tools] Convert JS-tools to ES6 modules 2020-09-28 14:40:18 +00:00
ic-processor-driver.mjs [tools][system-analyzer] Add local symbol server 2021-07-01 09:59:17 +00:00
index.html [tools] Fix index.html links 2022-08-29 14:14:56 +00:00
linux-tick-processor [tools] Port more tools to ES6 classes 2020-11-04 14:45:41 +00:00
lldb_commands.py [tools] Set up source map for lldb 2021-07-01 18:04:17 +00:00
locs.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
logreader.mjs [tools][system-analyzer] Add profiler-panel 2022-04-26 17:57:03 +00:00
mac-nm [tools] Profview fixes 2022-08-30 13:55:08 +00:00
mac-tick-processor
Makefile.tags
objdump-v8 [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
OWNERS [owners] Includes infra team as tools owners 2021-11-25 07:55:08 +00:00
package-lock.json [tools] Add README.md file and package.json for local development 2021-07-05 16:43:28 +00:00
package.json [tools] Add README.md file and package.json for local development 2021-07-05 16:43:28 +00:00
parse-processor [tools] Convert JS-tools to ES6 modules 2020-09-28 14:40:18 +00:00
parse-processor-driver.mjs [tools][system-analyzer] Add local symbol server 2021-07-01 09:59:17 +00:00
parse-processor.html [tools] Add placeholder comment for analytics code 2022-08-22 12:41:18 +00:00
parse-processor.mjs [tools] Fix parse-processor 2022-08-04 19:00:33 +00:00
perf-compare.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
predictable_wrapper.py [tools] Clean up predictable wrapper py2 code 2022-07-18 07:38:42 +00:00
PRESUBMIT.py Reland "[tools] Clean up py2 code" 2022-03-25 12:14:26 +00:00
process-wasm-compilation-times.py [wasm] Tweak constants for estimating code space size 2022-03-15 14:41:13 +00:00
profile_view.js
profile_view.mjs [tools] Modernize tools .mjs files 2020-11-10 12:32:26 +00:00
profile.mjs [tools][profile] Add support for maglev optimisation markers 2022-07-27 08:48:25 +00:00
README.md [tools] Add README.md file and package.json for local development 2021-07-05 16:43:28 +00:00
regexp-sequences.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
run_perf.py [mips32] Delete mips32 from v8 2022-09-13 07:54:54 +00:00
run-clang-tidy.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
run-num-fuzzer.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
run-tests.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
run-wasm-api-tests.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
run.py [tools] Switch script wrapper to Py3 2022-04-04 14:13:17 +00:00
shell-utils.h
sourcemap.mjs [tools][system-analyzer] Add profiler-panel 2022-04-26 17:57:03 +00:00
splaytree.mjs [tools] Improve system analyzer 2022-03-02 13:41:32 +00:00
stats-viewer.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
test262-results-parser.js
tickprocessor-driver.mjs [tools][system-analyzer] Add local symbol server 2021-07-01 09:59:17 +00:00
tickprocessor.mjs [tools][profile] Add support for maglev optimisation markers 2022-07-27 08:48:25 +00:00
try_perf.py [infra] Change all Python shebangs to Python3 2022-08-05 14:55:00 +00:00
turbolizer-perf.py
update-object-macros-undef.py
v8_presubmit.py [gcmole] Simplify gcmole directives and add tests 2022-11-14 13:07:15 +00:00
v8heapconst.py [runtime] Refactor GetOwnPropertyDescriptor to use dispatching stub 2022-12-01 16:38:43 +00:00
whitespace.txt Whitespace change to test CQ 2022-11-14 08:23:18 +00:00
windbg.js
windows-tick-processor.bat [tools] Use .mjs in windows-tick-processor.bat 2021-01-07 09:00:23 +00:00
wpr.wprp Step 1 (of 3-ish): Basic ETW Instrumentation in V8 2021-01-05 20:11:03 +00:00

TOOLS

This directory contains debugging and investigation tools for V8.

The contents are regularly mirrored to http://v8.dev/tools.

Local Development

For local development you have to start a local webserver under http://localhost:8000:

  cd tools/;
  npm install;
  ws;

Local Symbol Server

The system-analyzer can symbolize profiles for local binaries by running a local symbol server

  cd tools/;
  ws --stack system-analyzer/lws-middleware.js lws-static cors;

Note that the local symbol server will run nm and objdump and has access to your files.