Commit Graph

136 Commits

Author SHA1 Message Date
Olivier Flückiger
bd7458cb7f Reland "[static-roots] Allow creating isolates from exist. r/o snapshot"
This is a reland of commit 331c577e1d

Fix DCHECK for single generation builds.

Original change's description:
> [static-roots] Allow creating isolates from existing r/o snapshot
>
> This adds the ability to create an isolate from scratch, except the
> read only roots, which are initialized from a read_only_data snapshot.
> To do this we split the heap setup in a read/only part and the rest.
> The goal of these changes is to later support writing serializer tests,
> even if the read only roots are static and have to be loaded from a
> fixed snapshot.
>
> Bug: v8:13466
> Change-Id: I078695b95710e5281da013ca0c08af0e153b4725
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4037271
> Reviewed-by: Igor Sheludko <ishell@chromium.org>
> Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
> Commit-Queue: Olivier Flückiger <olivf@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#84750}

Bug: v8:13466
Change-Id: Ic680b7620fc3176260c5d550e474412afef5f598
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4092575
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Commit-Queue: Olivier Flückiger <olivf@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84773}
2022-12-12 10:44:23 +00:00
Dominik Inführ
458cda96fe Revert "[heap] Iterate promoted pages during sweeping"
This reverts commit 1e3dd39d09.

Reason for revert: Causes failures with --shared-string-table

https://bugs.chromium.org/p/chromium/issues/detail?id=1399489
https://bugs.chromium.org/p/chromium/issues/detail?id=1399491
https://bugs.chromium.org/p/chromium/issues/detail?id=1399488
https://bugs.chromium.org/p/v8/issues/detail?id=13574

Original change's description:
> [heap] Iterate promoted pages during sweeping
>
> Promoted pages are iterated to record slots containing old to new and
> old to shared references. This takes a significant amount of time during
> the atomic pause.
> Instead we offload this task to the concurrent sweepers, record slots to
> a local cache, and merge it when finalizing sweeping.
>
> Array buffer sweeping depends on iteration of promoted pages, so it is
> frozen until iteration is done.
>
> See design doc at https://docs.google.com/document/d/1JzXZHguAnNAZUfS7kLeaPVXFfCYbf5bGCtyKgyiMDH4/edit?usp=sharing
>
> Bug: v8:12612
> Change-Id: Icdc79a7a70c53352e3a1b3961cfe369e8563b65b
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4062041
> Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
> Auto-Submit: Omer Katz <omerkatz@chromium.org>
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#84706}

Bug: v8:12612
Change-Id: I4ed4a6ad954cb294b569c94b4d8ea25a0956a223
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4092734
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84760}
2022-12-09 14:53:05 +00:00
Nico Hartmann
1d733b97a2 Revert "[static-roots] Allow creating isolates from existing r/o snapshot"
This reverts commit 331c577e1d.

Reason for revert: https://ci.chromium.org/ui/p/v8/builders/ci/V8%20Linux64%20-%20debug%20-%20single%20generation%20-%20builder/9912/overview

Original change's description:
> [static-roots] Allow creating isolates from existing r/o snapshot
>
> This adds the ability to create an isolate from scratch, except the
> read only roots, which are initialized from a read_only_data snapshot.
> To do this we split the heap setup in a read/only part and the rest.
> The goal of these changes is to later support writing serializer tests,
> even if the read only roots are static and have to be loaded from a
> fixed snapshot.
>
> Bug: v8:13466
> Change-Id: I078695b95710e5281da013ca0c08af0e153b4725
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4037271
> Reviewed-by: Igor Sheludko <ishell@chromium.org>
> Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
> Commit-Queue: Olivier Flückiger <olivf@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#84750}

Bug: v8:13466
Change-Id: I91512140abb98993578d27e65e6080e1d3f317e6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4089967
Auto-Submit: Nico Hartmann <nicohartmann@chromium.org>
Commit-Queue: Nico Hartmann <nicohartmann@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Nico Hartmann <nicohartmann@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84751}
2022-12-09 12:06:43 +00:00
Olivier Flückiger
331c577e1d [static-roots] Allow creating isolates from existing r/o snapshot
This adds the ability to create an isolate from scratch, except the
read only roots, which are initialized from a read_only_data snapshot.
To do this we split the heap setup in a read/only part and the rest.
The goal of these changes is to later support writing serializer tests,
even if the read only roots are static and have to be loaded from a
fixed snapshot.

Bug: v8:13466
Change-Id: I078695b95710e5281da013ca0c08af0e153b4725
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4037271
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Olivier Flückiger <olivf@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84750}
2022-12-09 11:43:52 +00:00
Omer Katz
1e3dd39d09 [heap] Iterate promoted pages during sweeping
Promoted pages are iterated to record slots containing old to new and
old to shared references. This takes a significant amount of time during
the atomic pause.
Instead we offload this task to the concurrent sweepers, record slots to
a local cache, and merge it when finalizing sweeping.

Array buffer sweeping depends on iteration of promoted pages, so it is
frozen until iteration is done.

See design doc at https://docs.google.com/document/d/1JzXZHguAnNAZUfS7kLeaPVXFfCYbf5bGCtyKgyiMDH4/edit?usp=sharing

Bug: v8:12612
Change-Id: Icdc79a7a70c53352e3a1b3961cfe369e8563b65b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4062041
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Auto-Submit: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84706}
2022-12-07 14:33:34 +00:00
Michael Achenbach
2d51120a9d [gcmole] Add API for running on multiple hosts
This adds new modes to run gcmole on multiple hosts:
- collect: Parse source files and store partial raw call graph in
a binary output file.
- merge: Read multiple partial call graphs stored by 'collect', merge
them and store the gc-suspect information.
- check: Use the stored suspect information to check source files for
problems.

The modes 'collect' and 'check' support sharding parameters, so that
each shard runs on a deterministic fraction of the source files.

The previous way of running gcmole locally is unchanged as 'full'
run. All modes also support the --test-run parameter, running against
a test double.

The parameter to reuse suspects is removed as the new 'check' mode
makes it redundant.

This also adds python tests to verify most additions for merging
call graphs and for distributing source files to shards.

The new API is used on the infra side via the gcmole_v3 config, which
we configure now in parallel to the previous version to test for
equality.

Based on the builders.pyl change, these additions also run on the
v8_linux_rel trybot on this CL.

Bug: v8:12660
Change-Id: Ibe66292cb00830fa1c23451081a8de4cbc4766a1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4075907
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84674}
2022-12-06 07:08:43 +00:00
Michael Achenbach
7136ea89d2 [gcmole] Fix and simplify test-run mode
The test-run mode was broken after output improvements and the
introduction of pathlib.

This fixes the string concatenation with paths and updates the test
output to match the status quo. This also changes the test-run mode
to run exclusively when the --test-run option is passed. Now it's
either a test run or a normal run. Like that we can add the test run
as a separate test step on a bot. If both are needed in sequence
for something, gcmole could be called twice.

Bug: v8:12660
Change-Id: I58179d50950fa76d8f66b974325a8fed84dc91b6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4075727
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84655}
2022-12-05 15:21:19 +00:00
Michael Achenbach
bc15e20a03 [gcmole] Optimize data dependencies
This (micro)optimizes data dependencies of gcmole in two ways:

1. Only bundle icu folders 'common' and 'i18n', omitting particularly
icu's data and test folder, which aren't needed for running gcmole.
This reduces gcmole bundle size from 377MB to 239MB, reducing upload
and download times by a few seconds on g1 bots.

2. Process gcmole data dependencies during GN time only when gcmole
is configured via gn flag. Currently, the dependency files are also
processed on all other bots that aren't running gcmole.

Bug: v8:12660
Change-Id: Ib708fa2957e6e33698e51b2aee45929f4d467935
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4076331
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84648}
2022-12-05 14:03:39 +00:00
Michael Achenbach
1211605a39 [build] Remove dead ENABLE_DEBUGGER_SUPPORT define
Bug: v8:9287
Change-Id: Ia031aa653b78bae1817e05c75ea508a7e2df8a83
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4055628
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84479}
2022-11-25 09:54:03 +00:00
Michael Achenbach
c9bdd23fb0 [gcmole] Separate call graph from suspects collector
This factors out a call graph from the suspects collector which in
a future CL can be serialized as a partial call graph and later
merged.

Bug: v8:12660
Change-Id: Ie6f682195a900ba0711b8f828c63bf41f142f2b1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4035131
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84403}
2022-11-21 16:32:27 +00:00
Michael Achenbach
3afea88feb [gcmole] Make wrapper backwards-compatible to gcmole modes
In a follow up we want to introduce a mode for just collecting or
processing. In the infrastructure however, we still call the wrapper
script with old arguments on release branches. This ensures that
we stay backwards compatible.

Bug: v8:12660
Change-Id: I08da713719d10cb58c2cd5ea7f711570bd57572c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4035976
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84380}
2022-11-21 09:45:04 +00:00
Michael Achenbach
8c37a1a8bc [gcmole] Add tests for suspect collector
This adds tests of the GCSuspectsCollector and its relevant usages
in GCMole. The relevant logic has now 100% code coverage, which
prepares refactoring of the code and serialization of only partially
collected suspect data in a future CL .

No-Try: true
Bug: v8:12660
Change-Id: I0b0cc6f8a5e65f14084e80cda50400232e0d2c45
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4031052
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84361}
2022-11-18 14:31:28 +00:00
Michael Achenbach
506fb93ebe [gcmole] Add subparsers
This will enable running the main gcmole script in different modes
in the future, full run (this CL), later suspect generation only and
analysis only.

Bug: v8:12660
Change-Id: Ica87366fb3c5b6e238f5a1efaf347ebbeb01003a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4026062
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84357}
2022-11-18 13:17:47 +00:00
Michael Achenbach
31c9da6dad [gcmole] Migrate from optparse to argparse
Use argparse to make using subparsers possible in the future. Also
remove some dead code.

Bug: v8:12660
Change-Id: I26fbef0c8cc48ada6ca9f1072fbc3784893e5be9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4023079
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84349}
2022-11-18 09:03:28 +00:00
Michael Achenbach
2b133ebc9e [gcmole] Simplify gcmole directives and add tests
This prepares work for speeding up gcmole infrastructure by adding
more checks and testing.

- Add a presubmit check that verifies gcmole directives in gn files.
- Simplify gcmole directives in gn files. Drop all but the
  architecture property as the others were unused. Simplify regular
  expressions.
- Fix some gcmole directives (fly by - only on untested
  architectures).
- Add a python test for the changed function in gcmole.py. There's
  no other testing yet and this starts somewhere. The test simulates
  real BUILD.gn files via checked-in test data.

Bug: v8:12660
Change-Id: I9ca3152b2c3b806de5d4509c152283b36294950d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/4020268
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Alexander Schulze <alexschulze@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#84239}
2022-11-14 13:07:15 +00:00
Camillo Bruni
1920426b13 [tools][gcmole] Add run_gcmole.py --help and drop env vars
- Use explicit options when running gmcole.py from run-gcmole.py
- Use gcmole.py-relative paths to find the default V8 root dir for
  maximum convenience when running locally

Change-Id: Iba0da90b99b0321129f1c4099f437c76dabb1186
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3582386
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Auto-Submit: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80058}
2022-04-20 15:26:35 +00:00
Alexander Schulze
ad7248787f [py3] Bump gcmole to py3
Bug: v8:12581
Change-Id: I6c665161a3f376321feb773d809c5453ef1d46f9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3568445
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Alexander Schulze <alexschulze@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79735}
2022-04-04 11:48:13 +00:00
Camillo Bruni
cff529a2cd [tools][gcmole] Rename suspects.whitelist to suspects.allowlist
Bug: v8:10009
Change-Id: I1f71b2783cd07331a65f46c316e87a46396e7eda
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3516153
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79541}
2022-03-21 10:24:39 +00:00
v8-ci-autoroll-builder
8231c651d5 [tools] Update gcmole
R=clemensb@chromium.org

Change-Id: I1d3513a75c32a505cb5cdb1a123f2d586f55ee02
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3513416
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79430}
2022-03-09 20:16:54 +00:00
Camillo Bruni
ecc3cd256a [tools] Improve gcmole part II
Prepare gcmole.cc for the next update:
- Print possible GC locations when discovering stale/dead variables
- Make error messages less confusing for the modern V8 engineer
- Prepare gcmole to read suspects.allowlist instead of .whitelist
- Use more readable variable names
- Only log non-found types with --verbose
- Change the currently unusued gccauses format in gcmole.py and
  support loading it back in gcmole.cc
- Implemented first basic gc call-chain printing (disabled by default)

GCmole packaging:
- Add debug mode to bootstrap.sh build script
- Update gcmole.py run instructions in bootstrap.sh and package.sh

Bug: v8:10009
Change-Id: I369d48baa2980455d2e8f57e7a803d0384fe83f1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3480095
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79357}
2022-03-04 01:06:11 +00:00
Camillo Bruni
4db43bfa2f [tools] Improve gcmole part I: command line
- Convert gcmole to python3-ish code
  - use local Path implementation for future full migration
- Use optparse and explicit arguments for gcmole
  - Add explicit directories flags
  - Use backwards compatible env vars as fallbacks
- Add gn target v8_gcmole_files to avoid issues with missing or
  incompatible generated files

Drive-by-fixes for running gcmole without ignored files:
- Disable gcmole in Isolate::UnwindAndFindHandle
- Partially disable gcmole in V8HeapExplorer::AddEntry

Bug: v8:10009
Change-Id: I5b4d1c6554db300190226361b6c518419109ff3d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3497320
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79348}
2022-03-03 15:01:15 +00:00
v8-ci-autoroll-builder
4d62c005f8 [tools] Update gcmole
R=machenbach@chromium.org

Change-Id: I9866742528d47210d39fcdeda522dcf29bdba38e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3486679
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79249}
2022-02-24 10:30:13 +00:00
Camillo Bruni
4130834484 [tools] Make gcmole less verbose by default
Don't print command line invocations by default.

Change-Id: Ie67c883c92557a645fc8a681722f49a2c7c4bd17
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3468901
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79130}
2022-02-16 16:41:33 +00:00
v8-ci-autoroll-builder
c2cdcee03f [tools] Update gcmole
R=leszeks@chromium.org

Change-Id: Iffe5593f3e6e5766f057ad872d16758527b252a8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3248211
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77586}
2021-10-28 07:53:44 +00:00
v8-ci-autoroll-builder
b824e88470 [tools] Update gcmole
R=mslekova@chromium.org

Change-Id: I47b9a286cc0f1b9dd64bf564423b4fb4af4a19ce
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3244954
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77545}
2021-10-26 11:43:52 +00:00
Leszek Swirski
9af08ccb86 [gcmole] Make packaging more deterministic
Tar and Gzip would include timestamps for files in the gcmole package,
which meant that they weren't deterministic across multiple builds. Now
we strip out the timestamps (as well as owner/group information) to make
the packaging more deterministic.

Change-Id: Iec3195ede4150c1603927fec748455329a8da677
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3245112
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77539}
2021-10-26 09:12:35 +00:00
v8-ci-autoroll-builder
293f2826e9 [tools] Update gcmole
R=clemensb@chromium.org

Change-Id: Ia804d29a61e9d716b5ee5acebce7cb9c273697d7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3241363
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77522}
2021-10-25 12:52:21 +00:00
Liviu Rau
46936947d8 Add owners file to gcmole folder
Narrow down the list of owners to choose from in the gcmole auto-update CLs.

Bug: v8:11169
Change-Id: I9f553124c43dc9fc094a8f4a6e2bcc832fe27636
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3236993
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77509}
2021-10-23 07:03:02 +00:00
Michael Lippautz
d67f060672 Revert "[base] Replace base::Optional with Abseil"
This reverts commit f2e25f8a09.

Reason for revert: Breaks GCC 8.4.0 compilation, see bug.

Original change's description:
> [base] Replace base::Optional with Abseil
>
> Change base::Optional to an alias of absl::optional. Eventually we
> should remove it entirely.
>
> Bug: v8:11006
> Change-Id: I687d44cc7e7cd0a49a84bcc207231eb6808eef2d
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2476318
> Commit-Queue: Leszek Swirski <leszeks@chromium.org>
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#76913}

Bug: v8:11006, v8:12252
Change-Id: I8048dee676a36551d15f25a31d1e171dae00d975
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3173672
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Michael Lippautz <mlippautz@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#76976}
2021-09-21 19:56:40 +00:00
Leszek Swirski
f2e25f8a09 [base] Replace base::Optional with Abseil
Change base::Optional to an alias of absl::optional. Eventually we
should remove it entirely.

Bug: v8:11006
Change-Id: I687d44cc7e7cd0a49a84bcc207231eb6808eef2d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2476318
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#76913}
2021-09-17 15:46:54 +00:00
Liviu Rau
b844d0f4b7 Build clang for gcmole bootstrap
Bug: v8:11169
Change-Id: I32e9bb1f91c9583e900c96e158d66407d5560868
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3003466
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Liviu Rau <liviurau@chromium.org>
Cr-Commit-Position: refs/heads/master@{#75555}
2021-07-05 12:39:37 +00:00
Clemens Backes
8890bb21f9 [no-wasm] Remove wasm debugging support
This removes all wasm includes from src/debug and src/inspector if
webassembly is disabled (v8_enable_webassembly=false). It also removes
the definition of {WasmValueObject} and {v8::debug::WasmScript}.
This will allow to later fully exclude the src/wasm directory from
compilation (once other components are fixed).

R=bmeurer@chromium.org, machenbach@chromium.org

Bug: v8:11238
Change-Id: I41a1d83d01fbb6c015cdfd6cc063bad90052505d
Cq-Include-Trybots: luci.v8.try:v8_linux64_no_wasm_compile_rel
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2726506
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73138}
2021-03-02 18:27:53 +00:00
Dominik Inführ
17cba4f0f0 [heap] Safepointing with an atomic state
To improve performance of parking, keep the thread state in an atomic
variable instead of protecting it with a mutex.

However the mutex was used e.g. to force Unpark() to block while the
safepoint operation was still running. Therefore the safepoint algorithm
has to change as well.

Park() and Unpark() use CAS operation to transition the state.
Safepoint() uses a relaxed load for checking whether a safepoint was
requested. Since Safepoint(), Park() and Unpark() all have a slow path,
there is no need for busy-waiting on the main thread.

We need two more ThreadStates:
* SafepointRequested: This state is set by GlobalSafepoint to force
    Running threads into the slow path on Safepoint() and Park(). This
    state also replaces the separate atomic<bool> safepoint_requested_
    field we used before.
* ParkedSafepoint: This state is set by GlobalSafepoint as well to force
    parked threads into the slow path on Unpark().

When stopping all threads, GlobalSafepoint transitions states from
Running --> SafepointRequested and Parked --> ParkedSafepoint to force
the slow path for all three methods. After performing the transition
for each thread we know the exact number of Running threads and wait
until each of them either reached a safepoint or parked itself.

Design doc: https://docs.google.com/document/d/1p9klWyqT_AScAnK_PdHZTcNhZGzoBiYWPkUciIh2C58/edit?usp=sharing

Bug: chromium:1177144, v8:10315
Change-Id: I8697da915c7d18e2fb941f1bedf6181226408feb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2704075
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73089}
2021-03-01 08:41:12 +00:00
Leszek Swirski
b84baffbe7 [tools] Fix whitespace
Fix some trailing whitespace issues in files in the tools directory.

Change-Id: If9e9b1dab1d6f521e20619a2a1d093749f0528d9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2671660
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#72498}
2021-02-03 15:42:11 +00:00
Camillo Bruni
d16a2a6884 [tools] Add DisableGCMole scope
Make sure gcmole detects issue in DisallowGarbageCollection scopes.

DisallowGarbageCollection is widely used in the codebase to document
code that doesn't allocate. However, this has the rather unexpected
side-effect that gcmole is not run when such a scope is active.

This CL changes the default behavior of gcmole to run even with
DisallowGarbageCollection scopes present. This will give us the best
results of both worlds, dynamic checks by the fuzzer, and static
analysis by gcmole.

To allow crazy local raw pointer operations there is a new
DisableGCMole scope that explicitly disables gcmole.

Change-Id: I0a78fb3b4ceaad35be9bcf7293d917a41f90c91f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2615419
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#72039}
2021-01-12 12:59:39 +00:00
Leszek Swirski
18138f2630 [gcmole] Improve performance
Improve the performance of gcmole by

  * Precompiling the regexes in GCSuspectsCollector.Resolve
  * Merging those regexes into a single regex, using '|'
  * Changing multiprocess clang plugin invocation to threaded (running
    the plugin releases the GIL so this can efficiently thread). This
    uses a simple worker pool with a single work queue.
  * Change clang plugin invocation loop to yield after each invocation.
    This pipelines the dump-callees plugin and GCSuspectsCollector
    Parse/Resolve, so that the parse can happen while waiting for other
    callee dumps to finish.

Change-Id: Ib9fca70dbcfd2f9d1aebc8bd11aa1d1f7d34e24a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2562242
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71617}
2020-12-04 16:26:51 +00:00
Leszek Swirski
f0df9b606a [gcmole] Python conversion followup
Fix the python call to sys.executable, and sort imports

Change-Id: I948291e0835c79445108f215e55885fdb2c01052
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2560592
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71423}
2020-11-26 09:40:45 +00:00
Leszek Swirski
f47e59e045 [asserts] Make assert scopes LocalHeap friendly
Because of LocalHeap safepoints, our existing assert scopes don't
necessarily maintain the same guarantees as desired. In particular,
DisallowHeapAllocation no longer guarantees that objects don't move.

This patch transitions DisallowHeapAllocation to
DisallowGarbageCollection, to ensure that code using this scope is
also protected against safepoints.

Change-Id: I0411425884f6849982611205fb17bb072881c722
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2540547
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71319}
2020-11-20 17:39:14 +00:00
Leszek Swirski
18cbf05e8e [gcmole] Re-write gcmole runner in Python
This removes the dependency on Lua for running gcmole, and Python's
expressiveness lets us clean up the code a bit, including getting rid
of global variables, embedding the multiprocessing (removing the need
for a separate `parallel.py`), and using difflib for printing the test
expectation diff.

Bug: v8:11169, v8:8590, chromium:1097212
Change-Id: If0ab5ea6f764864855d73cd0ba63cb37c1823955
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2543927
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Liviu Rau <liviurau@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71253}
2020-11-18 11:19:39 +00:00
Leszek Swirski
076687ab85 [asserts] Add combination assert scopes
Add a "combination" assert scope class, which combines multiple existing
assert scopes.  This will allow scopes with functional overlap, e.g.
DisallowGarbageCollection and DisallowHeapAllocation, to share an assert
type rather than rather than requiring users to remember to set both. To
demonstrate this, this redefines DisallowGarbageCollection to a
combination of DisallowHeapAllocation and a new DisallowSafepoints, and
some of the DCHECKs checking both are simplified to only check one or
the other, as appropriate.

The combination classes become subclasses of the existing assert scopes,
so that they can be used in their place as e.g. a function parameter,
e.g. DisallowGarbageCollection can be passed to a function expecting
const DisallowHeapAllocation&.

As a drive-by, this also changes the per-thread assert scopes to use a
bitmask, rather than a bool array, to store their per-thread data. The
per-isolate scopes already used a bitmask, so this unifies the
behaviour between the two.

Change-Id: I209e0a56f45e124c0ccadbd9fb77f39e070612fe
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2534814
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71231}
2020-11-17 13:27:46 +00:00
Leszek Swirski
7645037c67 [gcmole] Update to LLVM 9.0.1, bootstrap from git
This resolves issues building gcmole under LLVM 8.0.0 with gcc 10 (see
https://bugs.gentoo.org/708730).

Change-Id: I932def25fb3cc841cf21414a513fbe2a2fa8a8dc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2539946
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#71204}
2020-11-16 13:05:40 +00:00
Jakob Gruber
c20ff735ba Revert "[DEPS] Add abseil to deps"
This reverts commit 3c49308ac6.

Reason for revert: https://ci.chromium.org/p/v8/builders/ci/V8%20Clusterfuzz%20Mac64%20ASAN%20-%20debug%20builder/18360

Original change's description:
> [DEPS] Add abseil to deps
>
> Add a dependency on the chromium abseil-cpp subdir mirror.
>
> Bug: v8:11006
> Change-Id: Icaad757269d27c65bc368ed539f84c5bb79ee62d
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464940
> Commit-Queue: Leszek Swirski <leszeks@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Reviewed-by: Victor Gomes <victorgomes@chromium.org>
> Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#70786}

TBR=rmcilroy@chromium.org,adamk@chromium.org,yangguo@chromium.org,hpayer@chromium.org,leszeks@chromium.org,victorgomes@chromium.org

Change-Id: Iff2ac3b0da8725ec2df69aa527e5a4255ca3009c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:11006
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2501843
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70787}
2020-10-27 10:48:05 +00:00
Leszek Swirski
3c49308ac6 [DEPS] Add abseil to deps
Add a dependency on the chromium abseil-cpp subdir mirror.

Bug: v8:11006
Change-Id: Icaad757269d27c65bc368ed539f84c5bb79ee62d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2464940
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70786}
2020-10-27 09:34:52 +00:00
Maya Lekova
87bc38e390 [gcmole] Fix false negatives with GC guards
GCMole mistakenly thought that GC guards such as DisallowHeapAllocation
covered the whole scope of the function they are declared in. This CL
fixes the false negatives and adds appropriate testing.

Bug: v8:10071
Change-Id: Iffb369977af90ca053a55ca8f451e037a4f460f2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2497451
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70763}
2020-10-26 17:02:00 +00:00
Michael Achenbach
c87bdbcf0d [gcmole] Fix gcmole after property change
The build location is now universally in out/build after infra change:
https://crrev.com/c/2426643

TBR=mslekova@chromium.org

Bug: chromium:1132088
Change-Id: I0d8867ed58adec79ed66f5e4dac375827e2679e3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2448451
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70296}
2020-10-04 15:20:18 +00:00
Tobias Tebbi
21b585165f Reland "[torque] refactor: use -tq only in filenames derived from .tq files"
This is a reland of 64caf2b0b2

Original change's description:
> [torque] refactor: use -tq only in filenames derived from .tq files
>
> This is to establish a naming rule for Torque-generated files:
> - If the file is called foo/bar-tq..., then it is derived from a
>   file foo/bar.tq
> - Otherwise it doesn't belong to a specific .tq file.
>
> So far, we attached -tq to all Torque-generated file names, where it
> sometimes corresponded to a .tq file name and sometimes not.
> It is not necessary to add -tq to file names to indicate that they are
> Torque-generated, since they are already in a directory called
> torque-generated, and we always refer to them as
> "torque-generated/filename", so there is no confusion even though some
> files now have the same name as a corresponding hand-written file, for
> example factory.cc.
>
> TBR: hpayer@chromium.org
> Bug: v8:7793
> Change-Id: Ie172babad1fc7422fd1059c48f5dafaa53e50c8b
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2414218
> Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
> Reviewed-by: Jakob Gruber <jgruber@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#70060}

Bug: v8:7793
TBR: hpayer@chromium.org jgruber@chromium.org
Change-Id: I6c492bc64aee1ff167e7ef401825eca9097a7f38
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2431565
Commit-Queue: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Cr-Commit-Position: refs/heads/master@{#70137}
2020-09-25 15:04:18 +00:00
Maya Lekova
d626121e70 [gcmole] Upload a new version supporting DisallowGarbageCollection
This is an attempt to fix a gcmole failure in
https://chromium-review.googlesource.com/c/v8/v8/+/2362960

Change-Id: Idff72ff3ae63863fb73f11231481469c700891c3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2410193
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69907}
2020-09-15 11:18:22 +00:00
Dominik Inführ
dfcd257910 [gcmole] Add support for DisallowGarbageCollection
Add support for DisallowGarbageCollection scope. This scope will be
introduced once this CL landed.

DisallowGarbageCollection works like DisallowHeapAllocation but also
disallows safepoints.

Bug: v8:10315
Change-Id: Ia7d777d4104b32810dd481ad9dbdf0edd075b6cf
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2395561
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69881}
2020-09-14 13:52:46 +00:00
Maya Lekova
e7606e6b69 [gcmole] Enable use-after-free detection
GCMole now comes with the long forgotten use-after-free detection
enabled by default. The CL also improves error logging when test
expectations mismatch with the actual output and updates the hash
of GCMole to be used with the newly built version with enabled UAF
detection.

The CL also contains an ignore for isolate.cc due to inability to
fix a warning there and fixes a couple of UAF warnings.

Bug: v8:9680
Change-Id: I7a009ffd5f67b1b5437567691ca4235ea873de70
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2257236
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68505}
2020-06-24 09:29:31 +00:00
Maya Lekova
c74e9596ec [gcmole] Add heap-snapshot-generator.cc to ignored_files
Bug: v8:9993
Change-Id: I06428edd7f01805d08927718e4711298c731d2e3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2253844
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68450}
2020-06-22 08:40:29 +00:00