This is a reland of commit 24e60017d4
The reland changes %ClearFunctionFeedback to clear *all* feedback
slot kinds including binary/compare/for-in slots. In the tests we
thus no longer have to resort to tricks to restore the function to
it's initial state, instead simply call %ClearFunctionFeedback.
Original change's description:
> [maglev] Deopt on overflow in >>>
>
> Re-enable the int32 fast path for ShiftRightLogical, but account for
> Maglev's missing signed/unsigned representation tracking by a)
> removing rhs==0 as the identity value (a shift by 0 is still a
> signed-unsigned conversion) and b) deoptimizing if the result cannot
> be converted to a non-negative smi.
>
> Note this is not a deopt loop, since a non-smi result will change the
> feedback to kSignedSmallInputs (from kSignedSmall).
>
> To fix this properly, we should track signed/unsigned representations
> and convert the result to a heap number if it doesn't fit within smi
> range.
>
> Bug: v8:7700
> Change-Id: Ifd538d227a6f1290eb7f008d9bfad586ff91ea0f
> Fixed: v8:13251
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3876366
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Commit-Queue: Jakob Linke <jgruber@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#83025}
Bug: v8:7700
Change-Id: I2f607a0fb863b80e8589c9c1e86ee31fbac48c25
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3879491
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Commit-Queue: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83057}
This reverts commit 24e60017d4.
Reason for revert: Change breaks maglev GC stress tests.
Original change's description:
> [maglev] Deopt on overflow in >>>
>
> Re-enable the int32 fast path for ShiftRightLogical, but account for
> Maglev's missing signed/unsigned representation tracking by a)
> removing rhs==0 as the identity value (a shift by 0 is still a
> signed-unsigned conversion) and b) deoptimizing if the result cannot
> be converted to a non-negative smi.
>
> Note this is not a deopt loop, since a non-smi result will change the
> feedback to kSignedSmallInputs (from kSignedSmall).
>
> To fix this properly, we should track signed/unsigned representations
> and convert the result to a heap number if it doesn't fit within smi
> range.
>
> Bug: v8:7700
> Change-Id: Ifd538d227a6f1290eb7f008d9bfad586ff91ea0f
> Fixed: v8:13251
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3876366
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Commit-Queue: Jakob Linke <jgruber@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#83025}
Bug: v8:7700
Change-Id: Ibd4c29cc57b999e1cfcd040cee3228ea27a74029
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3876374
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Matthias Liedtke <mliedtke@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83029}
Re-enable the int32 fast path for ShiftRightLogical, but account for
Maglev's missing signed/unsigned representation tracking by a)
removing rhs==0 as the identity value (a shift by 0 is still a
signed-unsigned conversion) and b) deoptimizing if the result cannot
be converted to a non-negative smi.
Note this is not a deopt loop, since a non-smi result will change the
feedback to kSignedSmallInputs (from kSignedSmall).
To fix this properly, we should track signed/unsigned representations
and convert the result to a heap number if it doesn't fit within smi
range.
Bug: v8:7700
Change-Id: Ifd538d227a6f1290eb7f008d9bfad586ff91ea0f
Fixed: v8:13251
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3876366
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#83025}
Inline DependOnStablePrototypeChain to iterate only those maps which
share a validity cell with the receiver map. This resolves an issue
where maps after the holder object violate the stability invariants, but
doesn't require looking up what the actual holder is.
Bug: v8:7700
Change-Id: Id06f0d13660f547e14dd25085799c0e6223c34b9
Fixed: chromium:1359215
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3871298
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82964}
At the start of the graph builder, we add merge states to exception
handlers basic block with ExceptionPhis (normal phis with no input,
but with an interpreter register "owner").
Every Node that can throw, can also lazy deopt, so we use the
lazy deopt IFS to recover the exception phi values in a trampoline
before jumping to the exception catch block.
Bug: v8:7700
Change-Id: I62fe7f19ce5e89c3df645224ea62f9fc2798207c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3865865
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82935}
We've previously reset profiler ticks on ML finalization to avoid
unexpected early tierups to TF. This CL additionally avoid early TF
tierup through the `small function` reason by disabling small-function
optimization.
Bug: v8:7700
Change-Id: I57ba294af0d1d189f76c2cb1ffc31af0837b1e42
Fixed: v8:13242
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3865550
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82886}
.. since ticks collected in lower tiers use a different budget than
ticks collected in ML code. Keeping ticks from a lower tier would lead
to earlier-than-expected TF compilation requests.
Also, remove the runtime call between loops in osr-from-ml-to-tf.js
to not give TF too much time to tier up. It should no longer happen
now that ticks are cleared, but there's also no reason for the runtime
function to between the loops.
Bug: v8:7700
Fixed: v8:13242
Change-Id: I901c636079de05cb62d2aae1c92f0285a0f8c1ad
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3867310
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82859}
This implementation sticks closely to what Ignition-to-Turbofan (and now
Sparkplug-to-TF) does. OSR is detected in the TieringManager by having
optimized code available, without having entered it. The osr_urgency is
increased to enable OSR for increasing loop depths. When a candidate
JumpLoop backedge is reached, we call into runtime to trigger OSR
compilation.
JumpLoop also detects the availability of cached OSR'd code. When a
matching OSR code object is available, Maglev 1) deoptimizes s.t. the
unoptimized frame layout is reconstructed, and 2) delegates the actual
OSR tierup to the unoptimized tier. For purposes of 1), we add a new
DeoptimizeReason that causes a one-time eager deopt without invalidating
any code.
Drive-by: Annotate OSR for more --trace-opt output.
Todo: Refactor non-Sparkplug-specific bits of the BaselineAssembler
into a generic spot that both SP and ML can use.
Bug: v8:7700
Change-Id: I6ebab2df8b87f9f70ffb78162a3c1226ec545468
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3859850
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82816}
Bump the limit to give TF enough time to finish compiling. The
`keep_going` limit is fairly ugly, but it lets us test the real
(=concurrent) pipeline.
Bug: v8:7700
Fixed: v8:13176
Change-Id: Iba97111d752c8a4894e99ab57e8f42abcc8c29bb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3862204
Auto-Submit: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82778}
We should only tier to Sparkplug from Ignition. Previously, the
'!ActiveTierIsSparkplug' condition was correct since only Ignition (and
Sparkplug itself) entered the TieringManager. But now that Maglev is
also present, the condition must be clarified to 'ActiveTierIsIgnition'.
Bug: v8:7700
Change-Id: Ic7ea898facf9ce65407d09cead35e0181e49db31
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3835562
Commit-Queue: Jakob Linke <jgruber@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82519}
This is a reland of commit 532ca59910
Fix interger overflow when result_location is invalid in
MaglevCompiler::InReturnValues.
Original change's description:
> [maglev] Support LdaModuleVariable and StaModuleVariable
>
> Bug: v8:7700
> Change-Id: I036ac71324e0c1c96a4da4aacdb5a6718726db31
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3821203
> Reviewed-by: Victor Gomes <victorgomes@chromium.org>
> Commit-Queue: 王澳 <wangao.james@bytedance.com>
> Cr-Commit-Position: refs/heads/main@{#82347}
Bug: v8:7700
Change-Id: I24f56691eefd1c6cb695fedd3b5c14264bb17943
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3824942
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82379}
ML-TF tiering remains very similar to Ignition-TF tiering:
- When the interrupt budget is exhausted, enter the TieringManager
which potentially decides to tier up and sets the appropriate
TieringState on the FeedbackVector.
- The prologue on ML function entry recognizes the TieringState (and
also available cached TF code) and starts compilation and/or jumps
into optimized code.
TODOs:
- ML-to-TF OSR is not yet supported.
- ML code is no longer cached on the FeedbackVector.
- Tracing is rudimentary.
- The generated function-entry prologue is fairly large and must be
either minimized or extracted into a builtin.
- Tiering involving Sparkplug is not entirely robust yet (Sparkplug
code may be installed with unexpected timing).
Bug: v8:7700
Change-Id: I86b0692477f51b9967f318a4093bc874344120b3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3629149
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#82351}
Add an implementation of BranchIfInt32Compare, which is emitted whenever
a compare op is immediately followed by a branch.
Bug: v8:7700
Change-Id: I2c56d9de199bac8de33b33201f8614aee8e9894e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3647693
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80694}
Add Int32/Float64 nodes for:
* Subtract
* Multiply
* Divide
and additionally Int32 nodes for
* BitwiseOr/And/Xor
* ShiftLeft/Right/RightLogical
The latter ones don't have Float64 equivalents since they're implicitly
Int32 operations. In the future we'll add support for Number feedback by
adding Float64-to-Int32 conversions and using the Int32 nodes.
The divide node does an Int32 division and deopts if there's a remainder
to the division -- we may want to make it output a Float64 instead if we
think that's more likely in real-world code. There's also no peephole
optimisations for constant operations, which would generate much better
code, especially for shifts.
Bug: v8:7700
Change-Id: Ief1d24b46557cf4d2b7929ed50956df7b0d25992
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3652301
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80670}
Consider the following
* A ValueNode has inputs A and B
* Input A has later uses, input B doesn't
* The ValueNode's result must be in the same register as A
It can then happen that UpdateUses frees B, and the result allocation
emits a gap move from A's register to B's old register (now free) to
preserve the value of A when the ValueNode writes into its register.
This gap move is emmitted before the ValueNode start, which means that
it clobbers B.
Now, UpdateUses only clears registers _after_ node result allocation, so
that the known free registers are still the ones before updating uses.
Done naively, this would have bad consequences -- in the case where A
has no later uses, it would still force the regalloc to save its value
thinking that it is still live. So, this patch also introduces a concept
of "AllocationStage" where we're either allocating at the start or end
of a Node. Inputs are allocated at the start, results at the end. When
walking registers during an allocation, nodes whose lifetimes end at the
current node are considered to be dead at the "end" allocation stage,
and we are allowed to a) use their registers, and b) drop them without
preserving their value.
Bug: v8:7700
Change-Id: I5ca764ed04b12269f189577e81eb7e2a27cd1b09
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3625978
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80371}
To be consistent with the all the other tiers and avoid confusion, we
rename --opt to ---turbofan, and --always-opt to --always-turbofan.
Change-Id: Ie23dc8282b3fb4cf2fbf73b6c3d5264de5d09718
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3610431
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80336}
- Supports Float64 Add for SmiAdd bytecode
- Adds a Float64Constant and ChangeInt32ToFloat64 nodes
- Converts floats to tagged in Phi node inputs
- Fixes spill double representation
- Fixes materialisation during a deopt of a double in the stack
Bug: v8:7700
Change-Id: I9217a64313b4bd5d0015f935c23771ecf9a2c7ca
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3610426
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80255}
Add an unboxing double field load node, and fix a couple of locations
where it might be used enough to pass tests.
Bug: v8:7700
Change-Id: Ic134484e87a4fa363cbd8a3de667ac8e8116d502
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3610623
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80221}
This fixes the ordering of DCHECKs which expect the value to not be in
the free list yet when it is dropped.
Bug: v8:7700
Change-Id: Ifb85d0e20cfe5c083d1f2bc971817143265fdc7f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3610444
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80218}
CheckedFloat64Unbox mutates the input value, but the register allocator
does not expects this behaviour and propagates a wrong value in the register.
In particular we deopt with the wrong value if the second Float64Unbox
in a Float64Add needs to deopt.
This fixes the input value after we convert to double.
Bug: v8:7700
Change-Id: Ib89573e9f728dc3a34b817fc84f1afcb96f14d18
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3610422
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Auto-Submit: Victor Gomes <victorgomes@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80214}
- For simplicity we call a builtin when allocating a number.
- Elision of boxing/unboxing nodes will be done in a followup CL.
Bug: v8:7700
Change-Id: Iec4422d84c6597d3369ab512a1662adb0f077c98
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3602514
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Auto-Submit: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80178}
In the simplest way possible.
Bug: v8:7700
Change-Id: I155aaf85192b75c89617820d6f127a2ae04c7d9b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3599484
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80089}
Add a --maglev-inlining flag, and add some half-baked support for
inlining functions when there is call feedback.
When the flag is enabled and there is call feedback, we create a nested
MaglevGraphBuilder for the current graph, and pause building the graph
of the outer function. We manually set up its prologue to set up its
frame with the arguments pass into the call, build the body with the
nested graph builder. This inner builder knows that it is building an
inlined function, and all Return bytecodes will instead emit a Jump to a
single merge block at the end of the function, where execution of the
outer function can resume.
These inner function basic blocks are wired into the outer graph with
new JumpToInline and JumpFromInline control nodes. The idea is that
subsequent passes will know what the inline function is, and will use
these to manage the function stack (particularly for codegen and
especially deopts).
Bug: v8:7700
Change-Id: I4e9b153f8cf4d06c56e7be6365e7a18b86a773c0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3585958
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Jakob Linke <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80077}
If we have a smi operation in the feedback vector, we emit SmiTag
Int32AddWithOverflow and SmiUntag nodes, instead of a generic
operation binary node.
Change-Id: Idb9ce2b60289fbe492bf269793660b32de23e2b3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3560641
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Victor Gomes <victorgomes@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79775}
Nodes can now hold a LazyDeoptSafepoint which stores the frame state in
case they trigger a lazy deopt. OpProperties have a new CanLazyDeopt
bit, and codegen emits a safepoint table entry + lazy deopt for all
nodes with this bit. Also, we now check the deoptimized code bit on
entry into the maglev compiled function.
An example use of these lazy deopts is added as a PropertyCell fast path
for LdaGlobal, which adds a code dependency on the property cell.
Bug: v8:7700
Change-Id: I663db38dfa7325d38fc6d5f079d263a958074e36
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3557251
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Jakob Linke <jgruber@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79688}
Under over-application (passing more arguments into a function than its
formal parameter count), we need to use the passed argc to clean up the
stack, rather than the formal parameter count. Fix Maglev's Return node
code to do the appropriate check and dynamic sized return.
Bug: v8:7700
Change-Id: I36037d29e14323b336974d4b75b75f5702ce8a28
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3555767
Reviewed-by: Victor Gomes <victorgomes@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79649}
When --maglev is set, tier up to Maglev from unoptimized tiers based on
--interrupt-budget-for-maglev, initially set to 40KB (which should very
roughly by 1/10th of the time until the TF tierup decision is made).
On the first interrupt, a non-concurrent optimization to Maglev is
requested, which the next call to the marked function will perform.
- There is no support for tiering from Maglev to TF yet.
- Maglev's language support is minimal and tests are not expected to
pass with --maglev.
- Disable --maglev by default for now.
Drive-by: fixes related to Maglev flag definitions.
Bug: v8:7700
Change-Id: I121bb3f4f3830fdd20e1d4a12d3e04f08a99be38
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3500302
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79328}
Maglev is mid-tier optimising compiler designed mainly for compilation
speed that can still generate good code for straightforward JS.
This initial commit is an MVP for Maglev which can compile and run some
very simple code, and sets up a framework that we can build upon.
Design:
https://docs.google.com/document/d/13CwgSL4yawxuYg3iNlM-4ZPCB8RgJya6b8H_E2F-Aek/edit#
Bug: v8:7700
Change-Id: I5ae074ae099126c2c0d50864ac9b3d6fa5c9e85a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3483664
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/main@{#79247}
