This new FreeList should be a reasonable replacement for our old
FreeListLegacy: it is slightly less efficient (~1%), but uses much
less memory (often 5% less old_space size).
It is based on FreeListMany, with the following additions:
- A cache to waste less time iterating empty categories
- A fast path for allocations done in the runtime and generated code
- A slow path (the same as FreeListMany actually) for allocations
done in the GC.
Bug: v8:9329
Change-Id: Ifc10b88df7861266a721afd2c6e6d8357255ec4e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1762292
Commit-Queue: Darius Mercadier <dmercadier@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63345}
Do not pass it as argument to CreateObjectFillerAt and remove the enum.
Bug: v8:9454
Change-Id: Iafa37acbfea73d3cabb1732dbec0944db859fac2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1762017
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63282}
This reverts commit 60843b426b.
Reason for revert: TSAN detected issue between Scavenge workers.
One task could invoke RefillFreeList(), while the other task iterates the remembered set of a swept page.
Original change's description:
> Use list of invalidated objects for old-to-new refs
>
> Instead of inserting "deletion" entries into the store buffer, keep a
> list of invalidated objects to filter out invalid old-to-new slots.
>
> The first CL https://crrev.com/c/1704109 got reverted because both the
> sweeper and the main task were modifying the invalidated slots data
> structure concurrently. This CL changes this, such that the sweeper
> only modifies the invalidated slots during the final atomic pause when
> the main thread is not running. The sweeper does not need to clean this
> data structure after the pause, since the "update pointers" phase
> already removed all invalidated slots.
>
> The second CL https://crrev.com/c/1733081 got reverted because the
> sweeper might find more free space than the full GC before it. If an
> object shrinks after the pause but before the sweep, the invalidated
> object might span free memory and potentially new allocated objects.
> Therefore shrink invalidated objects when processing swept pages on
> the main thread. Also clean recorded slots in the gap.
>
> TBR=petermarshall@chromium.org
>
> Bug: v8:9454
> Change-Id: I80d1fa3bbc24e97f7c97a373aaad66f105456f12
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1751795
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63239}
TBR=ulan@chromium.org,hpayer@chromium.org,dinfuehr@chromium.org
Change-Id: I9c6a371ebe36a1873acbe0d6c6a75dd2f5a55f4e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:9454
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1760817
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63253}
Instead of inserting "deletion" entries into the store buffer, keep a
list of invalidated objects to filter out invalid old-to-new slots.
The first CL https://crrev.com/c/1704109 got reverted because both the
sweeper and the main task were modifying the invalidated slots data
structure concurrently. This CL changes this, such that the sweeper
only modifies the invalidated slots during the final atomic pause when
the main thread is not running. The sweeper does not need to clean this
data structure after the pause, since the "update pointers" phase
already removed all invalidated slots.
The second CL https://crrev.com/c/1733081 got reverted because the
sweeper might find more free space than the full GC before it. If an
object shrinks after the pause but before the sweep, the invalidated
object might span free memory and potentially new allocated objects.
Therefore shrink invalidated objects when processing swept pages on
the main thread. Also clean recorded slots in the gap.
TBR=petermarshall@chromium.org
Bug: v8:9454
Change-Id: I80d1fa3bbc24e97f7c97a373aaad66f105456f12
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1751795
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63239}
This reverts commit e2f98ec22c.
Reason for revert: Caused performance regression in ArrayLiteralInitialSpreadSmallHoley.
Original change's description:
> Use list of invalidated objects for old-to-new refs
>
> Instead of inserting "deletion" entries into the store buffer, keep
> a list of invalidated objects to filter out invalid old-to-new slots.
>
> The first CL https://crrev.com/c/1704109 got reverted because both the sweeper and the main task were modifying the invalidated slots data structure concurrently. This CL changes this, such that the sweeper only modifies the invalidated slots during the final atomic pause when the main thread is not running. The sweeper does not need to clean this data structure after the pause, since the "update pointers" phase already removed all invalidated slots.
>
> Bug: v8:9454
> Change-Id: Iffb5bf96de2c89eee1ee1231a3414a0f2a155cbc
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1733081
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Peter Marshall <petermarshall@chromium.org>
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63087}
TBR=ulan@chromium.org,petermarshall@chromium.org,dinfuehr@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:9454
Change-Id: I328b9f72df45fc9570d4a4d1b5389eac010638c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1743970
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63131}
Instead of inserting "deletion" entries into the store buffer, keep
a list of invalidated objects to filter out invalid old-to-new slots.
The first CL https://crrev.com/c/1704109 got reverted because both the sweeper and the main task were modifying the invalidated slots data structure concurrently. This CL changes this, such that the sweeper only modifies the invalidated slots during the final atomic pause when the main thread is not running. The sweeper does not need to clean this data structure after the pause, since the "update pointers" phase already removed all invalidated slots.
Bug: v8:9454
Change-Id: Iffb5bf96de2c89eee1ee1231a3414a0f2a155cbc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1733081
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63087}
This reverts commit dcac02ee81.
Reason for revert: TSAN discovered issue with cleaning invalidated slots in sweeper threads and inserting new slots in the main thread.
Original change's description:
> Use list of invalidated objects for old-to-new refs
>
> Instead of inserting "deletion" entries into the store buffer, keep
> a list of invalidated objects to filter out invalid old-to-new slots.
>
> Bug: v8:9454
> Change-Id: Icd21d8cb2159190457f54d0f8b56742ecc820419
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1695474
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Peter Marshall <petermarshall@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#62744}
TBR=ulan@chromium.org,petermarshall@chromium.org,dinfuehr@chromium.org
Change-Id: I4278e9100c76657663e0a6a62f5d86bb3a343c0e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:9454
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1704109
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62746}
Instead of inserting "deletion" entries into the store buffer, keep
a list of invalidated objects to filter out invalid old-to-new slots.
Bug: v8:9454
Change-Id: Icd21d8cb2159190457f54d0f8b56742ecc820419
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1695474
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62744}
This CL was generated by an automatic clang AST rewriter using this
matcher expression:
callExpr(
callee(
cxxMethodDecl(
hasName("operator->"),
ofClass(isSameOrDerivedFrom("v8::internal::Object"))
)
),
argumentCountIs(1)
)
The "->" at the expression location was then rewritten to ".".
R=jkummerow@chromium.orgTBR=mstarzinger@chromium.org,verwaest@chromium.org,yangguo@chromium.org
Bug: v8:9183, v8:3770
No-Try: true
No-Tree-Checks: true
Change-Id: I0a7ecabdeafe51d0cf427f5280af0c7cab96869e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1624209
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Yang Guo <yangguo@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61764}
Even though both are allowed in the style guide, it recommends to use
'using', as its syntax is more consistent with the rest of C++.
This CL turns all typedefs in heap code to 'using' declarations.
R=mstarzinger@chromium.org
Bug: v8:8834
Change-Id: I8a9f6e1eebdd0adca4373c866e95ebab0a1e992d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1545892
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60523}
This replaces InNewSpace with InYoungGeneration, which is
a prerequisite for young large objects.
Additional changes:
- FROM_SPACE, TO_SPACE flags are renamed to FROM_PAGE, TO_PAGE.
- A new LARGE_PAGE flag is added.
- The external string table is refactored to track young string
instead of new space strings.
Bug: chromium:924547
Change-Id: Ia4e3ba1b72995c3400257a1f98559f091533e811
Reviewed-on: https://chromium-review.googlesource.com/c/1437274
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59156}
Mostly signed integer overflows, and a few cases of double
division by zero (which is defined by IEEE-754 to return
Infinity (or NaN for 0/0) but is UB in C++).
In base/ieee754.cc, use constants for NaN and Infinity instead
of computing these values.
In spaces-unittest.cc, ensure that a large enough allocation
is used.
Bug: v8:3770
Change-Id: I50d9a77dc860ef9993b7b269a5f8c117b0f62f9d
Reviewed-on: https://chromium-review.googlesource.com/c/1403454
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58701}
The two names refer to the same thing by now, so this patch is
entirely mechanical.
Bug: v8:3770
Change-Id: Ia360c06c89af6b3da27fd21bbcaeb2bdaa28ce22
Reviewed-on: https://chromium-review.googlesource.com/c/1397705
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58615}
Merging the temporary HeapObjectPtr back into HeapObject.
Bug: v8:3770
Change-Id: I5bcd23ca2f5ba862cf5b52955dca143e531c637b
Reviewed-on: https://chromium-review.googlesource.com/c/1386492
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58410}
It was accidentally removed in CL that introduced BoundedPageAllocator.
This CL also cleans up the CodeRangeAddressHint a bit.
Bug: v8:8096, chromium:887252
Change-Id: Idc84796dd1ff1b440cbe3515732984264defcf2d
Reviewed-on: https://chromium-review.googlesource.com/1249125
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56287}
This patch adds a singleton that tracks recently freed code range
regions and provides hints for newly created code ranges such that
the freed addresses are reused.
This is a workaround for the CFG leak described in the linked bug.
Bug: chromium:870054
Change-Id: Ice237a056268379f0fef40abdb1accad125a56b3
Reviewed-on: https://chromium-review.googlesource.com/1174837
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55139}
Instead of actually allocating an objects just test the corner cases
around the page boundary by casting addresses.
Bug: v8:7984
Change-Id: I27615cc193d6f85abc91cfe898719a4a9b761f23
Reviewed-on: https://chromium-review.googlesource.com/1151114
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54717}
Move write barrier essentials into heap/heap-write-barrier-inl.h. Avoid
including further heap inline headers by relying on constant to load
flags from.
Bug: v8:7490
Change-Id: I2891299f1b1ca2c3e2031cb9c63b583b1665e3f9
Reviewed-on: https://chromium-review.googlesource.com/1148448
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54710}
In the spirit of the full MC, we evacuate and update pointers in parallel for
the young generation.
The collectors are connected during incremental marking when mark bits are
transferred from the young generation bitmap to the old generation bitmap.
The evacuation phase cannot (yet) move pages and relies completely on copying
objects.
BUG=chromium:651354
Review-Url: https://codereview.chromium.org/2796233003
Cr-Commit-Position: refs/heads/master@{#45074}
