Fix returning from TryAllocateLinearlyAligned without updating the
allocation stats if a preceding filler was required. Also makes
AllocateRaw take an int instead of size_t in line with other Spaces.
Bug: v8:8875, chromium:1097389
Change-Id: If0932caa94dce1cd45b41f44fa225a2007772ea1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2264354
Auto-Submit: Dan Elphick <delphick@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68516}
After https://chromium-review.googlesource.com/c/v8/v8/+/2250254,
allocated_object_size in RO_SPACE is incorrect. This changes it to use
the accounting_stats_ value. This also fixes the Capacity() which was
previously uninitialized. Both are tested in new ReadOnlySpace allocation
tests in test-spaces.cc.
Couple of cleanups:
* area_size_ becomes const since its value is fixed after construction.
* Deletes incorrect comment in base-space.h
Bug: v8:10454
Change-Id: I9bbbc1ef2548722eee9dae1bb8d67448eccf8955
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2259937
Commit-Queue: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68489}
Add nullptr guard for the return value of AllocateBasicChunk.
Bug: chromium:1097502
Change-Id: Ia4642151a119ccabe58d7084077808aac93e5d1c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2257221
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dan Elphick <delphick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68454}
Also makes memory-chunk.h accessible from outside heap which allows
removal of some heap-inl.h includes.
Bug: v8:10473, v8:10496
Change-Id: Iec4fc5ce8ad201f6ee5fd924cc3cd935324429fc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2172088
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Auto-Submit: Dan Elphick <delphick@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67551}
Creates a new large-spaces.h and large-spaces.cc to contain
LargeObjectSpace and subclasses.
Bug: v8:10473
Change-Id: Ifdb4eac9df5c8213f992d549e04b612b62f6df0b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2170826
Commit-Queue: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67456}
The scope disables garbage collection and should be only used in
heap, deserializer, isolate bootstrap, and testing.
Change-Id: Ide95926ef32fd9362cd9134e883e1bd626cc3b11
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2083292
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66557}
After support for ARCH_PPC was dropped, it became a subset of
ARCH_PPC64. If you compile for ppc64, then you set the ARCH_PPC64
define which also sets the ARCH_PPC define.
To be able to again support ppc (32 bit) those defines should be
split up again.
This commit only splits up the defines but does not introduce a
working ARCH_PPC variant.
Bug: v8:10102
Change-Id: I64e0749f8e5a7dc078ee7890d92e57b82706a849
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1989826
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Milad Farazmand <miladfar@ca.ibm.com>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66390}
Both LO_SPACE and NEW_LO_SPACE use the basic page management system of
LargeObjectSpace, but implement different AllocateRaw methods (with
the NEW_LO_SPACE version shadowing the LO_SPACE version).
To clean this up, and allow other future LargeObjectSpace implementations
(in particular, an off-thread variant), refactored the current
LargeObjectSpace into a base class, and make both LargeObjectSpace
(renamed to OldLargeObjectSpace) and NewLargeObjectSpace extend this
class.
Bug: chromium:1011762
Change-Id: I41b45b97f2611611dcfde677213131396df03a5e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1876824
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#64560}
Do not pass it as argument to CreateObjectFillerAt and remove the enum.
Bug: v8:9454
Change-Id: Iafa37acbfea73d3cabb1732dbec0944db859fac2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1762017
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63282}
This reverts commit 60843b426b.
Reason for revert: TSAN detected issue between Scavenge workers.
One task could invoke RefillFreeList(), while the other task iterates the remembered set of a swept page.
Original change's description:
> Use list of invalidated objects for old-to-new refs
>
> Instead of inserting "deletion" entries into the store buffer, keep a
> list of invalidated objects to filter out invalid old-to-new slots.
>
> The first CL https://crrev.com/c/1704109 got reverted because both the
> sweeper and the main task were modifying the invalidated slots data
> structure concurrently. This CL changes this, such that the sweeper
> only modifies the invalidated slots during the final atomic pause when
> the main thread is not running. The sweeper does not need to clean this
> data structure after the pause, since the "update pointers" phase
> already removed all invalidated slots.
>
> The second CL https://crrev.com/c/1733081 got reverted because the
> sweeper might find more free space than the full GC before it. If an
> object shrinks after the pause but before the sweep, the invalidated
> object might span free memory and potentially new allocated objects.
> Therefore shrink invalidated objects when processing swept pages on
> the main thread. Also clean recorded slots in the gap.
>
> TBR=petermarshall@chromium.org
>
> Bug: v8:9454
> Change-Id: I80d1fa3bbc24e97f7c97a373aaad66f105456f12
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1751795
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63239}
TBR=ulan@chromium.org,hpayer@chromium.org,dinfuehr@chromium.org
Change-Id: I9c6a371ebe36a1873acbe0d6c6a75dd2f5a55f4e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:9454
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1760817
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63253}
Instead of inserting "deletion" entries into the store buffer, keep a
list of invalidated objects to filter out invalid old-to-new slots.
The first CL https://crrev.com/c/1704109 got reverted because both the
sweeper and the main task were modifying the invalidated slots data
structure concurrently. This CL changes this, such that the sweeper
only modifies the invalidated slots during the final atomic pause when
the main thread is not running. The sweeper does not need to clean this
data structure after the pause, since the "update pointers" phase
already removed all invalidated slots.
The second CL https://crrev.com/c/1733081 got reverted because the
sweeper might find more free space than the full GC before it. If an
object shrinks after the pause but before the sweep, the invalidated
object might span free memory and potentially new allocated objects.
Therefore shrink invalidated objects when processing swept pages on
the main thread. Also clean recorded slots in the gap.
TBR=petermarshall@chromium.org
Bug: v8:9454
Change-Id: I80d1fa3bbc24e97f7c97a373aaad66f105456f12
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1751795
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63239}
This reverts commit e2f98ec22c.
Reason for revert: Caused performance regression in ArrayLiteralInitialSpreadSmallHoley.
Original change's description:
> Use list of invalidated objects for old-to-new refs
>
> Instead of inserting "deletion" entries into the store buffer, keep
> a list of invalidated objects to filter out invalid old-to-new slots.
>
> The first CL https://crrev.com/c/1704109 got reverted because both the sweeper and the main task were modifying the invalidated slots data structure concurrently. This CL changes this, such that the sweeper only modifies the invalidated slots during the final atomic pause when the main thread is not running. The sweeper does not need to clean this data structure after the pause, since the "update pointers" phase already removed all invalidated slots.
>
> Bug: v8:9454
> Change-Id: Iffb5bf96de2c89eee1ee1231a3414a0f2a155cbc
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1733081
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Peter Marshall <petermarshall@chromium.org>
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#63087}
TBR=ulan@chromium.org,petermarshall@chromium.org,dinfuehr@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:9454
Change-Id: I328b9f72df45fc9570d4a4d1b5389eac010638c7
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1743970
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63131}
Instead of inserting "deletion" entries into the store buffer, keep
a list of invalidated objects to filter out invalid old-to-new slots.
The first CL https://crrev.com/c/1704109 got reverted because both the sweeper and the main task were modifying the invalidated slots data structure concurrently. This CL changes this, such that the sweeper only modifies the invalidated slots during the final atomic pause when the main thread is not running. The sweeper does not need to clean this data structure after the pause, since the "update pointers" phase already removed all invalidated slots.
Bug: v8:9454
Change-Id: Iffb5bf96de2c89eee1ee1231a3414a0f2a155cbc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1733081
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63087}
This reverts commit dcac02ee81.
Reason for revert: TSAN discovered issue with cleaning invalidated slots in sweeper threads and inserting new slots in the main thread.
Original change's description:
> Use list of invalidated objects for old-to-new refs
>
> Instead of inserting "deletion" entries into the store buffer, keep
> a list of invalidated objects to filter out invalid old-to-new slots.
>
> Bug: v8:9454
> Change-Id: Icd21d8cb2159190457f54d0f8b56742ecc820419
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1695474
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Peter Marshall <petermarshall@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#62744}
TBR=ulan@chromium.org,petermarshall@chromium.org,dinfuehr@chromium.org
Change-Id: I4278e9100c76657663e0a6a62f5d86bb3a343c0e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:9454
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1704109
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62746}
Instead of inserting "deletion" entries into the store buffer, keep
a list of invalidated objects to filter out invalid old-to-new slots.
Bug: v8:9454
Change-Id: Icd21d8cb2159190457f54d0f8b56742ecc820419
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1695474
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Peter Marshall <petermarshall@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62744}
The linked lists of FreeLists could contain empty elements
(FreeListCategories whose `top()` is null). The code is carefuly
written so that this case won't break anything (probably just slow
things a little bit). When `FreeList::FindNodeIn` (the fast path of
`FreeList::Allocate`) found such an empty `FreeListCategory`, it
removed it by calling `FreeList::RemoveCategory`, and looked in the
next `FreeListCategory` of the same size. However, on the slow path of
`FreeList::Allocate`, the functions that iterates the
`FreeListCategory` are `FreeList::TryFindNodeIn` and
`FreeListCategory::PickNodeFromList`, none of which removed empty
elements. Therefore, it could happen that a `FreeListCategory` "real"
first element could be used, but was never considered due to the top of
the linked list being empty. The behavior for the slow path should be
the same as for the fast path on that regard.
The problem was actually deeper than that: FreeListCategories were not
always in a consistent state, since they could have empty members. The
removal of those empty elements should be done as soon as they are
created, ie when allocating the last element.
This CL ensures that empty FreeListCategories are removed as soon as
they become empty.
Bug: v8:9329
Change-Id: Idda8096dc5978745894854a0405da59f7e8691a6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1648476
Commit-Queue: Darius Mercadier <dmercadier@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62058}
When trying to find a spot in the freelists for a tiniest object,
the tiny freelist was never searched.
This was fixed by modifying FreeList::Allocate in order to handle
that special case.
A test was added in cctest/heap/test-spaces.cc. It allocates a
Tiny object on a new page, then fills up the page, then frees the
first object, and finally tries to allocate a Tiniest
object. Before, this Tiniest object would go on a different page;
now it goes on the same one (which is what the test checks for).
Bug: v8:9329
Change-Id: Ia810726d1bfe1dae4ef2055a7f5b314b1514ee9c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1647162
Commit-Queue: Darius Mercadier <dmercadier@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#62055}
This CL was generated by an automatic clang AST rewriter using this
matcher expression:
callExpr(
callee(
cxxMethodDecl(
hasName("operator->"),
ofClass(isSameOrDerivedFrom("v8::internal::Object"))
)
),
argumentCountIs(1)
)
The "->" at the expression location was then rewritten to ".".
R=jkummerow@chromium.orgTBR=mstarzinger@chromium.org,verwaest@chromium.org,yangguo@chromium.org
Bug: v8:9183, v8:3770
No-Try: true
No-Tree-Checks: true
Change-Id: I0a7ecabdeafe51d0cf427f5280af0c7cab96869e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1624209
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Yang Guo <yangguo@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61764}
Also drive-by cleanup the TestMemoryAllocatorScope class so that it
takes ownership of the old allocator while it holds onto it, and so
that the MemoryAllocator for testing is constructed inside the scope
rather than passed into it. This means users don't need to explicitly
call TearDown() and delete the allocator as the scope does it for them.
Change-Id: Id7da3c074618a376d2edfe3385bb185ba8287cea
Reviewed-on: https://chromium-review.googlesource.com/c/1392194
Commit-Queue: Peter Marshall <petermarshall@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59627}
HeapNumbers and MutableHeapNumber requires alignment for their double
value field but for now this field can be misaligned on 32-bit platforms.
According to code in Heap::GetFillToAlign() function, kDoubleUnaligned
doesn't actually mean "unaligned" but "aligned to half of double".
This CL fixes this misalignment.
Change-Id: I9b9c58d580bb287e7dad44bc96cd6b4593707b5e
Reviewed-on: https://chromium-review.googlesource.com/c/1470113
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59599}
Only Heap::GcSafeFindCodeForInnerPointer requires the chunk map.
Other large object spaces use more the efficient
MemoryChunk::FromAnyPointerAddress.
Additionally, this patch renames Register/Unregister to AddPage/RemovePage
to be consistent with other spaces and makes them virtual.
Bug: chromium:852420
Change-Id: I8d637bb59e15bd61fe452fda7f4a55049d32030c
Reviewed-on: https://chromium-review.googlesource.com/c/1439417
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59207}
It is slightly faster than Page::FromAddress(o->address()) (saves
one instruction), and more concise than Page::FromAddress(o->ptr()).
Same for MemoryChunk::FromHeapObject().
Bug: v8:8562
Change-Id: I9dc2b787aed5cad2d4087850dfa2e8eb157ad225
Reviewed-on: https://chromium-review.googlesource.com/c/1405031
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58806}
Merging the temporary HeapObjectPtr back into HeapObject.
Bug: v8:3770
Change-Id: I5bcd23ca2f5ba862cf5b52955dca143e531c637b
Reviewed-on: https://chromium-review.googlesource.com/c/1386492
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58410}
The flag was not used anymore and any CollectGarbage call will finalize
marking.
Change-Id: I29ee60b187c9038acc4b42b8334546498f54f117
Reviewed-on: https://chromium-review.googlesource.com/1228013
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56018}
VirtualMemory objects can be moved since https://crrev.com/c/1213062,
so there is no need any more to return them via pointer argument. This
also makes the {AllocVirtualMemory} and {AlignedAllocVirtualMemory}
functions superfluous.
R=ishell@chromium.org, titzer@chromium.org
Bug: v8:8015
Change-Id: Id72921e1c66a6c10be6647194603b8283e010e24
Reviewed-on: https://chromium-review.googlesource.com/1226972
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ben Titzer <titzer@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55947}
This is a reland of 16816e53be
Bug: v8:8096
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Change-Id: I257fc391931a0a4bf01f2e8136183aaed044231c
Reviewed-on: https://chromium-review.googlesource.com/1226915
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55928}
The provided page allocator will serve all the memory requests done by the virtual
memory object.
This is a necessary cleanup before introducing BoundedPageAllocator.
Bug: v8:8096
Change-Id: I95477d67e5f532013322a991db3ee1a1f2e821e6
Reviewed-on: https://chromium-review.googlesource.com/1210122
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55703}
... like AllocatePage[s](), FreePages() and SetPermissions().
This CL also changes base::PageAllocator to cache AllocatePageSize and CommitPageSize
values returned by the OS.
This is a necessary cleanup before introducing BoundedPageAllocator.
Bug: v8:8096
Change-Id: Ifb7cdd2caa6a1b029ce0fca6545c61df9d281be2
Reviewed-on: https://chromium-review.googlesource.com/1209343
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55690}
In future the RO_SPACE root accessors in Heap will become private, so
instead convert them all to use ReadOnlyRoots.
Bug: v8:7786
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Change-Id: I2f2c031c03d56d360ef940fc925e0583e6ae31dc
Reviewed-on: https://chromium-review.googlesource.com/1125720
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Dan Elphick <delphick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54202}
