The handle was always created empty which resulted in a DCHECK crash
in debug builds and in never-cancelled tasks in release builds.
Bug: chromium:1056170
Change-Id: I798ce65c37738bbe9c60b44b692ff04536f6d830
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2388101
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69653}
Heap growing estimates when to start incremental gc such that it
will finish when we are expecting to finalize (i.e. when an atomic
gc would be triggered).
There is also a minimum ratio between limit for atomic gc and limit
for incremental gc, to guarantee that incremental gc get's some time to
run even with the application rarely allocates.
This is a continuation of:
https://chromium-review.googlesource.com/c/v8/v8/+/2377691
Bug: chromium:1056170
Change-Id: I8c87e98d60b6f8b5748558771a236f15385f7858
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2381454
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69630}
With conservative stack scanning enabled, a snapshot of the call stack
upon entry to GC will be used to determine part of the root-set. When
the collector walks the stack, it looks at each value and determines
whether it could be a potential on-heap object pointer. However, unlike
with Handles, these on-stack pointers aren't guaranteed to point to the
start of the object: the compiler may decide hide these pointers, and
create interior pointers in C++ frames which the GC doesn't know about.
The solution to this is to include an object start bitmap in the header
of each page. Each bit in the bitmap represents a word in the page
payload which is set when an object is allocated. This means that when
the collector finds an arbitrary potential pointer into the page, it can
walk backwards through the bitmap until it finds the relevant object's
base pointer. To prevent the bitmap becoming stale after compaction, it
is rebuilt during object sweeping.
This is experimental, and currently only works with inline allocation
disabled, and single generational collection.
Bug: v8:10614
Change-Id: I28ebd9562f58f335f8b3c2d1189cdf39feaa1f52
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2375195
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69615}
Explicit nullification aims to simplify migration to Oilpan, in the
case when unique_ptrs are converted to Member and user code relies on
source pointers to be in "empty" state.
Change-Id: Ia54137d53ca03f93932b3c1f2eaba439a416a06e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2379857
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Auto-Submit: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69603}
Schedule is simpler compared to the schedule in blink since it now
returns deadlines based on marked bytes instead of time.
If marking is ahead of schedule, return the minimum step size.
Otherwise, set step size to catch up to schedule (ignoring the time
passed while performing the step).
No more default initial step size (needed in blink since marking speed
was unknown).
If estimated schedule is exceeded (marking takes longer than 500ms), the
steps will try to mark all remaining objects but would still be capped
by the maximum step duration of 2ms.
Bug: chromium:1056170
Change-Id: I09857db161c621a12d064f9c8c21b646c34f9d71
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2375200
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69602}
Starting marking required Creating a Marker and calling StartMarking.
StartMarking should always have been called immediately after creating
the marker.
Since markers are not persisted between GC (a marker exists only while
marking is in progress), it makes sense to start marking implicitly when
a marker is created.
Calling StartMarking in MarkerBase ctor is inadvisable since subclasses
might still to initialize fields.
Using MarkerFactory instead guarantees that StartMarking is always
called immediately after creating a Marker.
Bug: chromium:1056170
Change-Id: Icbf11afd848e1618c204ca6bf951600b3ae9fef2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2375199
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69601}
- JobHandle::IsCompleted()
- JobDelegate::GetTaskId()
- worker_count passed as argument to GetMaxConcurrency().
Jobs implementation must call the new GetMaxConcurrency(), but Jobs
users aren't migrated yet.
Bug: chromium:1114823
Change-Id: I0f4295ccaf9eba866dd771f30e2e49aa3eae9551
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2352484
Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69553}
This CL adds a bytes based deadline to draining of worklist.
The time based deadline is also kept because:
1) Unified heap can't transition to bytes-based deadlines yet.
2) Unified heap with concurrent marking needs to flush v8 references
which don't count as marked_bytes and can cause very long incremental
pauses.
Bug: chromium:1056170
Change-Id: I5ab57754e7ff0b5821f3acb76e1e6f59fc9d68b8
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2299374
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69506}
This CL adds a basic implementation of incremental marking for standalone GC.
Followup CLs include:
* Use bytes instead of time as deadline
* Port incremental marking schedule from blink
* Mark on allocation
* Guarantees for progres/termination for standalone GC
* etc...
Calling StartIncrementalGarbageCollection triggers StartMarking which
schedules incremental marking as non-nestable tasks.
For unified heap, marking will continue running until it runs out of
work but it won't finalize independently.
For standalone, when incremental runs out of work it will schedule a new
task in which it will finalize marking and trigger the rest of the GC.
Users of standalone can also force finalization before incremental
marking as finished using FinalizeIncrementalGarbageCollectionIfRunning.
Calling CollectGarbage would also finalize an on-going incremental GC
if one exists. Otherwise it will trigger an atomic GC.
See the following doc for explanation of the various methods:
https://docs.google.com/document/d/1ZhJY2fOoD8sH53ZxMh2927Zl8sXqA7azJgcQTWx-YKs/edit?usp=sharing
Bug: chromium:1056170
Change-Id: I75ead414eb9da9f8b7f71c4638b9830fce7708ca
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2298009
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69480}
Allow AllocationObserver::Step to remove allocation observers as well.
They could already add new observers (to start incremental marking),
removing will be used when starting StressConcurrentAllocator from
an allocation observer.
Bug: v8:10315
Change-Id: I823ded2f9a408b3fa5269ee8416060d0cabb3162
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2357690
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69420}
This patch introduces a new LocalIsolate and LocalFactory, which use
LocalHeap and replace OffThreadIsolate and OffThreadFactory. This allows
us to remove those classes, as well as the related OffThreadSpace,
OffThreadLargeObjectSpace, OffThreadHeap, and OffThreadTransferHandle.
OffThreadLogger becomes LocalLogger.
LocalHeap behaves more like Heap than OffThreadHeap did, so this allows
us to additionally remove the concept of "Finish" and "Publish" that the
OffThreadIsolate had, and allows us to internalize strings directly with
the newly-concurrent string table (where the implementation can now move
to FactoryBase).
This patch also removes the off-thread support from the deserializer
entirely, as well as removing the LocalIsolateWrapper which allowed
run-time distinction between Isolate and OffThreadIsolate. LocalHeap
doesn't support the reservation model used by the deserializer, and we
will likely move the deserializer to use LocalIsolate unconditionally
once we figure out the details of how to do this.
Bug: chromium:1011762
Change-Id: I1a1a0a72952b19a8a4c167c11a863c153a1252fc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2315990
Commit-Queue: Andreas Haas <ahaas@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69397}
Move external memory counters out of IsolateData back into Heap.
The class ExternalMemoryAccounting now stores all counters and is
responsible for updates. This change will allow turning counters into
atomic variables.
Bug: v8:10315
Change-Id: I2abeda298d3cfcc630fd04ca78a3d6d703e3b419
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2346647
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69356}
This is the first step in refactoring Worklist to allow arbitrary
number of local worklists with private segments:
- Introduce MarkingWorklistImpl<> which will eventually replace
(and will be renamed to) Worklist.
- MarkingWorklistImpl<> owns the global pool of segments but does not
keep track of private segments.
- MarkingWorklistImpl<>::Local owns private segments and can be
constructed dynamically on background threads.
- Rename the existing MarkingWorklistsHolder to MarkingWorklists.
- Rename the existing MarkingWorklists to MarkingWorklists::Local.
- Rename the existing marking_workists_holder to marking_worklists.
- Rename the existing marking_worklists to local_marking_worklists.
Design doc: https://bit.ly/2XMtjLi
Bug: v8:10315
Change-Id: I9da34883ad34f4572fccd40c51e51eaf50c617bc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2343330
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69330}
This is a revival of https://chromium-review.googlesource.com/c/v8/v8/+/2228332
The CL establishes the following:
*) Objects are marked before being pushed to the worklists.
*) Live bytes are always accounted after tracing an object (i.e. move
from Gray to Black below).
*) Previously not fully constructed objects are traced immediately
instead of pushed to the marking worklist.
This establishes the following invariants for all marking worklists:
1) White = !object.is_marked() && !worklist.contains(object)
2) Gray = object.is_marked() && worklist.contains(object)
3) Black = object.is_marked() && !worklist.contains(object)
Bug: chromium:1056170
Change-Id: I821573b3fbc057e6ffb836154271ff986ecb4d2b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2336797
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69268}
DetachPersistent() sets up PersistentHandles, but didn't properly set
up ordered_blocks_. So PersistentHandles::Contains failed for handles
that were detached from the main thread into PersistentHandles.
Bug: v8:10315
Change-Id: I5374ad64743cd519a9c5e92900c1fa401c4d93ab
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2336801
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69243}
Test now passes even if RO_SPACE sharing is enabled.
Bug: v8:10454
Change-Id: Ic7377c3367199383bb6a96a9beedcc52bbc3362f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2335184
Commit-Queue: Dan Elphick <delphick@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Auto-Submit: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69221}
This is a reland of b354e344fd
This CL adds 3 fixes:
* Unprotect code object before creating filler
* Allows AllocationObserver::Step to add more AllocationObservers
* Update limit in NewSpace::UpdateLinearAllocationArea
Original change's description:
> [heap] Refactor allocation observer in AllocationCounter
>
> Moves accounting of allocation observers into the AllocationCounter
> class. This CL removes top_on_previous_step_ for counters that are
> increased regularly in the slow path of the allocation functions.
>
> AdvanceAllocationObservers() informs the AllocationCounter about
> allocated bytes, InvokeAllocationObservers() needs to be invoked when
> an allocation step is reached. NextBytes() returns the number of bytes
> until the next AllocationObserver::Step needs to run.
>
> Bug: v8:10315
> Change-Id: I8b6eb8719ab032d44ee0614d2a0f2645bfce9df6
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2320650
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#69170}
Bug: v8:10315
Change-Id: I89ab4d5069a234a293471f613dab16b47d8fff89
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2332805
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69216}
This reverts commit b354e344fd.
Reason for revert: Clusterfuzz found issues with this CL.
Original change's description:
> [heap] Refactor allocation observer in AllocationCounter
>
> Moves accounting of allocation observers into the AllocationCounter
> class. This CL removes top_on_previous_step_ for counters that are
> increased regularly in the slow path of the allocation functions.
>
> AdvanceAllocationObservers() informs the AllocationCounter about
> allocated bytes, InvokeAllocationObservers() needs to be invoked when
> an allocation step is reached. NextBytes() returns the number of bytes
> until the next AllocationObserver::Step needs to run.
>
> Bug: v8:10315
> Change-Id: I8b6eb8719ab032d44ee0614d2a0f2645bfce9df6
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2320650
> Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#69170}
TBR=ulan@chromium.org,dinfuehr@chromium.org
Change-Id: Icd713207bfb2085421fd82009be24a0211ae86da
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:10315
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2332667
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69187}
Moves accounting of allocation observers into the AllocationCounter
class. This CL removes top_on_previous_step_ for counters that are
increased regularly in the slow path of the allocation functions.
AdvanceAllocationObservers() informs the AllocationCounter about
allocated bytes, InvokeAllocationObservers() needs to be invoked when
an allocation step is reached. NextBytes() returns the number of bytes
until the next AllocationObserver::Step needs to run.
Bug: v8:10315
Change-Id: I8b6eb8719ab032d44ee0614d2a0f2645bfce9df6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2320650
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69170}
PersistentHandlesScope works similar to the DeferredHandleScope, but
returns PersistentHandles instead of DeferredHandles on Detach().
Since PersistentHandlesScope takes over filled blocks from the
main thread local handle, remove the block_size_ field and use
kHandleBlockSize instead. This way all blocks have exactly the same size.
Bug: v8:10315
Change-Id: I295cad6f84852f87c55d95572905069443f5698c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2324254
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69138}
The following adds support for JSMember through the existing
GlobalHandles implementation also used for TracedReference.
In addition, JSMember now supports set, clear, copy, move, comparison
and interaction with Local.
Bug: chromium:1056170
Change-Id: Ia50218bcfe4c056b3533a5b14eea954ade1da243
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2310357
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#69028}
This CL ports MarkingVerifier from blink.
The existing verifier checks only references on heap.
This new verifier checks references both on heap and on stack.
Bug: chromium:1056170
Change-Id: I083dcb0087125312cca34a2201015a9aecfe6ea4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2300484
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68891}
This CL removes the GetTraceDescriptor virtual call from garbage
collected mixins and replaces it with querying the object start
bitmap.
The CL also removes the mixin macros which are now no longer needed.
Bug: chromium:1056170
Change-Id: I27ed299f93025d09a3bb3f0d17b14bed3c200565
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2287508
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68787}
This is a reland of bebb2bdc06
Original change's description:
> [heap] Allow LocalHeap on the main thread
>
> This changes the safepoint scope to skip LocalHeap that is active
> for the current thread to avoid deadlocking.
>
> Bug: v8:10315
> Change-Id: I45e80ae66d0dbbe768107aa9cf0603204c644d9f
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2289983
> Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
> Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#68769}
Bug: v8:10315
Tbr: dinfuehr@chromium.org
Change-Id: I1974d8b6ffffbf3244e7ede2d20d9b2d623df150
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2290851
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68785}
This reverts commit bebb2bdc06.
Reason for revert: Breaks GarbageCollectionWithLocalHeap test.
https://ci.chromium.org/p/v8/builders/ci/V8%20Linux%20-%20debug/31107
Original change's description:
> [heap] Allow LocalHeap on the main thread
>
> This changes the safepoint scope to skip LocalHeap that is active
> for the current thread to avoid deadlocking.
>
> Bug: v8:10315
> Change-Id: I45e80ae66d0dbbe768107aa9cf0603204c644d9f
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2289983
> Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
> Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#68769}
TBR=ulan@chromium.org,dinfuehr@chromium.org
Change-Id: Ifba8218f9104afed3f2d92258296067f8f424062
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:10315
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2290573
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68770}
This changes the safepoint scope to skip LocalHeap that is active
for the current thread to avoid deadlocking.
Bug: v8:10315
Change-Id: I45e80ae66d0dbbe768107aa9cf0603204c644d9f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2289983
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68769}
This is needed for write-barrier and persistent-handle code that does
not otherwise get an instance of LocalHeap
Bug: v8:10315
Change-Id: I480e31f32141510f2f9e678af3449d5841e3156e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2284492
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68720}
- Adds JSVisitor that is used for unified heap marking.
- Adds JSMember as supported reference type that also encapsulates a
write barrier in future. JSMember is a replacement for
TracedReference which can be deprecated with EmbedderHeapTracer once
the library is used to handle unified heap collections.
The dispatch for v8::JSMember on cppgc::Visitor is provided through a
specialization of TraceTrait.
Bug: chromium:1056170
Change-Id: I60d976ae66db3e5fa2e690a21627bdcb8c6871af
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2284488
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68716}
Split off MarkingWorklists and from Marker and introduce MarkerBase.
MarkerBase refers just to interfaces types for passing along visitors.
The concrete Marker provides the impl for these interfaces. Unified
heap marker uses different marking visitors internally but provides an
implementation for the same interface.
Change-Id: Ibc4b2c88e2e69bd303a95da7d167a701934f4a07
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270539
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68676}
Split off MarkingState from MarkingVisitor.
With this CL the marking implementation is moved to "MarkingState"
which is the new bottleneck for marking a single object.
MarkingVisitor merely forwards to MarkingState, which knows how to set
the markbit and add the object to the worklist accordingly. This
allows to have a "UnifiedHeapMarkingVisitor" in future which can
easily reuse Marking to provide C++ marking.
Change-Id: I87ebbe37e8e8cd841e872cae9dc3490e2b55c4dd
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2270172
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68660}
This allows the implementation of different stack scanning mechanisms in
V8 (e.g. conservative scanning) while re-using the stack walking API.
Change-Id: I9b9c3b8ffe5d527ca3f7105776821776b509b187
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2238194
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68457}
This is a reland of e0c1a349ea
The issue was passing SentinelPointer (== +1) through T*.
The fix is disabling cfi unrelated cast diagnostic for the bottlenecks
(Get()). This means that nullptr is treated the same as
kSentinelPointer.
The alternative would be a DCHECK that Get() does not return
kSentinelPointer and adjusting all Member and Persistent logic that
uses Get() to work on void*. This is quite intrusive as it involves
Swap(), heterogeneous assignments, comparisons, etc.
Original change's description:
> cppgc: Properly clear (Weak)Peristent and WeakMember pointers
>
> The CL addresses two issues with (Weak)Persistent and WeakMember:
> 1. (Weak)Persistent pointers are cleared on heap teardown. Before this
> CL the pointers would contain stale values which could lead to UAF.
> 2. WeakPersistent and WeakMember are cleared using a combination of
> internal clearing methods and mutable fields which avoids the use
> of const_cast<>.
>
> Bug: chromium:1056170
> Change-Id: Ibf2b0f0856771b4f6906608cde13a6d43ebf81f3
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2248190
> Reviewed-by: Omer Katz <omerkatz@chromium.org>
> Reviewed-by: Anton Bikineev <bikineev@chromium.org>
> Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#68394}
Bug: chromium:1056170
Change-Id: I3d74b43464c2973df1956f51b1419d755dd9f519
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2250240
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68426}
The CL addresses two issues with (Weak)Persistent and WeakMember:
1. (Weak)Persistent pointers are cleared on heap teardown. Before this
CL the pointers would contain stale values which could lead to UAF.
2. WeakPersistent and WeakMember are cleared using a combination of
internal clearing methods and mutable fields which avoids the use
of const_cast<>.
Bug: chromium:1056170
Change-Id: Ibf2b0f0856771b4f6906608cde13a6d43ebf81f3
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2248190
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68394}
Since ReadOnlySpace pages will soon not be MemoryChunks, change most
uses of MemoryChunk::FromHeapObject and FromAddress to use the
BasicMemoryChunk variants and which use the new MemoryChunk::cast
function that takes a BasicMemoryChunk and DCHECKs !InReadOnlySpace().
To enable this, it also moves into BasicMemoryChunk several MemoryChunk
functions that just require a BasicMemoryChunk.
Bug: v8:10454
Change-Id: I80875b2c2446937ac2c2bc9287d36e71cc050c38
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2243216
Commit-Queue: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68390}
This is a reland of 539f0ed23b
The reland fixes creating TimeDelta from double which requires
saturated_cast<>. Improvements to this constructions are tracked
in v8:10620.
Original change's description:
> cppgc,heap: Implement atomic unified heap GC
>
> Add v8::CppHeap as an implementation of a cppgc heap that
> integrates with V8's existing EmbedderHeapTracer API. The
> current implementation only supports non-incremental marking.
>
> Bug: chromium:1056170
> Change-Id: I4a09eb5ae57f5c7defe35eb3fe346627eb492473
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2245610
> Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
> Reviewed-by: Anton Bikineev <bikineev@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Omer Katz <omerkatz@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#68374}
Bug: chromium:1056170,v8:10620
Change-Id: I39e15790e5cafe24da2a14d0bae6543391ebb536
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2248191
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68387}
This adds the following things:
- age table for 4K regions;
- generational barrier for mixed 4K regions;
- unmarking for major collections;
- young generation flags.
Bug: chromium:1029379
Change-Id: Ief1229f0dac5f90c5f06d3168c8ffb4b7d1f1b53
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2246566
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68379}
Add v8::CppHeap as an implementation of a cppgc heap that
integrates with V8's existing EmbedderHeapTracer API. The
current implementation only supports non-incremental marking.
Bug: chromium:1056170
Change-Id: I4a09eb5ae57f5c7defe35eb3fe346627eb492473
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2245610
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68374}
Since the registration requires calling into the library, there's no
reason to get the heap through a magic getter on API level.
Bug: chromium:1056170
Change-Id: I8d2b1d0fcee8c855908bd26c71a22826c493ed29
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2238568
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68315}
Unified heap support in V8 requires having another (at least internal)
heap that implements a unfied garbage collection strategy. This will
not re-use the already existing cppgc::Heap because there should be no
way in creating such a heap externally or scheduling stand-alone
garbage collections.
In order to have a common token, this CL introduces AllocationHandle
which can be passed to MakeGarbageCollected to allocate C++ objects.
V8 (soon) and the stand-alone heap both have methods to retrieve such
a handle.
This works around a problem with creating diamond class hierarchies
when a base class would be exposed on the public API level.
Fast paths for Blink are still possible because allocation handles can
be cached the same way (e.g. global, or TLS) as a heap can be cached.
Tbr: yangguo@chromium.org
Bug: chromium:1056170
Change-Id: I8e9472a2c24ef82d1178953e8429b1fd8a2344bc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2238027
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68310}
This fixes two issues:
- labs resetting didn't account bytes as beeing freed;
- large object were not accounted.
The CL introduces a single bottleneck for labs resetting in
ObjectAllocator, which is aware of StatsCollector. This way
NormalSpace is treated as a value object and all invariants
are maintained by ObjectAllocator (and Sweeper).
Bug: chromium:1056170
Change-Id: I027cc01fe5028a3dfa81905d7ea53dd12d1c1f20
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2237629
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68286}
Adds allocation-based heap growing strategy that triggers GC based on
some limit. The limit is computed based on previous live memory and a
constant growing factor.
For invoking GC, we support two modes: with and without conservative
stack scanning. Without conservative stack scanning, an invoker makes
sure that we schedule a GC without stack using the existing platform.
Bug: chromium:1056170
Change-Id: I1808aeb5806a6ddd5501b556d6b6b129a85b9cda
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2228887
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68235}
The observers can use ResetAllocatedObjectSize() to e.g. implement a
growing strategy that resets its limit on this call.
Bug: chromium:1056170
Change-Id: Ib9553e00cc530ff89f44e4258c13d47f0b70568e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2228885
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68205}
This ports HeapStatsCollector (former ThreadHeapStatsCollector) from
Blink. The CL only ports accounting of allocated object size which is
needed for a simple growing strategy in a follow up.
HeapStatsCollector is a global dependency for most sub components as
it provides infrastructure for measuring time (through trace scopes)
and space.
The general idea of HeapStatsCollector is to act as sink where all sub
components push time and space information. This information is then
gathered and made available via an event that is implemented as POD.
Time-dependent info is available through regular getters (pull) and
observers (push).
Change-Id: I40b4d76e1a40c56e5df1a7353622318cde730e26
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2225902
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68150}
This fixes the gcov build.
Bug: chromium:1056170
Change-Id: I40ac42b6fce77367c6a366544abf89fc21075cdc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2228510
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Auto-Submit: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68125}
Split platform into a process-global initialization part and per-heap
platform objects.
These platform objects still contain allocators and executors. With
per-heap platforms GetForegroundTaskRunner() returns by definition the
correct runner.
In future, when initialized throuhg V8, an adapter can be used to
translate between the different platforms, avoiding the needed for V8
embedders to provide additional information.
Bug: chromium:1056170
Change-Id: I11bdd15e945687cfbdf38cae4137facb02559e0a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2218030
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68059}
Make sure that any GCs required for off-thread heap merging happen
before any off-thread handle transferring (both transferring using
OffThreadTransferHandle, and the handles created for the string slot
fixups). This is to avoid the marker from walking Handle roots that
point into off-thread pages which the sweeper doesn't see (and can't
clear mark bits on)
Now, the merging and handle creation is atomic as far as the GC is
concerned. The merging is done before handle creation to avoid the
incremental marker from entering off-thread pages, but we ensure that
the raw objects pointers that point into the off-thread pages (which
are used for creating the main-thread handles) stay valid until the
handle creation completes.
Since handle transfer now happens in the middle of publishing, this
patch also moves the OffThreadTransferHandleStorage ownership over to
OffThreadHeap. This requires some header juggling to avoid leaking
OffThreadTransferHandleStorage into the off-thread-isolate header.
Bug: chromium:1086478, chromium:1011762
Change-Id: Id5e7622d6b5520400a4872c5f6ad396c74b30ca6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2218058
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68043}
This reverts commit a35d0e8cb5.
The original CL is likely not a culprit for the infra failures.
Bug: chromium:1056170
Change-Id: I8fa85db8a737fb01328021782f0c43626fa52b0d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2215826
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67977}
This moves concurrent and incremental sweeping from Blink. This also
adds TestPlatform that makes it easier to test concurrent and
incremental sweeping.
Drive-by: fix unmarking of large pages.
Bug: chromium:1056170
Change-Id: Ifd50ff67b9df17ff117a5f4d4eb5a2937d3023be
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2207132
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67969}
This allows embedding objects in each other and recursively trace
through them.
Bug: chromium:1056170
Change-Id: I4e4ae4c1669109c01003cb6b69797cf271a74033
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2198977
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67841}
Splits out all of PagedSpace and subclasses into paged-spaces.h. Also
moves CodeObjectRegistry to code-object-registry.h.
Bug: v8:10473, v8:10506
Change-Id: I35fab1e545e958eb32f3e39a5e2ce8fb087c2a53
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2201763
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dan Elphick <delphick@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67811}
This reverts commit 580917d252.
Reason for revert: fix in patchset 2
Original change's description:
> Revert "cppgc: Stack scanning using ObjectStartBitmap"
>
> This reverts commit d3a72e3c2a.
>
> Reason for revert: MSAN failures (https://ci.chromium.org/p/v8/builders/ci/V8%20Linux%20-%20arm64%20-%20sim%20-%20MSAN/32360)
>
> Original change's description:
> > cppgc: Stack scanning using ObjectStartBitmap
> >
> > This CL implements stack scanning for cppgc.
> > Given a value on the stack, the MarkingVisitor uses
> > PageBackend::Lookup to checks whether that address is on
> > the heap. If it is, BasePage::TryObjectHeaderFromInnerAddress
> > (introduced in this CL) is used to get the relevant object
> > header. Note that random addresses on the heap might point to
> > free memory, object-start-bitmap, etc.
> >
> > If a valid object header is found:
> > * If the object is not in construction, the GCInfoIndex is used
> > the get the relevant Trace method and the object is traced.
> > * Otherwise, the object is conservatively scanned - i.e. the
> > payload of the object is iterated word by word and each word is
> > treated as a possible pointer.
> >
> > Only addresses pointing to the payload on non-free objects are
> > traced.
> >
> > BasePage::TryObjectHeaderFromInnerAddress assumes on LAB on the
> > relevant space, thus all LABs are reset before scanning the stack.
> >
> > Bug: chromium:1056170
> > Change-Id: I172850f6f1bbb6f0efca8e44ad8fdfe222977b9f
> > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2190426
> > Commit-Queue: Omer Katz <omerkatz@chromium.org>
> > Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> > Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> > Reviewed-by: Anton Bikineev <bikineev@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#67795}
>
> TBR=ulan@chromium.org,mlippautz@chromium.org,bikineev@chromium.org,omerkatz@chromium.org
>
> Change-Id: I3caef6f9f55911fd1a86e895c3495d1b98b1eac2
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: chromium:1056170
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2201136
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Commit-Queue: Leszek Swirski <leszeks@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#67796}
TBR=ulan@chromium.org,mlippautz@chromium.org,leszeks@chromium.org,bikineev@chromium.org,omerkatz@chromium.org
# Not skipping CQ checks because this is a reland.
Bug: chromium:1056170
Change-Id: If7ea4fe5cb794c07544d5545f5d6548e3375d3ae
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2201137
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67803}
The existing non-builtin implementation is returning wrong results.
For example, given the value 63 as a uint8_t it returns 38 (should be 6).
The new implementation follows the naive algorithm presented in figure 5-1
in Hacker's Delight section 5-1.
Note that the algorithm in the book is designed for 32 bit numbers, so we
extended it to support 64 bit as well.
Bug: chromium:1056170
Change-Id: I8fed9c449f80b01b8cc93d339529c0e1e0863fc0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2199345
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67801}
This reverts commit d3a72e3c2a.
Reason for revert: MSAN failures (https://ci.chromium.org/p/v8/builders/ci/V8%20Linux%20-%20arm64%20-%20sim%20-%20MSAN/32360)
Original change's description:
> cppgc: Stack scanning using ObjectStartBitmap
>
> This CL implements stack scanning for cppgc.
> Given a value on the stack, the MarkingVisitor uses
> PageBackend::Lookup to checks whether that address is on
> the heap. If it is, BasePage::TryObjectHeaderFromInnerAddress
> (introduced in this CL) is used to get the relevant object
> header. Note that random addresses on the heap might point to
> free memory, object-start-bitmap, etc.
>
> If a valid object header is found:
> * If the object is not in construction, the GCInfoIndex is used
> the get the relevant Trace method and the object is traced.
> * Otherwise, the object is conservatively scanned - i.e. the
> payload of the object is iterated word by word and each word is
> treated as a possible pointer.
>
> Only addresses pointing to the payload on non-free objects are
> traced.
>
> BasePage::TryObjectHeaderFromInnerAddress assumes on LAB on the
> relevant space, thus all LABs are reset before scanning the stack.
>
> Bug: chromium:1056170
> Change-Id: I172850f6f1bbb6f0efca8e44ad8fdfe222977b9f
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2190426
> Commit-Queue: Omer Katz <omerkatz@chromium.org>
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Anton Bikineev <bikineev@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#67795}
TBR=ulan@chromium.org,mlippautz@chromium.org,bikineev@chromium.org,omerkatz@chromium.org
Change-Id: I3caef6f9f55911fd1a86e895c3495d1b98b1eac2
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2201136
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67796}
This CL implements stack scanning for cppgc.
Given a value on the stack, the MarkingVisitor uses
PageBackend::Lookup to checks whether that address is on
the heap. If it is, BasePage::TryObjectHeaderFromInnerAddress
(introduced in this CL) is used to get the relevant object
header. Note that random addresses on the heap might point to
free memory, object-start-bitmap, etc.
If a valid object header is found:
* If the object is not in construction, the GCInfoIndex is used
the get the relevant Trace method and the object is traced.
* Otherwise, the object is conservatively scanned - i.e. the
payload of the object is iterated word by word and each word is
treated as a possible pointer.
Only addresses pointing to the payload on non-free objects are
traced.
BasePage::TryObjectHeaderFromInnerAddress assumes on LAB on the
relevant space, thus all LABs are reset before scanning the stack.
Bug: chromium:1056170
Change-Id: I172850f6f1bbb6f0efca8e44ad8fdfe222977b9f
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2190426
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67795}
This is needed to trace objects found durinbg stack scanning.
Bug: chromium:1056170
Change-Id: I1280d98f2fe69281c514b3a7d4a57f909a2eed96
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2190425
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67788}
Add const Page iterators to Spaces, and add whichever const methods are
necessary for this to work. This and a couple more const methods allows
us to make Heap::Contains const.
Change-Id: I1b63a10575ccdb8a3979aef4fa63a97b288ff836
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2198975
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67776}
This CL adds 2 new values to the EmbedderStackState enum with more
explicit names. The old values are updated as aliases to the new
values and marked as soon to be deprecated. This CL also moves the
enum to v8-platform.h so that it can be reused by cppgc.
Depracating individual values in an enum is supported by GCC only
since version 6. Thus new macros were needed for the deprecation
(which delegate to the existing macros when supported). GCC versions
older than 6 are still used by the CQ bots.
Bug: chromium:1056170
Change-Id: Id1ea73edfbbae282b0d8a3bb103dbbbf8ebd417e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2188971
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67744}
The visitor was removing pages while at the same time iterating them on
NormalPagedSpace.
Removing all pages at once is safe and should also be faster.
Bug: chromium:1056170
Change-Id: I56eedf6f09498f126cb09238e01962b48e75b657
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2190427
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67687}
Background threads can now start incremental marking when necessary. In
contrast to the main thread they always need to schedule a job and can't
start incremental marking right away. Background threads also use a
simpler heuristic for deciding whether to start incremental marking.
Bug: v8:10315
Change-Id: I2b94e8273c8be860157fe9670797048ed1c5c3da
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2184149
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Dominik Inführ <dinfuehr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67675}
This patch provides infrastructure to pin object types to specific
spaces. This allows embedders to create mutual exclusive arenas for
certain (base) types. In future, this will also be used to provide
sliding-window compaction on certain custom spaces.
We mainly preserve the existing infrastructure with the difference
that spaces are now slightly more dynamic than in Blink as they are
kept in a vector instead of a fixed-size array.
The mechanism differs from Blink in that it does not allow the user
object to call allocation methods directly but instead provides a
trait that can be overridden to specify a custom space.
The patch preserves templatization for objects that do not go into
custom spaces to safe a branch in the allocation hot path.
Change-Id: I08aa6932348e2d6258e19c4a32d189865f459f02
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2187611
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67666}
This CL introduces:
- Worklist
- MarkingHandler to manage gc marking phase
- Integration into CollectGarbage for atomic pause GC
- MarkingVisitor for main thread marking
Still missing from this CL:
- Proper handling for stack scanning
- Handling of previously not fully constructed objects
Bug: chromium:1056170
Change-Id: I70ac8534dfb898777cf3a06e3119cac8072174fd
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2170526
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67642}
Returned LABs can be of size less than sizeof(FreeListEntry).
Bug: chromium:1056170
Change-Id: Ib4094701472ce7cb5ee20b9fe632651570832dc9
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2183051
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67635}
Destructors are allowed to allocate without triggering recursive
garbage collections.
This changes NoGCScope to provide a soft-bailout for garbage
collections to avoid introducing yet another scope.
Bug: chromium:1056170
Change-Id: I0fe51a21977ae954221b6b64b2f6e938ff6d3264
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2185131
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67632}
Adds a public method that embedders can use to trigger garbage
collections. Such garbage collections are always required to have a
source and reason specifying which components calls it why.
Change-Id: I6ae983f99227febc1b7f0dd15c191d5b1eaaf3f3
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2181332
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67631}
Move rewriting, scope analysis, and internalization, to be unconditional
operations done after parsing rather than a separate compile phase. This
removes some of the complexity about rememberering when to call
Compiler::Analyze, and makes these paths a bit more uniform.
Also, forbid allocating any more AST strings after AstValueFactory
internalization, by nulling out the Zone. Add an InternalizePartial
method which doesn't null out the zone for those cases where we do want
to be able to allocate after internalizing (e.g. internalization before
scope analysis).
Change-Id: Id444246d8362a1d169baf664fc37657d9576fd96
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2182458
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67608}
This ports sweeper logic from Blink into a separate entity - Sweeper.
Concurrent sweeping is in a followup.
Bug: chromium:1056170
Change-Id: I41196225f0d882cb0ab5190d23e297ee2498df6b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2167858
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67581}
This adds PostConstructionCallbackTrait which can be used to get a
callback that is executed right after an object instance is created.
This can be useful for hooks that require to be able to call into
virtual methods.
Bug: chromium:1074061
Change-Id: Idd5ef677fed291bcba81b9a47f2932c9bb5832b4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2179385
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67557}
Also makes memory-chunk.h accessible from outside heap which allows
removal of some heap-inl.h includes.
Bug: v8:10473, v8:10496
Change-Id: Iec4fc5ce8ad201f6ee5fd924cc3cd935324429fc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2172088
Commit-Queue: Jakob Gruber <jgruber@chromium.org>
Auto-Submit: Dan Elphick <delphick@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67551}
The worklist in this CL is a merge of the worklists of Oilpan and V8. This implementation supports both use cases and should serve as the shared worklist once we start merging the codebase.
Bug: chromium:1056170
Change-Id: I4ecdb475f3900c33eced9249efa112a69c1b2707
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2170828
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67459}
HeapVisitor is a simple visitor implementing both visitation and accept
interfaces. HeapVisitor is aimed to replace functions that traverse the
heap in Blink: PrepareForSweep, Sweep, FinalizeSweep, MakeConsistentForGC,
ObjectPayloadSize, VerifyObjectStartBitmap, CollectStatistics, etc.
This CL also introduces a Heap::ObjectPayloadSize function.
Bug: chromium:1056170
Change-Id: I85a2b8b572486bea29704fc436b002d8f7405201
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2165733
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67458}
Creates a new large-spaces.h and large-spaces.cc to contain
LargeObjectSpace and subclasses.
Bug: v8:10473
Change-Id: Ifdb4eac9df5c8213f992d549e04b612b62f6df0b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2170826
Commit-Queue: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67456}
When creating a filler, we pass through whether we need to clear slots
in the old to new remembered set.
This patch adds a verification check that, when we claim we don't need
to clear slots, checks that no slots are set in the remembered set for
the range of the filler. Effectively, this is a range counterpart to
VerifyClearedSlot.
Change-Id: Id994c56d941988cc282463304bc7307a51943e99
Bug: chromium:1075999
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2139572
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67453}
base::List is only used inside of heap and has some pretty strange
semantics that don't lend themselves to it being a general purpose data
structure so this moves it to heap where it can be safe isolated.
Bug: v8:10454
Change-Id: I7921c22286276432956005c72143b22b0364fc93
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2170029
Auto-Submit: Dan Elphick <delphick@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Clemens Backes <clemensb@chromium.org>
Commit-Queue: Clemens Backes <clemensb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67448}
This adds the following:
1) Heap object structure classes: RawHeap, BaseArena and BasePage.
- freelist
- linear allocation block
2) ObjectAllocator, a class responsible for object (and page) allocation.
The design doc with UML design: https://bit.ly/2VVTcqc
User defined arenas are followup.
Bug: chromium:1056170
Change-Id: I69a82974bd08e3cf3da90041b1628297cc890891
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2167392
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67425}
Introduce an OffThreadTransferHandle (and OffThreadTransferMaybeHandle),
which points to a piece of storage known to (and owned by) the
OffThreadIsolate. On Publish, the OffThreadIsolate converts this storage
stub from a raw pointer to an off-thread object into a main-thread
Handle.
This allows users of an OffThreadIsolate to not have to worry (as much)
about the dance of saving raw object pointers before off-thread finishes
and converting those to Handles before it off-thread isolate is
published.
Bug: chromium:1011762
Change-Id: I7ceb4ed85e770bd6e1867649188597bbcaedb32d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2161066
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67337}
