Currently the memory reducer is activated only after the first mark-
compact GC, which triggered after the old generation reaches 8 MB.
That threshold is too large for mobile. This patch adds a heuristic
to activate the memory reducer if the old generation expands by more
than 1 MB after the bootstrap.
Change-Id: Ic38bc6e2fe8887677f764246c45e38d237e49a94
Reviewed-on: https://chromium-review.googlesource.com/c/1425898
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58982}
Now a descriptor array tracks the number of descriptors that were
already marked. The marking visitor of a map only marks the subset
of the descriptors that it needs and that are not already marked.
If a descriptor array is shared between M maps and has N descriptos,
then the number of marking operations is reduced from O(M*N) to O(N).
This patch also adds a marking barrier for descriptors.
The marked descriptor counter in a descriptor array is not cleared
after mark-compact GC. Instead, it embeds two bits from the global
mark-compact epoch counter and is considered 0 if the bits do not match
the current value of the global epoch counter.
Bug: v8:8486
Change-Id: I2a7822a6833f3143e1d351e5e4819c2ef2c07fb0
Reviewed-on: https://chromium-review.googlesource.com/c/1382746
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58372}
This reverts commit 5b434929a3.
Changes after the original CL:
- Right-trimming registers the array as an object with invalidated
slots.
- Left-trimming moves the array start in the invalidated slots map.
Original change's description:
> Fix invalidation of old-to-old slots after object trimming.
>
> A recorded old-to-old slot may be overwritten with a pointer to a new
> space object. If the object containing the slot is trimmed later on,
> then the mark-compactor may crash on a stale pointer to new space.
>
> This patch ensures that:
> 1) On trimming of an object we add it to the invalidated_slots sets.
> 2) The InvalidatedSlotsFilter::IsValid returns false for slots outside
> the invalidated object unless the page was already swept.
>
> Array left-trimming is handled as a special case because object start
> moves and cannot be added to the invalidated set. Instead, we clear
> the freed memory so that the recorded slots contain Smi values.
>
> Bug: chromium:870226,chromium:816426
> Change-Id: Iffc05a58fcf52ece45fdb085b5d1fd4b3acb5d53
> Reviewed-on: https://chromium-review.googlesource.com/1163784
> Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Hannes Payer <hpayer@chromium.org>
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#54953}
Change-Id: I1f1080f680196c581f62aef8d3a00a595f9bb9b0
Reviewed-on: https://chromium-review.googlesource.com/1165555
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55066}
This reverts commit 719d23c032.
Reason for revert: TSAN failures
Original change's description:
> Fix invalidation of old-to-old slots after object trimming.
>
> A recorded old-to-old slot may be overwritten with a pointer to a new
> space object. If the object containing the slot is trimmed later on,
> then the mark-compactor may crash on a stale pointer to new space.
>
> This patch ensures that:
> 1) On trimming of an object we add it to the invalidated_slots sets.
> 2) The InvalidatedSlotsFilter::IsValid returns false for slots outside
> the invalidated object unless the page was already swept.
>
> Array left-trimming is handled as a special case because object start
> moves and cannot be added to the invalidated set. Instead, we clear
> the freed memory so that the recorded slots contain Smi values.
>
> Bug: chromium:870226,chromium:816426
> Change-Id: Iffc05a58fcf52ece45fdb085b5d1fd4b3acb5d53
> Reviewed-on: https://chromium-review.googlesource.com/1163784
> Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
> Reviewed-by: Hannes Payer <hpayer@chromium.org>
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#54953}
TBR=ulan@chromium.org,hpayer@chromium.org,mlippautz@chromium.org
Change-Id: I2e1ff83c2db7902488951a8f597d38133aeb3b04
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:870226, chromium:816426
Reviewed-on: https://chromium-review.googlesource.com/1165862
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54954}
A recorded old-to-old slot may be overwritten with a pointer to a new
space object. If the object containing the slot is trimmed later on,
then the mark-compactor may crash on a stale pointer to new space.
This patch ensures that:
1) On trimming of an object we add it to the invalidated_slots sets.
2) The InvalidatedSlotsFilter::IsValid returns false for slots outside
the invalidated object unless the page was already swept.
Array left-trimming is handled as a special case because object start
moves and cannot be added to the invalidated set. Instead, we clear
the freed memory so that the recorded slots contain Smi values.
Bug: chromium:870226,chromium:816426
Change-Id: Iffc05a58fcf52ece45fdb085b5d1fd4b3acb5d53
Reviewed-on: https://chromium-review.googlesource.com/1163784
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54953}
If it points to a new space object which doesn't get scavenged, we need to drop
the reference.
BUG=v8:7308, v8:7768
Change-Id: I4485a7abcac3a26781811cc9bf134fd80e5f35b5
Reviewed-on: https://chromium-review.googlesource.com/1069127
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53287}
ToDirectStringAssembler::PointerToData returns a raw pointer, which
is invalidated when GC moves the original string and hence must not
be accessed after any allocations. This fixes the bug introduced in
b4ebbc57a9 / r53260.
Bug: chromium:845060
Tbr: jgruber@chromium.org
Change-Id: I248d0dd2a275bf9308269b3f65d00c4c4c3d4292
Reviewed-on: https://chromium-review.googlesource.com/1068213
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#53272}
There is no good reason to have the meat of most objects' initialization
logic in heap.cc, all wrapped by the CALL_HEAP_FUNCTION macro. Instead,
this CL changes the protocol between Heap and Factory to be AllocateRaw,
and all object initialization work after (possibly retried) successful
raw allocation happens in the Factory.
This saves about 20KB of binary size on x64.
Original review: https://chromium-review.googlesource.com/c/v8/v8/+/959533
Originally landed as r52416 / f9a2e24bbc
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Id072cbe6b3ed30afd339c7e502844b99ca12a647
Reviewed-on: https://chromium-review.googlesource.com/1000540
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52492}
This reverts commit f9a2e24bbc.
Reason for revert: gc stress failures not all fixed by follow up.
Original change's description:
> [cleanup] Refactor the Factory
>
> There is no good reason to have the meat of most objects' initialization
> logic in heap.cc, all wrapped by the CALL_HEAP_FUNCTION macro. Instead,
> this CL changes the protocol between Heap and Factory to be AllocateRaw,
> and all object initialization work after (possibly retried) successful
> raw allocation happens in the Factory.
>
> This saves about 20KB of binary size on x64.
>
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: Icbfdc4266d7be8b48d2fe085f03411743dc6a0ca
> Reviewed-on: https://chromium-review.googlesource.com/959533
> Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
> Reviewed-by: Hannes Payer <hpayer@chromium.org>
> Reviewed-by: Yang Guo <yangguo@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#52416}
TBR=jkummerow@chromium.org,yangguo@chromium.org,mstarzinger@chromium.org,hpayer@chromium.org
Change-Id: Idbbc53478742f3e9525eee83342afc6aedae122f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/999414
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52420}
There is no good reason to have the meat of most objects' initialization
logic in heap.cc, all wrapped by the CALL_HEAP_FUNCTION macro. Instead,
this CL changes the protocol between Heap and Factory to be AllocateRaw,
and all object initialization work after (possibly retried) successful
raw allocation happens in the Factory.
This saves about 20KB of binary size on x64.
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Icbfdc4266d7be8b48d2fe085f03411743dc6a0ca
Reviewed-on: https://chromium-review.googlesource.com/959533
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52416}
Sweeping a page while currently scavenging it is broken as the scavenger
might override the slot it is currently processing.
Bug: chromium:779503
Change-Id: I224a144b84e97a956bf10ba018132c2713e8f78d
Reviewed-on: https://chromium-review.googlesource.com/752081
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49092}
Both the top_ pointer and the top_on_previous_step_ pointer can be one
byte beyond the current page. Page::FromAddress call should take that
into account.
Bug: chromium:777177
Change-Id: I9cbb5bc6eab932afc6d0c915fd70a9a7b20ba62c
Reviewed-on: https://chromium-review.googlesource.com/738204
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48962}
On advancing the iterator we need to reset the current object,
so that it can be lazily reloaded later on.
TBR=mlippautz@chromium.org
Bug: chromium:694255
Change-Id: If7ddd8670df9d11837f491503312919b55b451fe
Reviewed-on: https://chromium-review.googlesource.com/600687
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47142}
This reverts commit d4a742fdf1.
Reason for revert: gc-stress failures
Original change's description:
> Reland "[heap] Add mechanism for tracking invalidated slots per memory chunk."
>
> This reverts commit c59b81d7b8.
>
> Original change's description:
> > [heap] Add mechanism for tracking invalidated slots per memory chunk.
>
> > For correct slots recording in concurrent marker, we need to resolve
> > the race that happens when
> > 1) the mutator is invalidating slots for double unboxing or string
> > conversions
> > 2) and the concurrent marker is recording these slots.
>
> > This patch adds a data-structure for tracking the invalidated objects.
> > Thus we can allow the concurrent marker to record slots without
> > worrying about clearing them. During old-to-old pointer updating phase
> > we re-check all slots that belong to the invalidated objects.
>
> BUG=chromium:694255
>
> Change-Id: Idf8927d162377a7bbdff34f81a87e52db27d6a9f
> Reviewed-on: https://chromium-review.googlesource.com/596868
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#47068}
TBR=ulan@chromium.org,mlippautz@chromium.org
Change-Id: I81c6059a092cc5834acd799c51fd30dc0ecf5b27
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:694255
Reviewed-on: https://chromium-review.googlesource.com/597787
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47078}
This reverts commit c59b81d7b8.
Original change's description:
> [heap] Add mechanism for tracking invalidated slots per memory chunk.
> For correct slots recording in concurrent marker, we need to resolve
> the race that happens when
> 1) the mutator is invalidating slots for double unboxing or string
> conversions
> 2) and the concurrent marker is recording these slots.
> This patch adds a data-structure for tracking the invalidated objects.
> Thus we can allow the concurrent marker to record slots without
> worrying about clearing them. During old-to-old pointer updating phase
> we re-check all slots that belong to the invalidated objects.
BUG=chromium:694255
Change-Id: Idf8927d162377a7bbdff34f81a87e52db27d6a9f
Reviewed-on: https://chromium-review.googlesource.com/596868
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47068}
This reverts commit 7a5a777c97.
Reason for revert: crashing in test-api
Original change's description:
> [heap] Add mechanism for tracking invalidated slots per memory chunk.
>
> For correct slots recording in concurrent marker, we need to resolve
> the race that happens when
> 1) the mutator is invalidating slots for double unboxing or string
> conversions
> 2) and the concurrent marker is recording these slots.
>
> This patch adds a data-structure for tracking the invalidated objects.
> Thus we can allow the concurrent marker to record slots without
> worrying about clearing them. During old-to-old pointer updating phase
> we re-check all slots that belong to the invalidated objects.
>
> BUG=chromium:694255
>
> Change-Id: Ifc3d82918cd3b96e5a5fb7125691626a56f4ab83
> Reviewed-on: https://chromium-review.googlesource.com/591810
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#47049}
TBR=ulan@chromium.org,mlippautz@chromium.org
Change-Id: I7f4f8e8cb027b921a82e9c0a0623536af02581fb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:694255
Reviewed-on: https://chromium-review.googlesource.com/595994
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47052}
For correct slots recording in concurrent marker, we need to resolve
the race that happens when
1) the mutator is invalidating slots for double unboxing or string
conversions
2) and the concurrent marker is recording these slots.
This patch adds a data-structure for tracking the invalidated objects.
Thus we can allow the concurrent marker to record slots without
worrying about clearing them. During old-to-old pointer updating phase
we re-check all slots that belong to the invalidated objects.
BUG=chromium:694255
Change-Id: Ifc3d82918cd3b96e5a5fb7125691626a56f4ab83
Reviewed-on: https://chromium-review.googlesource.com/591810
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47049}
Background: the first page of each space is implicitly immovable.
Recently, our builtin code objects have reached a size at which we
fill up the first page of code space during initialization. Once
that occurs, newly requested allocations of immovable code are
allocated in a large object space page of 512K.
This CL mitigates these effects by simply marking pages as immovable
during snapshot creation instead of going into LO space.
On snapshot builds, this should just work: deserialized pages are
trimmed and marked immovable when deserialization finishes.
However, non-snapshot builds and allocations of immovable CEntryStub
code at runtime are still affected.
BUG=v8:5831
Review-Url: https://codereview.chromium.org/2635973002
Cr-Commit-Position: refs/heads/master@{#42411}
come from the runtime.
This patch fixes an issue of heap growing to max capacity when incremental
marking is finished but cannot finalize due to GC stack guard not triggering.
It can happen if all allocations come from the runtime, for example,
from JSON parser or compiler.
Now before expanding the heap we check if we are above the allocation limit
and the incremental marking needs to be finalized. If so we do not expand
the heap and force GC, which will finalize the incremental marking.
The check is performed for paged spaces and large-object space.
BUG=chromium:670675
Review-Url: https://codereview.chromium.org/2552613004
Cr-Commit-Position: refs/heads/master@{#41524}
Reason for revert:
OOMs in nosnap debug:
https://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20nosnap%20-%20debug/builds/9572
Original issue's description:
> [heap] New heuristics for starting of incremental marking.
>
> The motivation for this patch is to move more marking work to tasks.
> This is done by postponing the start of incremental marking until
> a marking task is running.
>
> This patch introduces a soft and a hard limits for incremental marking.
> When the soft limit is reached, the marking task is scheduled.
> If the hard limit is reached before the task is running, then
> incremental marking is started without waiting for the task.
>
> BUG=chromium:616434
> LOG=NO
>
> Committed: https://crrev.com/55683ddd2a32e0dfb8df66271fbf53e3618cce9d
> Cr-Commit-Position: refs/heads/master@{#39831}
TBR=hpayer@chromium.org,ulan@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:616434
Review-Url: https://codereview.chromium.org/2375983002
Cr-Commit-Position: refs/heads/master@{#39833}
The motivation for this patch is to move more marking work to tasks.
This is done by postponing the start of incremental marking until
a marking task is running.
This patch introduces a soft and a hard limits for incremental marking.
When the soft limit is reached, the marking task is scheduled.
If the hard limit is reached before the task is running, then
incremental marking is started without waiting for the task.
BUG=chromium:616434
LOG=NO
Review-Url: https://codereview.chromium.org/2364923002
Cr-Commit-Position: refs/heads/master@{#39831}
This reverts commit 9146bc5e20.
This contains a fix for the following crash:
1. We record slots for a fixed array.
2. We trim the fixed array, so that some recorded slots are now in free space.
3. During mark-compact we sweep the page with the fixed array. Now free list items contain memory with recorded slots.
4. We evacuate a byte array using the new free list items.
5. We iterate slots that are now inside the byte array and crash.
BUG=chromium:589413,chromium:578883
LOG=NO
Review URL: https://codereview.chromium.org/1735523002
Cr-Commit-Position: refs/heads/master@{#34302}
If sweeping is in progress then we need to filter out slots in free space after
array trimming, because the sweeper will add the free space into free list.
This CL also fixes a bug in SlotSet::RemoveRange.
BUG=chromium:587004
LOG=NO
TBR=hpayer@chromium.org
Review URL: https://codereview.chromium.org/1701963003
Cr-Commit-Position: refs/heads/master@{#34071}
Reason for revert:
Bisection results show that this was not the culprit.
Original issue's description:
> Revert of [heap] Simplify distribution of remaining memory during sweeping & compaction (patchset #2 id:80001 of https://codereview.chromium.org/1653973003/ )
>
> Reason for revert:
> Very likely blocking roll: https://codereview.chromium.org/1652413003/
>
> Original issue's description:
> > [heap] Simplify distribution of remaining memory during sweeping & compaction
> >
> > BUG=chromium:524425
> > LOG=N
> >
> > Committed: https://crrev.com/f72923526ccaa8faef5c977267b0c074c4a44dfa
> > Cr-Commit-Position: refs/heads/master@{#33668}
>
> TBR=hpayer@chromium.org,mlippautz@chromium.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:524425
>
> Committed: https://crrev.com/a9441b0e7a2a56c2047482a3cc66e3ca2255444b
> Cr-Commit-Position: refs/heads/master@{#33695}
TBR=hpayer@chromium.org,mlippautz@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:524425
Review URL: https://codereview.chromium.org/1663013002
Cr-Commit-Position: refs/heads/master@{#33696}
Tests for
* aborting a full page.
* partially aborting a page.
* partially aborting a page with pointers between aborted pages.
* partially aborting a page with store buffer entries.
Also introduces force_oom() which prohibits a old space to
expand
BUG=chromium:524425
LOG=N
CQ_EXTRA_TRYBOTS=tryserver.v8:v8_linux_nosnap_rel,v8_linux_nosnap_dbg,v8_win_nosnap_shared_rel,v8_win_nosnap_shared_compile_rel
Review URL: https://codereview.chromium.org/1518803005
Cr-Commit-Position: refs/heads/master@{#32899}
Reason for revert:
Failing on Win 32bit nosnap:
https://chromegw.corp.google.com/i/client.v8/builders/V8%20Win32%20-%20nosnap%20-%20shared/builds/10602
Original issue's description:
> [cctest] Add tests for aborting compaction of pages
>
> Tests for
> * aborting a full page.
> * partially aborting a page.
> * partially aborting a page with pointers between aborted pages.
> * partially aborting a page with store buffer entries.
>
> Also introduces force_oom() which prohibits a old space to
> expand
>
> BUG=chromium:524425
> LOG=N
TBR=ulan@chromium.org,hpayer@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:524425
Review URL: https://codereview.chromium.org/1514603008
Cr-Commit-Position: refs/heads/master@{#32785}
Tests for
* aborting a full page.
* partially aborting a page.
* partially aborting a page with pointers between aborted pages.
* partially aborting a page with store buffer entries.
Also introduces ShouldForceOOM() which prohibits a PagedSpace from expanding.
Compaction spaces refer to the corresponding actual space.
BUG=chromium:524425
LOG=N
Review URL: https://codereview.chromium.org/1511933002
Cr-Commit-Position: refs/heads/master@{#32783}
