When a RelocatingCharacterStream is Seeked, it's buffer_pos_ could be set a non-zero value.
However, UpdateBufferPointers was assuming the position was zero to relocate the buffer_start_
and buffer_end_, which would lead to the stream becoming misaligned. Fix this and add a
unittest and the clusterfuzz script which highlighted the issue.
BUG=chromium:991133
Change-Id: I20dd510b3dcc5df6df058b7e06d2c8a838aef855
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1751782
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#63190}
This CL was generated by an automatic clang AST rewriter using this
matcher expression:
callExpr(
callee(
cxxMethodDecl(
hasName("operator->"),
ofClass(isSameOrDerivedFrom("v8::internal::Object"))
)
),
argumentCountIs(1)
)
The "->" at the expression location was then rewritten to ".".
R=jkummerow@chromium.orgTBR=mstarzinger@chromium.org,verwaest@chromium.org,yangguo@chromium.org
Bug: v8:9183, v8:3770
No-Try: true
No-Tree-Checks: true
Change-Id: I0a7ecabdeafe51d0cf427f5280af0c7cab96869e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1624209
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Yang Guo <yangguo@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61764}
Our {Vector} template provides both {start} and {begin} methods. They
return exactly the same value. Since the {begin} method is needed for
iteration, and is also what standard containers provide, this CL
switches all uses of the {start} method to use {begin} instead.
Patchset 1 was auto-generated by using this clang AST matcher:
callExpr(
callee(
cxxMethodDecl(
hasName("start"),
ofClass(hasName("v8::internal::Vector")))
),
argumentCountIs(0))
Patchset 2 was created by running clang-format. Patchset 3 then
removes the now unused {Vector::start} method.
R=jkummerow@chromium.orgTBR=mstarzinger@chromium.org,yangguo@chromium.org,verwaest@chromium.org
Bug: v8:9183
Change-Id: Id9f01c92870872556e2bb3f6d5667463b0e3e5c6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1587381
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#61081}
Not even when copying 0 bytes. Same for memmove and memcmp.
Bug: v8:3770
Change-Id: I3ed45a4572467ec7a9fc697ac28c004aa9b8b274
Reviewed-on: https://chromium-review.googlesource.com/c/1436217
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59101}
Directly access the std::vector buffer backing store to write bytes. By
reserving enough space upfront we can avoid many superflous bounds checks
that were previously necessary when using push_back.
Change-Id: I9b2fb467809e40743b0d9409c9cccb0c6f36f8c1
Reviewed-on: https://chromium-review.googlesource.com/c/1425910
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59036}
Many values stored in the preparse data for the skippable functions
fit in one byte most of the time. The varint encoding uses a single
continue bit per byte to tell whether there is a following byte.
Change-Id: Ia0a622ba42a338fc91eea1e0c1a72d2582d9f867
Reviewed-on: https://chromium-review.googlesource.com/c/1400842
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58972}
By using a shared byte buffer on the preparser we can drastically
reduce the number of ZoneChunkLists.
Each PreparseDataBuilder now explicitly keeps track of all inner
builders/functions and writes out the data in consecutive order.
Change-Id: I0aada118d869b150108c1f633d9960474ad2f9a1
Reviewed-on: https://chromium-review.googlesource.com/c/1411600
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58926}
This is a reland of e2d44ede95
Original change's description:
> [parser] Inline byte scope data into PreparseData object
>
> Each PreparseData object had at least one pointer to a PodArray for its
> serialized scope data. These objects usually have only tens of bytes of
> payload. By inlining the byte data we save 3 words per PreparseData object.
> This optimization saves 140KB of data on cnn.com.
>
>
> - Store data_length and inner_length as int32 saving a words on 64bit
> - Inline store byte data into PreparseData
> - OnHeapConsumedPreparseData directly uses the PreparseData object
> - get_inner, set_inner no longer allow Null sentinels
>
> Change-Id: I1f62154d05ea2f98a6574efa738b32a8a84319d5
> Reviewed-on: https://chromium-review.googlesource.com/c/1406673
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Commit-Queue: Camillo Bruni <cbruni@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58751}
Change-Id: I1f0a22c641d0d67f435b01c82daf8da7f144bff4
Reviewed-on: https://chromium-review.googlesource.com/c/1407066
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58785}
This reverts commit e2d44ede95.
Reason for revert: Breaks GC stress tests - https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Linux64%20GC%20Stress%20-%20custom%20snapshot/23527
Original change's description:
> [parser] Inline byte scope data into PreparseData object
>
> Each PreparseData object had at least one pointer to a PodArray for its
> serialized scope data. These objects usually have only tens of bytes of
> payload. By inlining the byte data we save 3 words per PreparseData object.
> This optimization saves 140KB of data on cnn.com.
>
>
> - Store data_length and inner_length as int32 saving a words on 64bit
> - Inline store byte data into PreparseData
> - OnHeapConsumedPreparseData directly uses the PreparseData object
> - get_inner, set_inner no longer allow Null sentinels
>
> Change-Id: I1f62154d05ea2f98a6574efa738b32a8a84319d5
> Reviewed-on: https://chromium-review.googlesource.com/c/1406673
> Reviewed-by: Leszek Swirski <leszeks@chromium.org>
> Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
> Commit-Queue: Camillo Bruni <cbruni@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#58751}
TBR=ulan@chromium.org,cbruni@chromium.org,leszeks@chromium.org
Change-Id: I39d92ee7bd2864e1b0c3a8fed4a11b68b3e14d58
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/1407073
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58753}
Each PreparseData object had at least one pointer to a PodArray for its
serialized scope data. These objects usually have only tens of bytes of
payload. By inlining the byte data we save 3 words per PreparseData object.
This optimization saves 140KB of data on cnn.com.
- Store data_length and inner_length as int32 saving a words on 64bit
- Inline store byte data into PreparseData
- OnHeapConsumedPreparseData directly uses the PreparseData object
- get_inner, set_inner no longer allow Null sentinels
Change-Id: I1f62154d05ea2f98a6574efa738b32a8a84319d5
Reviewed-on: https://chromium-review.googlesource.com/c/1406673
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58751}
We plan to store additional information that is not related to scopes.
The new name will reflect this fact better.
Change-Id: I4ddb1017bc255e6ad271e4448848ed630f367d5b
Reviewed-on: https://chromium-review.googlesource.com/c/1388538
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58591}
This changes how rewind upon preparser abort works. It now rewinds to the start
of the parameter scope. In the case of "function X(" it is before the "(". In
the case of arrow functions it's before the start of the arrow function. This
allows us to reparse the arrow function from the start so all parameters are
declared properly.
Bug: v8:2728, v8:7390
Change-Id: I1c40056a49ec198560e63cd73949a59221ee0401
Reviewed-on: https://chromium-review.googlesource.com/c/1382736
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58332}
Moves the unicode predicate cache tables out of the unicode cache,
and turns them into generic predicates in char-predicates.h which
use static constexpr tables.
This drops the per-isolate cost of unicode caches, and removes the
need for accessing the unicode cache from most files. It does remove
the mutability of the cache, which means that there may be regressions
when parsing non-ASCII identifiers. Most likely the benefits to ASCII
identifiers/keywords will outweigh any non-ASCII costs.
Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399
Reviewed-on: https://chromium-review.googlesource.com/c/1335564
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#57506}
This flag has been on by default for some time. Once
https://chromium-review.googlesource.com/c/v8/v8/+/1270578 lands we need it to
be able to find duplicate parameters (to be spec-compliant).
Change-Id: I222023d7cd955127d3ecca42283b37063e962c58
Reviewed-on: https://chromium-review.googlesource.com/c/1270581
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56468}
This is a reland of 31228a69c3TBR=yangguo@chromium.org,verwaest@chromium.org
Original change's description:
> [Compile] Refactor BackgroundCompileTask to enable its use by CompilerDispatcher
>
> Splits background compilation data out of ScriptStreamingData and into
> BackgroundCompileTask. Also makes BackgroundCompileTask no longer a sub-class
> of ScriptStreamingTask, and instead have ScriptStreamingTask delegate to a
> BackgroundCompileTask.
>
> As part of this change, we now create the CharacterStream on the main thread,
> and therefore have to set the (thread-local) runtime_call_stats on the already
> created CharacterStream when the BackgroundCompileTask is run on the background
> thread. As such, changes to CharacterStream were needed to feed the
> runtime_call_stats through appropriately.
>
> Deprecates Source::GetCachedData and StreamedSource::GetCachedData since they are
> no longer used, and the streamed source never has cached data (streaming is
> suppressed if cached data is available). Also removes Utf8ChunkedStream which
> is dead code.
>
> BUG=v8:8041, v8:8015
> TBR=yangguo@chromium.org
>
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
> Change-Id: Ifcc723ebf930a1dc01135fcb70929d6168471cb3
> Reviewed-on: https://chromium-review.googlesource.com/1236353
> Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
> Reviewed-by: Toon Verwaest <verwaest@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56182}
Bug: v8:8041, v8:8015
Change-Id: Ied5132c537d4c25c6e355f289c2a9cc1f8ff98e9
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/1242097
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56201}
This reverts commit 31228a69c3.
Reason for revert: Introduces new pretty consistent flakes on Linux sanitizers, one of which block LKGR. Please see https://crbug.com/v8/8219
Original change's description:
> [Compile] Refactor BackgroundCompileTask to enable its use by CompilerDispatcher
>
> Splits background compilation data out of ScriptStreamingData and into
> BackgroundCompileTask. Also makes BackgroundCompileTask no longer a sub-class
> of ScriptStreamingTask, and instead have ScriptStreamingTask delegate to a
> BackgroundCompileTask.
>
> As part of this change, we now create the CharacterStream on the main thread,
> and therefore have to set the (thread-local) runtime_call_stats on the already
> created CharacterStream when the BackgroundCompileTask is run on the background
> thread. As such, changes to CharacterStream were needed to feed the
> runtime_call_stats through appropriately.
>
> Deprecates Source::GetCachedData and StreamedSource::GetCachedData since they are
> no longer used, and the streamed source never has cached data (streaming is
> suppressed if cached data is available). Also removes Utf8ChunkedStream which
> is dead code.
>
> BUG=v8:8041, v8:8015
> TBR=yangguo@chromium.org
>
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
> Change-Id: Ifcc723ebf930a1dc01135fcb70929d6168471cb3
> Reviewed-on: https://chromium-review.googlesource.com/1236353
> Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
> Reviewed-by: Toon Verwaest <verwaest@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#56182}
TBR=rmcilroy@chromium.org,yangguo@chromium.org,verwaest@chromium.org
Change-Id: Ib05bcbde2e9a588bd4008d2155f75cdac5cc47f5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:8041, v8:8015
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/1241958
Reviewed-by: Maya Lekova <mslekova@chromium.org>
Commit-Queue: Maya Lekova <mslekova@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56187}
Splits background compilation data out of ScriptStreamingData and into
BackgroundCompileTask. Also makes BackgroundCompileTask no longer a sub-class
of ScriptStreamingTask, and instead have ScriptStreamingTask delegate to a
BackgroundCompileTask.
As part of this change, we now create the CharacterStream on the main thread,
and therefore have to set the (thread-local) runtime_call_stats on the already
created CharacterStream when the BackgroundCompileTask is run on the background
thread. As such, changes to CharacterStream were needed to feed the
runtime_call_stats through appropriately.
Deprecates Source::GetCachedData and StreamedSource::GetCachedData since they are
no longer used, and the streamed source never has cached data (streaming is
suppressed if cached data is available). Also removes Utf8ChunkedStream which
is dead code.
BUG=v8:8041, v8:8015
TBR=yangguo@chromium.org
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Change-Id: Ifcc723ebf930a1dc01135fcb70929d6168471cb3
Reviewed-on: https://chromium-review.googlesource.com/1236353
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#56182}
Splits PreParsedScopeDataBuilder out of ProducedPreParserScopeData to make the split between
building PreParsedScopeData and using already build PreParserScopeData more explicit.
BUG=v8:8041
Change-Id: Iab42cab84c247152c14ac39f3136f985753160ec
Reviewed-on: https://chromium-review.googlesource.com/1202104
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55718}
Adds support for zone allocated (off-heap) ConsumedPreParsingScopeData to
enable worker-thread access to PreParsingScopeData during parallel IIFE
compile tasks.
In order to avoid code-duplication, a templated
BaseConsumedPreParsingScopeData is added which implements the logic for
decoding the bytestream into scope data. Two implementations of this
base class are instantiated for each of the underlying serialized scope date:
- ZoneConsumedPreParsedScopeData for exposing ZonePreParsedScopeData
- OnHeapConsumedPreParsedScopeData for exposing on-heap PreParsedScopeData
The interface for each of these classes is the ConsumedPreParsingScopeData,
which exposes the methods required by the parser to deserialize the required
data.
As a side-cleanup, moved Ucs2CharLength and Utf8LengthHelper implementations
to cc file so that we don't get a linker error if one of them are unused by
the cc file including the header.
BUG=v8:8041
Change-Id: Id502312d32fe4a9ddb6f5d2d9d3e3a9d30b9b27d
Reviewed-on: https://chromium-review.googlesource.com/1199462
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55711}
Utf*Characterstream caches the data pointer of ExternalStrings through
ExternalStringStream, so lock the strings in ExternalStringStream.
Bug: chromium:877044
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Change-Id: I241caaf64e109b33e2f9982573e11c514410509c
Reviewed-on: https://chromium-review.googlesource.com/1194003
Commit-Queue: Benoit L <lizeb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55613}
Currently chunked and streaming character streams don't have cloning support as this
requires additional changes on the Blink side.
BUG=v8:8041
Change-Id: I167b3b1cd5ae1ac4038f3715b6a679d3e65d9a85
Reviewed-on: https://chromium-review.googlesource.com/1183429
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55444}
This reverts the following 3 CLs:
Revert "[scanner] Templatize scan functions by encoding"
Revert "[asm] Remove invalid static cast of character stream"
Revert "[scanner] Prepare CharacterStreams for specializing scanner and parser by character type"
The original idea behind this work was to avoid copying, converting and
buffering characters to be scanned by specializing the scanner functions. The
additional benefit was for scanner functions to have a bigger window over the
input. Even though we can get a pretty nice speedup from having a larger
window, in practice this rarely helps. The cost is a larger binary.
Since we can't eagerly convert utf8 to utf16 due to memory overhead, we'd also
need to have a specialized version of the scanner just for utf8. That's pretty
complex, and likely won't be better than simply bulk converting and buffering
utf8 as utf16.
Change-Id: Ic3564683932a0097e3f9f51cd88f62c6ac879dcb
Reviewed-on: https://chromium-review.googlesource.com/1183190
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55258}
This way we can avoid reencoding everything to utf16 (buffered) and avoid the
overhead of needing to check the encoding for each character individually.
This may result in a minor asm.js scanning regression due to one-byte tokens
possibly being more common.
Change-Id: I90b51c256d56d4f4fa2d235d7e1e58fc01e43f31
Reviewed-on: https://chromium-review.googlesource.com/1172437
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Leszek Swirski <leszeks@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55217}
This templatizes CharacterStream by char type, and makes them subclass ScannerStream.
Methods that are widely used by tests are marked virtual on ScannerStream and final on
CharacterStream<T> so the specialized scanner will know what to call. ParseInfo passes
around ScannerStream, but the scanner requires the explicit CharacterStream<T>. Since
AdvanceUntil is templatized by FunctionType, I couldn't mark that virtual; so instead
I adjusted those tests to operate directly on ucs2 (not utf8 since we'll drop that in
the future).
In the end no functionality was changed. Some calls became virtual in tests. This is
mainly just preparation.
Change-Id: I0b4def65d3eb8fa5c806027c7e9123a590ebbdb5
Reviewed-on: https://chromium-review.googlesource.com/1156690
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54848}
Refactoring the code base to use noexcept for their move constructors and move
assignment operators.
Bug: v8:7999
Cq-Include-Trybots: luci.chromium.try:linux_chromium_headless_rel;master.tryserver.blink:linux_trusty_blink_rel
Change-Id: I13d24eddba3bfa601cff26fd680a040cf4e71426
Reviewed-on: https://chromium-review.googlesource.com/1152817
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Commit-Queue: Florian Sattler <sattlerf@google.com>
Cr-Commit-Position: refs/heads/master@{#54841}
