This reverts the following 3 CLs:
Revert "[scanner] Templatize scan functions by encoding"
Revert "[asm] Remove invalid static cast of character stream"
Revert "[scanner] Prepare CharacterStreams for specializing scanner and parser by character type"
The original idea behind this work was to avoid copying, converting and
buffering characters to be scanned by specializing the scanner functions. The
additional benefit was for scanner functions to have a bigger window over the
input. Even though we can get a pretty nice speedup from having a larger
window, in practice this rarely helps. The cost is a larger binary.
Since we can't eagerly convert utf8 to utf16 due to memory overhead, we'd also
need to have a specialized version of the scanner just for utf8. That's pretty
complex, and likely won't be better than simply bulk converting and buffering
utf8 as utf16.
Change-Id: Ic3564683932a0097e3f9f51cd88f62c6ac879dcb
Reviewed-on: https://chromium-review.googlesource.com/1183190
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#55258}
This templatizes CharacterStream by char type, and makes them subclass ScannerStream.
Methods that are widely used by tests are marked virtual on ScannerStream and final on
CharacterStream<T> so the specialized scanner will know what to call. ParseInfo passes
around ScannerStream, but the scanner requires the explicit CharacterStream<T>. Since
AdvanceUntil is templatized by FunctionType, I couldn't mark that virtual; so instead
I adjusted those tests to operate directly on ucs2 (not utf8 since we'll drop that in
the future).
In the end no functionality was changed. Some calls became virtual in tests. This is
mainly just preparation.
Change-Id: I0b4def65d3eb8fa5c806027c7e9123a590ebbdb5
Reviewed-on: https://chromium-review.googlesource.com/1156690
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54848}
This patch normalizes the casing of hexadecimal digits in escape
sequences of the form `\xNN` and integer literals of the form
`0xNNNN`.
Previously, the V8 code base used an inconsistent mixture of uppercase
and lowercase.
Google’s C++ style guide uses uppercase in its examples:
https://google.github.io/styleguide/cppguide.html#Non-ASCII_Characters
Moreover, uppercase letters more clearly stand out from the lowercase
`x` (or `u`) characters at the start, as well as lowercase letters
elsewhere in strings.
BUG=v8:7109
TBR=marja@chromium.org,titzer@chromium.org,mtrofin@chromium.org,mstarzinger@chromium.org,rossberg@chromium.org,yangguo@chromium.org,mlippautz@chromium.org
NOPRESUBMIT=true
Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.linux:linux_chromium_rel_ng
Change-Id: I790e21c25d96ad5d95c8229724eb45d2aa9e22d6
Reviewed-on: https://chromium-review.googlesource.com/804294
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49810}
This CL fixes all occurences that don't require special OWNER reviews,
or can be reviewed by Michi.
After this one, we should be able to reenable the readability/check
cpplint check.
R=mstarzinger@chromium.org
Bug: v8:6837, v8:6921
Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng;master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Ic81d68d5534eaa795b7197fed5c41ed158361d62
Reviewed-on: https://chromium-review.googlesource.com/721120
Commit-Queue: Clemens Hammacher <clemensh@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48670}
Instead of creating a new character stream to re-parse the asm.js module,
use the existing stream which was used by the parser. By doing this, we
avoid accessing the heap if the original character stream is a streaming
source or an external string, which will enable asm.js verification to run
off-thread in those situations.
BUG=v8:5203
Change-Id: I5dbf83c993512eb2f3dd709120e152e3f9900bdf
Reviewed-on: https://chromium-review.googlesource.com/616723
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47500}
This fixes the bounds checking of "unsigned" numeric literals (those
that do not contains dots) by the parser. In particular this fixes a
bogus truncation to 32-bit in the scanner. It also makes the scanner
more robust by limiting the range of those numeric literals, hence
completely avoiding rounding loss or truncation errors.
R=clemensh@chromium.org
TEST=unittests/AsmJsScannerTest.UnsignedNumbers
BUG=v8:6298
Change-Id: Id31ab3c652e99fa8d3d6663315768e1bfaf3b773
Reviewed-on: https://chromium-review.googlesource.com/486881
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44890}
This switches the parser to use token positions (i.e. {Position})
instead of stream positions (i.e. {GetPosition}) everywhere. Access to
the latter is being removed as it is unsupported when the scanner is in
rewind state anyways. This prevents "skipping" a token when seeking.
R=bradnelson@chromium.org
BUG=v8:6127
Change-Id: I9c13dd20a981061a2bccc4fb57e5c57d2a64ac5c
Reviewed-on: https://chromium-review.googlesource.com/480300
Reviewed-by: Brad Nelson <bradnelson@chromium.org>
Commit-Queue: Brad Nelson <bradnelson@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44699}
Adding a custom lexer for asm.js parsing.
It takes advantage of a number of asm.js properties to simply things:
* Assumes 'use asm' is the only string.
* Does not handle unicode for now (tools don't emit it).
* Combines global + local string table with lexer.
R=marja@chromium.org,vogelheim@chromium.org,kschimpf@chromium.org
BUG=v8:4203
BUG=v8:6090
Review-Url: https://codereview.chromium.org/2751693002
Cr-Commit-Position: refs/heads/master@{#43874}
