Commit Graph

83 Commits

Author SHA1 Message Date
Marja Hölttä
d553c94309 [reland] [parser] Skipping inner funcs: Use less memory for variables.
- Make it possible to store quarter-bytes instead of full bytes.

- Don't store is_used; it can be recovered correctly based on the actual full
  parse (when a lazy function is eventually called) and
  has_forced_scope_allocation.

- With the is_used change, the old testing approach (which compared a scope for
  which we didn't do scope allocation to the baseline) no longer made
  sense. Replaced it with a new testing approach, which is also closer to the
  actual usage.

- First version (reverted): https://chromium-review.googlesource.com/725422

BUG=v8:5516

Change-Id: I1468af6670b689a104bd867377caa1d236070820
Reviewed-on: https://chromium-review.googlesource.com/733123
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48903}
2017-10-25 08:49:37 +00:00
Marja Hölttä
271e3b108d Revert "[parser] Skipping inner funcs: Use less memory for variables."
This reverts commit 97ead4338e.

Reason for revert: makes the PreParserScopeAnalysis test much slower.

Original change's description:
> [parser] Skipping inner funcs: Use less memory for variables.
> 
> - Make it possible to store quarter-bytes instead of full bytes.
> 
> - Don't store is_used; it can be recovered correctly based on the actual full
>   parse (when a lazy function is eventually called) and
>   has_forced_scope_allocation.
> 
> - With the is_used change, the old testing approach (which compared a scope for
>   which we didn't do scope allocation to the baseline) no longer made
>   sense. Replaced it with a new testing approach, which is also closer to the
>   actual usage.
> 
> BUG=v8:5516
> 
> Change-Id: I02bac24e482126689dcdbabe8b3a04977be29b0c
> Reviewed-on: https://chromium-review.googlesource.com/725422
> Commit-Queue: Marja Hölttä <marja@chromium.org>
> Reviewed-by: Toon Verwaest <verwaest@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48828}

TBR=marja@chromium.org,verwaest@chromium.org

Change-Id: I8cb87bcd55462b1cef4444dabb5cbfa2ecb24c7c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:5516
Reviewed-on: https://chromium-review.googlesource.com/732878
Reviewed-by: Marja Hölttä <marja@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48829}
2017-10-23 12:42:19 +00:00
Marja Hölttä
97ead4338e [parser] Skipping inner funcs: Use less memory for variables.
- Make it possible to store quarter-bytes instead of full bytes.

- Don't store is_used; it can be recovered correctly based on the actual full
  parse (when a lazy function is eventually called) and
  has_forced_scope_allocation.

- With the is_used change, the old testing approach (which compared a scope for
  which we didn't do scope allocation to the baseline) no longer made
  sense. Replaced it with a new testing approach, which is also closer to the
  actual usage.

BUG=v8:5516

Change-Id: I02bac24e482126689dcdbabe8b3a04977be29b0c
Reviewed-on: https://chromium-review.googlesource.com/725422
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48828}
2017-10-23 11:48:45 +00:00
Marja Hölttä
f6f5bafe41 [parser] Skipping inner funcs: Fix hoisting.
The catch variable is a special VAR-mode variable which is not in a declaration
scope. Normally creating such a variable is not possible with DeclareVariable,
but Parser bypasses it by calling DeclareLocal directly (which doesn't have the
hoisting check).

PreParser used to cut corners and declare the catch variable as a LET-mode
variable to prevent hoisting.

But since LET and VAR variables behave differently when deciding whether they
block sloppy block function hoisting, that approach doesn't fly.

BUG=v8:5516,chromium:771474

Change-Id: Ic6f5f4996416c9fa59132725c8b0b6b570c72f48
Reviewed-on: https://chromium-review.googlesource.com/700634
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48308}
2017-10-05 12:00:45 +00:00
Mathias Bynens
d3c9812143 [parser] Add use counter for U+2028 & U+2029
The context is the following proposal to make JSON a subset of
JavaScript: https://github.com/tc39/proposal-json-superset

There’s interest in performing a side investigation to answer the
question of what would happen if we stopped treating U+2028 and U+2029
as `LineTerminator`s *entirely*. (Note that this is separate from the
proposal, which just changes how these characters are handled in
ECMAScript strings.) This is technically a breaking change, and IMHO it
would be wonderful if we could get away with it, but no one really has
any data on whether or not we could. Adding this use counter lets us get
that data.

BUG=v8:6827

Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
Change-Id: Ia22e8db1634df4d3f965bec8e1cfa11cc7b5e9aa
Reviewed-on: https://chromium-review.googlesource.com/693155
Commit-Queue: Mathias Bynens <mathias@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48260}
2017-10-02 11:46:36 +00:00
Marja Hölttä
a02580636f [parser] Skipping inner funcs: Make the data on heap smaller.
We were unnecessarily storing everything as uint32_t, even though many items in
the preparsed scope data can be stored as uint8_t. This CL also adds an
(internal) API which abstracts away the actual data storing, so the backing
store can be made even more efficient (e.g., use only 1-3 bytes for some
uint32_t values, if they fit) without affecting other parts of the code.

BUG=v8:5516,chromium:762492

Change-Id: I7cd4d91dc11f87f8aec9c7584044a6f2a59b73ba
Reviewed-on: https://chromium-review.googlesource.com/684182
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48231}
2017-09-29 13:31:13 +00:00
Marja Hölttä
6389b7e6b6 [unicode] Return (the correct) errors for overlong / surrogate sequences.
This fix is two-fold:

1) Incremental UTF-8 decoding: Unify incorrect UTF-8 handling between V8 and
Blink.

Incremental UTF-8 decoding used to allow some overlong sequences / invalid code
points which Blink treated as errors. This caused the decoder and the Blink
UTF-8 decoder to produce a different number of bytes, resulting in random
failures when scripts were streamed (especially, this was detected by the
skipping inner functions feature which adds CHECKs against expected function
positions).

2) Non-incremental UTF-8 decoding: return the correct amount of invalid characters.

According to the encoding spec ( https://encoding.spec.whatwg.org/#utf-8-decoder
), the first byte of an overlong sequence / invalid code point generates an
invalid character, and the rest of the bytes are not processed (i.e., pushed
back to the byte stream). When they're handled, they will look like lonely
continuation bytes, and will generate an invalid character each.

As a result, an overlong 4-byte sequence should generate 4 invalid characters
(not 1).

This is a potentially breaking change, since the (non-incremental) UTF-8
decoding is exposed via the API (String::NewFromUtf8). The behavioral difference
happens when the client is passing in invalid UTF-8 (containing overlong /
surrogate sequences).

However, afaict, this doesn't change the semantics of any JavaScript program:
according to the ECMAScript spec, the program is a sequence of Unicode code
points, and there's no way to invoke the UTF-8 decoding functionalities from
inside JavaScript. Though, this changes the behavior of d8 when decoding source
files which are invalid UTF-8.

This doesn't change anything related to URI decoding (it already throws
exceptions for overlong sequences / invalid code points).

BUG: chromium:765608, chromium:758236, v8:5516
Bug: 
Change-Id: Ib029f6a8e87186794b092e4e8af32d01cee3ada0
Reviewed-on: https://chromium-review.googlesource.com/671020
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Franziska Hinkelmann <franzih@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48105}
2017-09-21 10:44:40 +00:00
Marja Hölttä
68310c9f69 [scanner] UTF-8 handling fix (errors near chunk end).
The bug occurred when we detected an erroneous char late, and put the last
character in a chunk into the "incomplete char" buffer. It was not correctly
retrieved when seeking.

BUG=v8:6836

Change-Id: I8ca946dfdb39244c5ca0bdcebe047047010b3a07
Reviewed-on: https://chromium-review.googlesource.com/670729
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48066}
2017-09-18 14:13:26 +00:00
Marja Hölttä
9f21cab8c8 Revert "Reland#2 [parser] Refactor streaming scanner streams."
This reverts commit de9269f3c3.

Something's still wrong in the encoding handling (see bug).

Bug: chromium:763106
Cq-Include-Trybots: master.tryserver.chromium.linux:linux_chromium_rel_ng
Change-Id: Icd19dd42b84b9d090e191375a2942b9941110bcf
Reviewed-on: https://chromium-review.googlesource.com/657386
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47924}
2017-09-08 13:36:04 +00:00
Marja Hölttä
025ea28bca [parser] Skipping inner funcs: fix sloppy block generators.
PreParser and Parser didn't agree whether a generator in a sloppy block is a
sloppy block function or not, and thus the data generated by PreParser was
inconsistent with what the Parser wanted to restore.

BUG=v8:5516, chromium:760116

Change-Id: I0fd3c267691b8afd63a1336774769caf551c143e
Reviewed-on: https://chromium-review.googlesource.com/642886
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47727}
2017-08-31 05:42:36 +00:00
Marja Hölttä
4e45342994 [script streaming] Fix U+feff handling.
U+feff is the UTF BOM but if it occurs inside the text, it's a "zero-width
no-break space". However, the UTF-8 decoder in script streaming still thought
it's a BOM and skipped it. The correct way to handle it would be to create a
U+feff code point instead - the Scanner will then handle it as whitespace.

This is a discrepancy between the Blink UTF-8 decoder and the V8 UTF-8 decoder,
and caused the source positions be off by one. This bug went unnoticed, since
normally off-by-one in this situation doesn't make the code to break.

BUG=chromium:758508,chromium:758236

Change-Id: Ib92a3ee65c402e21b77e42537db2a021cff55379
Reviewed-on: https://chromium-review.googlesource.com/632096
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47583}
2017-08-24 19:35:12 +00:00
Marja Hölttä
c3cf3d0661 [parser] Skipping inner funcs: drop "experimental" from flag name.
Makes ClusterFuzz start fuzzing with the flag on.

BUG=v8:5516

Change-Id: Ia80f7d22f12fe25efb226102a896e8b0e3537947
Reviewed-on: https://chromium-review.googlesource.com/610000
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47366}
2017-08-16 08:49:27 +00:00
Mostyn Bramley-Moore
cddbe282c6 Start preparing test/cctest for jumbo compilation
* Avoid "using namespace" statements, which trigger clang's -Wheader-hygiene
  warnings in jumbo builds.
* Undefine created macros at the end of source files.

BUG=chromium:746958

Change-Id: I5d25432c314437f607b0e1be22765a6764267ba6
Reviewed-on: https://chromium-review.googlesource.com/610962
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Commit-Queue: Mostyn Bramley-Moore <mostynb@opera.com>
Cr-Commit-Position: refs/heads/master@{#47347}
2017-08-14 20:58:10 +00:00
Marja Hölttä
70f6913159 [parser] Skipping inner funcs: remove untrue DCHECK.
- See bug for the reduced test case.

- Not adding a regression test here: I don't want to assert that PreParser
  doesn't detect the redeclaration error, OTOH I don't want to make it detect
  the error either (in order to not couple detecting the error with
  FLAG_experimental_preparser_analysis).

BUG=chromium:753896, v8:5516

Change-Id: I0f1beffe30e5cb48d6dbec35181980864e6df153
Reviewed-on: https://chromium-review.googlesource.com/608976
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47326}
2017-08-12 16:43:40 +00:00
Marja Hölttä
d50b196246 [parser] Skipping inner funcs: add more tests.
These now pass because of https://chromium-review.googlesource.com/603787.

BUG=v8:5516

Change-Id: Ifcb576056c3cba523e56c5c9c563724efe096012
Reviewed-on: https://chromium-review.googlesource.com/608972
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47325}
2017-08-12 16:42:35 +00:00
Ross McIlroy
265e87e59e [Compiler] Seperate AttachOuterScopeInfo out of DeclarationScope::Analyze
Splits out AttachOuterScopeInfo from DeclarationScope::Analyze and attaches
the outer scope info after parsing has completed (when parsing on the main
thread, which is the only time we have an outer scope info) instead of
during Compiler::Analyse().

BUG=v8:5203

TBR=yangguo@chromium.org

Change-Id: Idd8d2409fb20f09a9f6bbf5cff7e6edcf90077d7
Reviewed-on: https://chromium-review.googlesource.com/605889
Commit-Queue: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47243}
2017-08-09 11:04:13 +00:00
Marja Hölttä
e7a46253f2 [parser] Skipping inner funcs: implement a bailout.
In some cases, PreParser cannot replicate the Scope structure created by
Parser. It happens esp. with arrow function parameters, since the relevant
information is already lost by the time we figure out it's an arrow function.

In these cases, PreParser should bail out of trying to create data for skipping
inner functions.

Implementation notes:

- The arrow function case is more fundamental; the non-arrow case could be
  hacked together somehow if we implemented tracking is_simple for each param
  separately; but now that it's possible to bail out consistently from both
  cases, I don't think the is_simple complication is worth it.

- The added mjsunit test cases are based on the test262 test cases which exposed
  the problem.

- cctest/preparser/PreParserScopeAnalysis was exercising similar cases, but the
  problem didn't show up because the function parameters didn't contain
  skippable functions. Those test cases have been repurposed for testing the
  bailout.

- Extra precaution: the bailout tests are in a separate file, to guard from the
  bug that a bailout case results in bailing out of *all* data creation, which
  would make all skipping tests in the same file useless.

BUG=v8:5516

Change-Id: I4324749a5ec602fa5d7dc27647ade0284a6842fe
Reviewed-on: https://chromium-review.googlesource.com/599849
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#47170}
2017-08-04 18:22:32 +00:00
Marja Hölttä
937b5011b8 [parser] Skipping inner funcs: Associate data to SharedFunctionInfo, not Script.
This way, each lazy function needs to handle only the data relevant to
itself. This reduced data handling overheads.

Other changes:

1) Don't deserialize the data; once it's on the heap, it can stay there. Lazy
function compilation is only done in the main thread.

2) Separate ProducedPreParsedScopeData and ConsumedPreParsedScopeData. It's clearer, because:

- The data looks fundamentally different when we're producing it and when we're
  consuming it.

- Cleanly separates the operations we can do in the "producing phase" and in the
  "consuming phase".

Bug: v8:5516
Change-Id: I6985a6621f71b348a55155724765624b5d5f7c33
Reviewed-on: https://chromium-review.googlesource.com/528094
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46347}
2017-06-30 11:12:52 +00:00
Marja Hölttä
2c260da393 [parser] Skipping inner funcs: fix destructuring catch variables.
Make PreParser match what Parser does.

BUG=v8:5516

Change-Id: I2801206fd17b9a5047bc43c6112f4945971596b7
Reviewed-on: https://chromium-review.googlesource.com/544949
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46169}
2017-06-23 11:26:49 +00:00
Marja Hölttä
1fce2d2d61 [parser] Skipping inner funcs: Fix function name declarations
let f = function g() { ... } declares "g" inside the function. This
CL makes the preparser declare it too, and saves + restores the scope data for
it.

BUG=v8:5516

Change-Id: Id4c64f446d30f5252038cfb0f0f473b85ba24a9b
Reviewed-on: https://chromium-review.googlesource.com/544816
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46133}
2017-06-22 13:38:26 +00:00
Marja Hölttä
2b730f6335 [parser] Skipping inner funcs: fix the test setup.
The test setup was as follows:
- Preparse function test() { ... }, get scope allocation data.
- Apply the scope allocation data to (function test() { ... })();
- Compare against normal scope allocation for (function test() { ... })();

But the IIFE is unnecessary - we already disable lazy parsing.

Cleaning this up is needed because in the next CL, I want to fix the Scopes
produced by PreParser in this case:

let f = function g() {
  // Here we should declare g!
}

And that fix will make the variables in
function test() {
  // Here we don't declare test
}
and
(function test() {
  // Here we do declare test
})();
not match any more, so it doesn't make sense to compare them against each other.

BUG=v8:5516

Change-Id: I93d154c6977bb3cbe405b6ca193cf6283df297bc
Reviewed-on: https://chromium-review.googlesource.com/543341
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46128}
2017-06-22 12:10:52 +00:00
Wiktor Garbacz
de9269f3c3 Reland#2 [parser] Refactor streaming scanner streams.
Unify, simplify logic, reduce UTF8 specific handling.

Intend of this is also to have stream views.
Stream views can be used concurrently by multiple threads, but
only one thread may fetch new data from the underlying source.
This together with unified stream view creation is intended to be
used for parse tasks.

BUG=v8:6093

Change-Id: I83c6f1e6ad280c28da690da41c466dfcbb7915e6
Reviewed-on: https://chromium-review.googlesource.com/535474
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45994}
2017-06-19 10:18:01 +00:00
Wiktor Garbacz
f4f723e818 [parsing] Fix past the end position for streaming streams.
Also, as this is hard to track down, always DCHECK position after ReadBlock().

Change-Id: Ie32c3a311dd8df91f651b6d82ccacc7c95e6fde0
Reviewed-on: https://chromium-review.googlesource.com/528196
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45811}
2017-06-09 11:35:24 +00:00
Marja Hölttä
4ca7022295 Revert "Reland [parser] Refactor streaming scanner streams."
This reverts commit 7fa071a48b.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=729482

Original change's description:
> Reland [parser] Refactor streaming scanner streams.
> 
> Unify, simplify logic, reduce UTF8 specific handling.
> 
> Intend of this is also to have stream views.
> Stream views can be used concurrently by multiple threads, but
> only one thread may fetch new data from the underlying source.
> This together with unified stream view creation is intended to be
> used for parse tasks.
> 
> BUG=v8:6093
> 
> Change-Id: I3bce48185fa2c986d16619a9a8ece3ff4c4f5e60
> Reviewed-on: https://chromium-review.googlesource.com/509489
> Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
> Reviewed-by: Marja Hölttä <marja@chromium.org>
> Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
> Cr-Commit-Position: refs/heads/master@{#45688}

TBR=marja@chromium.org,vogelheim@chromium.org,wiktorg@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
BUG=v8:6093

Change-Id: Iefa7c43a2f6ae3a7f3ef0f77d87b6ae36ae4be99
Reviewed-on: https://chromium-review.googlesource.com/525712
Reviewed-by: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45725}
2017-06-06 11:42:30 +00:00
Wiktor Garbacz
7fa071a48b Reland [parser] Refactor streaming scanner streams.
Unify, simplify logic, reduce UTF8 specific handling.

Intend of this is also to have stream views.
Stream views can be used concurrently by multiple threads, but
only one thread may fetch new data from the underlying source.
This together with unified stream view creation is intended to be
used for parse tasks.

BUG=v8:6093

Change-Id: I3bce48185fa2c986d16619a9a8ece3ff4c4f5e60
Reviewed-on: https://chromium-review.googlesource.com/509489
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
Cr-Commit-Position: refs/heads/master@{#45688}
2017-06-02 13:50:08 +00:00
Marja Hölttä
06f05ec231 [parser] Skipping inner funcs: make more functions skippable.
- Enable aggressive lazy inner funcs (make non-declaration funcs lazy, ie let f =
  function() { ... } when --experimental-preparser-scope-analysis is on.
- Turn on variable tracking for lazy top level functions: this makes their inner
  functions skippable.
- Test fix for an testing bug uncovered by this work: when restoring the data
  for the relevant scope, don't assume it's the outermost scope for which we
  have data.
- Fix: if we abort lazy parsing a function, we shouldn't produce any data for
  it.

BUG=v8:5516

Change-Id: I0606fbabb5886dc57dbb53ab5f3fb894ff5d032e
Reviewed-on: https://chromium-review.googlesource.com/518165
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45615}
2017-05-31 07:49:55 +00:00
Adam Klein
9397c1b73a Revert "[parser] Refactor streaming scanner streams."
This reverts commit ce538f70c1.

Reason for revert: breaks BOM handling (thus breaking Outlook web apps).

Original change's description:
> [parser] Refactor streaming scanner streams.
> 
> Unify, simplify logic, reduce UTF8 specific handling.
> 
> Intend of this is also to have stream views.
> Stream views can be used concurrently by multiple threads, but
> only one thread may fetch new data from the underlying source.
> This together with unified stream view creation is intended to be
> used for parse tasks.
> 
> BUG=v8:6093
> 
> Change-Id: Ied8e93090c506d4735080298f0fdaeed32043915
> Reviewed-on: https://chromium-review.googlesource.com/501789
> Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
> Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
> Reviewed-by: Marja Hölttä <marja@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#45336}

TBR=marja@chromium.org,vogelheim@chromium.org,jochen@chromium.org,wiktorg@google.com
BUG=v8:6093, chromium:724166

Change-Id: I022a23b8052d20d83a640c07b7864c622548bf90
Reviewed-on: https://chromium-review.googlesource.com/508888
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45404}
2017-05-18 19:28:58 +00:00
Marja Hölttä
05696362ad [parser] Skipping inner funcs: fix preparsing super.
Super calls need to refer to .this_function, .new.target and this, and super
property references need to refer to .this_function and this, so that the
is_used for those variables will be set and they will be allocated correctly.

BUG=v8:5516

Change-Id: Idc58539fccad70c995e029051b59a67ea66bff91
Reviewed-on: https://chromium-review.googlesource.com/506094
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45376}
2017-05-17 13:42:41 +00:00
Wiktor Garbacz
ce538f70c1 [parser] Refactor streaming scanner streams.
Unify, simplify logic, reduce UTF8 specific handling.

Intend of this is also to have stream views.
Stream views can be used concurrently by multiple threads, but
only one thread may fetch new data from the underlying source.
This together with unified stream view creation is intended to be
used for parse tasks.

BUG=v8:6093

Change-Id: Ied8e93090c506d4735080298f0fdaeed32043915
Reviewed-on: https://chromium-review.googlesource.com/501789
Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45336}
2017-05-16 11:34:41 +00:00
Marja Hölttä
fab691b8ba [parser] Skipping inner funcs: add more tests.
These tests already pass.

BUG=v8:5516

Change-Id: I7f4796cca98aa6c276143d83bcb018a67e0cbe55
Reviewed-on: https://chromium-review.googlesource.com/506193
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45327}
2017-05-16 09:18:22 +00:00
Marja Hölttä
3e12ed1f78 [parser] Skipping inner funcs: Fix related to classes.
- Default constructor scopes won't need the scope data for deciding the scope
allocation of variables inside them. Also, PreParser doesn't construct them. So
they should be just skipped when applying the scope data.

- PreParser needs to declare the class name + have a proper end position for
the class scope.

- This makes all mjsunit tests pass with --experimental-preparser-scope-analysis.

- Also added several DCHECKs which were useful for debugging.

BUG=v8:5516

Change-Id: I5b3e6c60ed79efe25f33576a3547d707c700c6dd
Reviewed-on: https://chromium-review.googlesource.com/503208
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45303}
2017-05-15 14:22:34 +00:00
Wiktor Garbacz
4f5dcd4b4d [parser] Fix chunked utf8 stream handling.
BUG=v8:6377

Change-Id: I5bdd41bdda83d7efe4b37d24d44e2e8c2339a30a
Reviewed-on: https://chromium-review.googlesource.com/500168
Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45204}
2017-05-09 18:41:00 +00:00
Sathya Gunasekaran
0015bbb5e1 [modules] Disallow HTML comments in modules
Bug: v8:5045
Change-Id: I1d8b6be8a65595dc357c4f721b1a03425e025e6e
Reviewed-on: https://chromium-review.googlesource.com/463811
Commit-Queue: Sathya Gunasekaran <gsathya@chromium.org>
Reviewed-by: Daniel Ehrenberg <littledan@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45107}
2017-05-04 18:42:12 +00:00
Marja Hölttä
b4c8d38387 [parser] Skipping inner funcs: minor fixes.
1) Function recording conditions need to be consistent (this same condition is used above)

2) byte is not wide enough for storing the backing store size.

Bugs uncovered by the existing tests with the flag on.

BUG=v8:5516

Change-Id: Iec6aff0cf1858afe1083526e4ada9a8eca08f062
Reviewed-on: https://chromium-review.googlesource.com/481320
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44724}
2017-04-19 13:29:40 +00:00
Marja Hölttä
7fcf658a7b [parser] Skipping inner funcs: make the flag experimental.
The feature is not quite ready for getting ClusterFuzzed.

BUG=v8:5516

Change-Id: I90a42f950727c8ecf46cb2987c9a459b2ba1f5a7
Reviewed-on: https://chromium-review.googlesource.com/480400
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44693}
2017-04-18 15:16:10 +00:00
Daniel Vogelheim
ae1e87596d [parser] Introduce 'contextual keyword tokens'.
Introduce 'contextual keyword' tokens, which are parsed as identifiers but
in some contexts are treated by the parser like proper keywords. These are
usually keywords introduced by recent ECMAScript versions, which for reasons
of backwards compatibility are still permissible as regular identifiers in
most contexts.

Current usage is to check for Token::IDENTIFIER and then do a string
compare. With this change the initial scan will scan them as usual, but
will then record the token as IDENTIFIER plus a secondary token with the
'contextual' value.

BUG=v8:6902

Change-Id: I6ae390382998cf756a23720bd481cb9c0eb78a72
Reviewed-on: https://chromium-review.googlesource.com/459479
Commit-Queue: Daniel Vogelheim <vogelheim@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44189}
2017-03-28 12:43:23 +00:00
Wiktor Garbacz
80752a29b6 Make isolate explicit param of parsing:: functions
A step towards removing isolate from ParseInfo.
Removing isolate from ParseInfo will make it easier to create and
execute parse tasks on background threads.

BUG=v8:6093

Change-Id: I0a3546618d01b9232014da94cf8d0f72427a0d1d
Reviewed-on: https://chromium-review.googlesource.com/458006
Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44176}
2017-03-28 08:20:56 +00:00
Wiktor Garbacz
048f89c80d Do NOT get isolate from ParseInfo in compiler.cc
A step towards removing isolate from ParseInfo.
Removing isolate from ParseInfo will make it easier to create and
execute parse tasks on background threads.

BUG=v8:6093

Change-Id: Ief4eb3c9873026a93338d5556985f31c9abe17e6
Reviewed-on: https://chromium-review.googlesource.com/458005
Commit-Queue: Wiktor Garbacz <wiktorg@google.com>
Reviewed-by: Andreas Haas <ahaas@chromium.org>
Reviewed-by: Daniel Clifford <danno@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44173}
2017-03-28 08:12:48 +00:00
bbudge
771e86fdf2 Remove Factory::NewStringFromASCII method.
BUG=none

Review-Url: https://codereview.chromium.org/2759513002
Cr-Commit-Position: refs/heads/master@{#43913}
2017-03-17 17:52:50 +00:00
Marja Hölttä
f489f7ab44 [parser] Skipping inner funcs: collect data needed for allocation, not the allocation result.
This pretty much rewrites the preparsed scope data collection. We used to store
the allocation result, but it's faster to just store the raw data which is
needed for deciding it later. (This way we don't need to run the allocation
algorithm for just getting this data.)

For each variable: is_used, maybe_assigned,
has_forced_context_allocation, and for each scope:
inner_scope_calls_eval_.

In addition, this CL moves data handling out of Scope and into
PreParsedScopeData where it belongs and simplifies the API for
PreParsedScopeData.

BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: Ia5a4fa52f585cd4f483ce9a92f2dd7d9754f34ed
Reviewed-on: https://chromium-review.googlesource.com/451273
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43641}
2017-03-07 14:23:14 +00:00
Marja Hölttä
5178fa0e76 [parser] Skipping inner funcs: add tests.
BUG=v8:5516
R=verwaest@chromium.org

Change-Id: Ica04f74788a4812128bce1297aeee23f8c3bec3d
Reviewed-on: https://chromium-review.googlesource.com/449675
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43591}
2017-03-03 15:01:12 +00:00
Marja Hölttä
12c2c15310 [parser] Skipping inner funcs: arrow functions now work too.
BUG=v8:5516
R=verwaest@chromium.org

Change-Id: I1bb88e981b81fec85ba6da314e3be927085e6e76
Reviewed-on: https://chromium-review.googlesource.com/447438
Reviewed-by: Adam Klein <adamk@chromium.org>
Reviewed-by: Toon Verwaest <verwaest@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43526}
2017-03-01 20:21:06 +00:00
Marja Hölttä
6b5ab92315 [parser] Skipping inner funcs: omit uninteresting scopes in the data.
This is also needed so that PreParser doesn't need to gather more data for arrow
function params in order to create the uninteresting varblock scopes matching
the scopes created in Parser::BuildParameterInitializationBlock.

This cancels the changes in https://chromium-review.googlesource.com/c/444747
which make PreParser create uninteresting scopes for the normal (non-arrow)
function "eval in default param" case.

R=vogelheim@chromium.org
BUG=v8:5516

Change-Id: I8957ac0796d8738c63492f7928bca6f00e4b4241
Reviewed-on: https://chromium-review.googlesource.com/446339
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43411}
2017-02-24 12:20:25 +00:00
Marja Hölttä
2a2fc69221 [parser] Skipping inner funcs: enable more strict mode tests.
Non-simple parameters are only disallowed when a function declares itself
strict, but they're otherwise ok in strict mode.

Enabling these tests will expose more problems when scope data for arrow
functions is tested (in a future CL).

BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: I839ad37d46305975a56aff20e8ca70505c16bf1d
Reviewed-on: https://chromium-review.googlesource.com/446497
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43389}
2017-02-23 12:53:32 +00:00
Marja Hölttä
7a457c711b [parser] Skipping inner funcs: params part 5
Handle eval in default parameters.

BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: Ib6543a4aef9a3cc9636e65d0337bc269c8a079dc
Reviewed-on: https://chromium-review.googlesource.com/444747
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43328}
2017-02-20 16:37:24 +00:00
Marja Hölttä
6b5f7d23dc [parser] Skipping inner funcs: add tests for strict mode.
BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: I7a39e49cc569b87064e5ac6764f4e3a65b2c6d11
Reviewed-on: https://chromium-review.googlesource.com/444765
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43311}
2017-02-20 10:56:12 +00:00
Marja Hölttä
3ee21f289b [parser] Skipping inner funcs: params part 4
Add some tests which already pass.

BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: Icf5cf5e40010deb2efb69dea750661854b055566
Reviewed-on: https://chromium-review.googlesource.com/444744
Commit-Queue: Marja Hölttä <marja@chromium.org>
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43278}
2017-02-17 12:19:35 +00:00
Marja Hölttä
712800a636 [parser] Skipping inner funcs: params part 3
Produce the same scopes / variables for parameters (part 3).

This CL fixes the ordering + variable types in PreParser when there are
simple parameters + a rest parameter. In that case, Parser declares
unnamed temporaries for the non-rest params, then the rest param, then
the named variables (which are not parameters) for the non-rest params.

BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: I9b006595039c8002b0508d1d2a200aa9a0f3eae0
Reviewed-on: https://chromium-review.googlesource.com/443527
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43241}
2017-02-16 10:24:52 +00:00
Marja Hölttä
8c449e39b5 [parsing/test] Skipping inner funcs: add more (structured tests).
Test both cases where the interesting constructs occur at the
laziness boundary and cases where they occur deeper.

BUG=v8:5501
R=vogelheim@chromium.org

Change-Id: I99e32cb0c829616011bf7d1f389a8d309b54d67e
Reviewed-on: https://chromium-review.googlesource.com/441844
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43157}
2017-02-13 15:04:34 +00:00
Marja Hölttä
bfd65a54fa [parser] Skipping inner funcs: produce the same scopes / variables for parameters (part 2).
This CL covers simple ("simple") rest param cases.

BUG=v8:5516
R=vogelheim@chromium.org

Change-Id: I254c2eb81d759eb2ea2a3d5e7c46bcdc2ccef707
Reviewed-on: https://chromium-review.googlesource.com/440984
Reviewed-by: Daniel Vogelheim <vogelheim@chromium.org>
Commit-Queue: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43106}
2017-02-10 14:59:59 +00:00