This patch refactors the parser and preparser interface to be more
readable and type-safe. It has no behavior changes.
Previously, parsers and preparsers were configured via bitfield called
parser_flags in the Parser constructor, and flags in
PreParser::PreParseProgram, ParserApi::Parse, and ParserApi::PreParse.
This was error-prone in practice: six call sites passed incorrectly
typed values to this interface (a boolean FLAG value, a boolean false
and a boolean true value). None of these errors were caught by the
compiler because it's just an "int".
The parser flags interface was also awkward because it encoded a
language mode, but the language mode was only used to turn on harmony
scoping or not -- it wasn't used to actually set the parser's language
mode.
Fundamentally these errors came in because of the desire for a
procedural parser interface, in ParserApi. Because we need to be able
to configure the parser in various ways, the flags argument got added;
but no one understood how to use the flags properly. Also they were
only used by constructors: callers packed bits, and the constructors
unpacked them into booleans on the parser or preparser.
The solution is to allow parser construction, configuration, and
invocation to be separated. This patch does that.
It passes the existing tests.
BUG=
Review URL: https://codereview.chromium.org/13450007
Patch from Andy Wingo <wingo@igalia.com>.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14151 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
This patchset begins by adding support for "yield", which is unlike other tokens
in JS. In a generator, whether strict or classic, it is a syntactic keyword.
In classic mode it is an identifier. In strict mode it is reserved.
This patch adds YIELD as a token to the scanner, and adapts the preparser and
parser appropriately. It also parses "function*", indicating that a function is
actually a generator, for both eagerly and lazily parsed functions.
Currently "yield" just compiles as "return".
BUG=v8:2355
TEST=mjsunit/harmony/generators-parsing
Review URL: https://codereview.chromium.org/12646003
Patch from Andy Wingo <wingo@igalia.com>.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14116 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
* src/preparser.cc (PreParser::ParsePrimaryExpression): No need to check
keywords for validity as identifiers, as ParseIdentifier (called in
the fallthrough) will take care of it.
Covered by existing tests.
BUG=
Review URL: https://codereview.chromium.org/12600004
Patch from Andy Wingo <wingo@igalia.com>.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13899 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
This CL fixes the preparser to have the same liberal automatic semicolon
insertion behaviour as the parser. In the case of a return statement in
global code we throw a syntax error at runtime rather than an early error
due to compatibility with KJS. However that hack allowed the following
syntactically incorrect program in global code in the parser but not in
the preparser:
if (false) return else {}
while the slightly saner version with the obligatory semicolon
if (false) return; else {}
was disallowed in the parser, but the preparser allowed it. This CL also
fixes that issue.
BUG=v8:1856
TEST=cctest/test-parsing.cc
Review URL: http://codereview.chromium.org/8844002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@10201 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
The ES.next drafts require that source code that matches the productions for
let and const bindings outside the extended mode trigger early syntax
errors. This CL adapts the parser / preparser accordingly under the harmony
scoping flag.
Summary:
* Harmony scoping flag not set: Old semantics allowing const in classic mode
with function level scope. Const binding in strict mode and let bindings in
classic and strict mode trigger early syntax errors.
* Harmony scoping is set: Use new harmony const and let in
extended mode and old const in classic mode. This is to preserve
compatibility with current web pages that already use
non-standard implementations of const. An early syntax error is
thrown on const in strict mode and on let in classic and strict
mode.
This depends on:
http://codereview.chromium.org/8562002/
TEST=mjsunit/harmony/block-early-errors.js
Review URL: http://codereview.chromium.org/8564001
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@10079 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
This CL introduces a third mode next to the non-strict
(henceforth called 'classic mode') and 'strict mode'
which is called 'extended mode' as in the current
ES.next specification drafts. The extended mode is based on
the 'strict mode' and adds new functionality to it. This
means that most of the semantics of these two modes
coincide.
The 'extended mode' is entered instead of the 'strict mode'
during parsing when using the 'strict mode' directive
"use strict" and when the the harmony-scoping flag is
active. This should be changed once it is fully specified how the 'extended mode' is entered.
This change introduces a new 3 valued enum LanguageMode
(see globals.h) corresponding to the modes which is mostly
used by the frontend code. This includes the following
components:
* (Pre)Parser
* Compiler
* SharedFunctionInfo, Scope and ScopeInfo
* runtime functions: StoreContextSlot,
ResolvePossiblyDirectEval, InitializeVarGlobal,
DeclareGlobals
The old enum StrictModeFlag is still used in the backend
when the distinction between the 'strict mode' and the 'extended mode' does not matter. This includes:
* SetProperty runtime function, Delete builtin
* StoreIC and KeyedStoreIC
* StubCache
Review URL: http://codereview.chromium.org/8417035
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@10062 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
This implements block scoped 'const' declared variables in harmony mode. They
have a temporal dead zone semantics similar to 'let' bindings, i.e. accessing
uninitialized 'const' bindings in throws a ReferenceError.
As for 'let' bindings, the semantics of 'const' bindings in global scope is not
correctly implemented yet. Furthermore assignments to 'const's are silently
ignored. Another CL will introduce treatment of those assignments as early
errors.
Review URL: http://codereview.chromium.org/7992005
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@9764 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
Previously the preparser always accepted natives syntax and let the
real parser throw the syntax error. In ES5, it should be an early error,
so the preparser must catch the error.
The perparser library does not expose parsing for natives syntax, it's
only used internally.
Review URL: http://codereview.chromium.org/8306024
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@9660 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
The preparser has been out of sync with the parser. As a reminder, we have the
following grammer for harmony mode
Block ::
{ SourceElement* }
SourceElement ::
Statement
FunctionDeclaration
LetDeclaration
instead of
Block ::
{ Statement* }
SourceElement ::
Statement
FunctionDeclaration
The extension to allow FunctionDeclarations in statement positions in
non-strict code is still active.
Review URL: http://codereview.chromium.org/7983006
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@9363 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
Duplicate identifier detection must be an early syntax error in strict code,
so errors in otherwise lazily compiled functions must be caught in the
preparser.
Originally introduced in r8541 and reverted in r8542.
Now really compiles on Windows.
Review URL: http://codereview.chromium.org/7782023
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@9172 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
This is a fix and reapply of r8516 with some comments addressed and more
tests added.
The difference from r8516 is that canonicalization of number literals is
no performed using the same methods as in v8, to avoid false positives/negatives
when detecting duplicates.
Review URL: http://codereview.chromium.org/7193045
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8541 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
A multi-line comment containing a newline is considered a line-terminator for
other purposes, but a "-->" following such a comment is considered as being
on the same line as the text preceeding the multi-line comment.
This behavior matches JSC matching Firefox.
TEST=cctest/test-parsing/ScanHTMLEndComments
Review URL: http://codereview.chromium.org/7218009
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8351 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
We now only recognize "native function" when it occurs in extension scripts
(parsing with a non-NULL extension), and only if there is no line-terminator
between "native" and "function" (so that it would otherwise be a Syntax Error).
Preparsing never recognizes native functions, which is acceptable since we
never preparse extension scripts (because we don't allow lazy functions
anyway).
BUG=v8:1097
Review URL: http://codereview.chromium.org/7206020
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8326 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
It should now be possible to build the preparser using 'scons preparser' in both release and debug modes.
Remove v8.h include from scanner-base.h and other files.
Remove NativeAllocationChecker and all of its kind.
Moved Isolate::PreallocatedStorage* to isolate.cc
Review URL: http://codereview.chromium.org/6749029
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7413 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
