AuroraMiddleware/v8
1
0
Fork 0
Code Issues 2 Pull Requests Releases Wiki Activity

[parser] Replacing ExpressionClassifier with ExpressionScope that knows what it's tracking

Browse Source 
Since it's explicit what we're tracking, we can immediately throw errors in
certain cases, and ignore irrelevant errors. We don't need to use the
classifier itself to track "let let", since we know whether we're parsing a
"let". Errors that were previously (almost) always accumulated are now
immediately pushed to the scopes that care (parameter initialization errors).

This CL drops avoiding allocation of classified errors, at least for now, but
that doesn't affect performance anymore since we don't aggressively blacklist
anymore. Classified errors are even less likely with the more precise approach.

ParseAssignmentExpression doesn't introduce its own scope immediately, but
reuses the outer scope.

Rather than using full ExpressionClassifiers + Accumulate to separate
expressions/patterns from each other while keeping track of the overall error
state, this now uses an explicit AccumulationScope.

When we parse (async) arrow functions we introduce new scopes
that track that they may be (async) arrow functions.

We track StrictModeFormal parameters in 2 different ways if it isn't
immediately certain that it is a strict-mode formal error: Either directly on
the (Pre)ParserFormalParameters, or on the NextArrowFunctionInfo in the case
we're not yet certain that we'll have an arrow function. In the latter case we
don't have a FormalParameter object yet, and we'll copy it over once we know
we're parsing an arrow function. The latter works because it's not allowed to
change strictness of a function with non-simple parameters.

Design doc:
https://docs.google.com/document/d/1FAvEp9EUK-G8kHfDIEo_385Hs2SUBCYbJ5H-NnLvq8M/

Change-Id: If4ecd717c9780095c7ddc859c8945b3d7d268a9d
Reviewed-on: https://chromium-review.googlesource.com/c/1367809
Commit-Queue: Toon Verwaest <verwaest@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Reviewed-by: Marja Hölttä <marja@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58307}
This commit is contained in:
Toon Verwaest 2018-12-17 10:28:27 +01:00 committed by Commit Bot
parent 14ebea15a4
commit d1b4e31bc5
12 changed files with 948 additions and 878 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
BUILD.gn
View File

@ -2433,9 +2433,9 @@ v8_source_set("v8_base") {
"src/optimized-compilation-info.h",
"src/ostreams.cc",
"src/ostreams.h",
"src/parsing/expression-classifier.h",
"src/parsing/expression-scope-reparenter.cc",
"src/parsing/expression-scope-reparenter.h",
"src/parsing/expression-scope.h",
"src/parsing/func-name-inferrer.cc",
"src/parsing/func-name-inferrer.h",
"src/parsing/parse-info.cc",

1
src/ast/scopes.cc
View File

@ -844,6 +844,7 @@ void DeclarationScope::AddLocal(Variable* var) {
}
void Scope::Snapshot::Reparent(DeclarationScope* new_parent) {
DCHECK(!IsCleared());
DCHECK_EQ(new_parent, outer_scope_and_calls_eval_.GetPointer()->inner_scope_);
DCHECK_EQ(new_parent->outer_scope_, outer_scope_and_calls_eval_.GetPointer());
DCHECK_EQ(new_parent, new_parent->GetClosureScope());

459
src/parsing/expression-classifier.h
View File

@ -1,459 +0,0 @@
// Copyright 2015 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef V8_PARSING_EXPRESSION_CLASSIFIER_H_
#define V8_PARSING_EXPRESSION_CLASSIFIER_H_
#include <type_traits>
#include "src/message-template.h"
#include "src/parsing/scanner.h"
namespace v8 {
namespace internal {
template <typename T>
class ZoneList;
#define ERROR_CODES(T) \
T(ExpressionProduction, 0) \
T(FormalParameterInitializerProduction, 1) \
T(PatternProduction, 2) \
T(BindingPatternProduction, 3) \
T(StrictModeFormalParametersProduction, 4) \
T(LetPatternProduction, 5) \
T(AsyncArrowFormalParametersProduction, 6)
// Expression classifiers serve two purposes:
//
// 1) They keep track of error messages that are pending (and other
// related information), waiting for the parser to decide whether
// the parsed expression is a pattern or not.
// 2) They keep track of expressions that may need to be rewritten, if
// the parser decides that they are not patterns. (A different
// mechanism implements the rewriting of patterns.)
//
// Expression classifiers are used by the parser in a stack fashion.
// Each new classifier is pushed on top of the stack. This happens
// automatically by the class's constructor. While on top of the
// stack, the classifier records pending error messages and tracks the
// pending non-patterns of the expression that is being parsed.
//
// At the end of its life, a classifier is either "accumulated" to the
// one that is below it on the stack, or is "discarded". The former
// is achieved by calling the method Accumulate. The latter is
// achieved automatically by the destructor, but it can happen earlier
// by calling the method Discard. Both actions result in removing the
// classifier from the parser's stack.
// Expression classifier is split into four parts. The base implementing the
// general expression classifier logic. Two parts that implement the error
// tracking interface, where one is the actual implementation and the other is
// an empty class providing only the interface without logic. The expression
// classifier class then combines the other parts and provides the full
// expression classifier interface by inheriting conditionally, controlled by
// Types::ExpressionClassifierReportErrors, either from the ErrorTracker or the
// EmptyErrorTracker.
//
// Base
// / \
// / \
// / \
// / \
// ErrorTracker EmptyErrorTracker
// \ /
// \ /
// \ /
// \ /
// ExpressionClassifier
template <typename Types>
class ExpressionClassifier;
template <typename Types, typename ErrorTracker>
class ExpressionClassifierBase {
public:
enum ErrorKind : unsigned {
#define DEFINE_ERROR_KIND(NAME, CODE) k##NAME = CODE,
ERROR_CODES(DEFINE_ERROR_KIND)
#undef DEFINE_ERROR_KIND
kUnusedError = 15 // Larger than error codes; should fit in 4 bits
};
struct Error {
V8_INLINE Error()
: location(Scanner::Location::invalid()),
message_(static_cast<int>(MessageTemplate::kNone)),
kind(kUnusedError) {}
V8_INLINE explicit Error(Scanner::Location loc, MessageTemplate msg,
ErrorKind k, const char* a = nullptr)
: location(loc), message_(static_cast<int>(msg)), kind(k) {}
Scanner::Location location;
// GCC doesn't like storing the enum class directly in 28 bits, so we
// have to wrap it in a getter.
MessageTemplate message() const {
STATIC_ASSERT(static_cast<int>(MessageTemplate::kLastMessage) <
(1 << 28));
return static_cast<MessageTemplate>(message_);
}
int message_ : 28;
unsigned kind : 4;
};
// clang-format off
enum TargetProduction : unsigned {
#define DEFINE_PRODUCTION(NAME, CODE) NAME = 1 << CODE,
ERROR_CODES(DEFINE_PRODUCTION)
#undef DEFINE_PRODUCTION
#define DEFINE_ALL_PRODUCTIONS(NAME, CODE) NAME |
AllProductions = ERROR_CODES(DEFINE_ALL_PRODUCTIONS) /* | */ 0
#undef DEFINE_ALL_PRODUCTIONS
};
// clang-format on
explicit ExpressionClassifierBase(typename Types::Base* base)
: base_(base),
invalid_productions_(0),
is_non_simple_parameter_list_(0) {}
virtual ~ExpressionClassifierBase() = default;
V8_INLINE bool is_valid(unsigned productions) const {
return (invalid_productions_ & productions) == 0;
}
V8_INLINE bool is_valid_expression() const {
return is_valid(ExpressionProduction);
}
V8_INLINE bool is_valid_formal_parameter_initializer() const {
return is_valid(FormalParameterInitializerProduction);
}
V8_INLINE bool is_valid_pattern() const {
return is_valid(PatternProduction);
}
V8_INLINE bool is_valid_binding_pattern() const {
return is_valid(BindingPatternProduction);
}
// Note: callers should also check
// is_valid_formal_parameter_list_without_duplicates().
V8_INLINE bool is_valid_strict_mode_formal_parameters() const {
return is_valid(StrictModeFormalParametersProduction);
}
V8_INLINE bool is_valid_let_pattern() const {
return is_valid(LetPatternProduction);
}
bool is_valid_async_arrow_formal_parameters() const {
return is_valid(AsyncArrowFormalParametersProduction);
}
V8_INLINE bool is_simple_parameter_list() const {
return !is_non_simple_parameter_list_;
}
V8_INLINE void RecordNonSimpleParameter() {
is_non_simple_parameter_list_ = 1;
}
V8_INLINE void Accumulate(ExpressionClassifier<Types>* const inner,
unsigned productions) {
#ifdef DEBUG
static_cast<ErrorTracker*>(this)->CheckErrorPositions(inner);
#endif
// Propagate errors from inner, but don't overwrite already recorded
// errors.
unsigned filter = productions & ~this->invalid_productions_;
unsigned errors = inner->invalid_productions_ & filter;
static_cast<ErrorTracker*>(this)->AccumulateErrorImpl(inner, productions,
errors);
this->invalid_productions_ |= errors;
}
protected:
typename Types::Base* base_;
unsigned invalid_productions_ : kUnusedError;
STATIC_ASSERT(kUnusedError <= 15);
unsigned is_non_simple_parameter_list_ : 1;
};
template <typename Types>
class ExpressionClassifierErrorTracker
: public ExpressionClassifierBase<Types,
ExpressionClassifierErrorTracker<Types>> {
public:
using BaseClassType =
ExpressionClassifierBase<Types, ExpressionClassifierErrorTracker<Types>>;
using typename BaseClassType::Error;
using typename BaseClassType::ErrorKind;
using TP = typename BaseClassType::TargetProduction;
explicit ExpressionClassifierErrorTracker(typename Types::Base* base)
: BaseClassType(base),
reported_errors_(base->impl()->GetReportedErrorList()) {
reported_errors_begin_ = reported_errors_end_ = reported_errors_->length();
}
~ExpressionClassifierErrorTracker() override {
if (reported_errors_end_ == reported_errors_->length()) {
reported_errors_->Rewind(reported_errors_begin_);
reported_errors_end_ = reported_errors_begin_;
}
DCHECK_EQ(reported_errors_begin_, reported_errors_end_);
}
protected:
V8_INLINE const Error& reported_error(ErrorKind kind) const {
if (!this->is_valid(1 << kind)) {
for (int i = reported_errors_begin_; i < reported_errors_end_; i++) {
if (reported_errors_->at(i).kind == kind)
return reported_errors_->at(i);
}
UNREACHABLE();
}
// We should only be looking for an error when we know that one has
// been reported. But we're not... So this is to make sure we have
// the same behaviour.
UNREACHABLE();
// Make MSVC happy by returning an error from this inaccessible path.
static Error none;
return none;
}
// Adds e to the end of the list of reported errors for this classifier.
// It is expected that this classifier is the last one in the stack.
V8_INLINE void Add(TP production, const Error& e) {
if (!this->is_valid(production)) return;
this->invalid_productions_ |= production;
DCHECK_EQ(reported_errors_end_, reported_errors_->length());
reported_errors_->Add(e, this->base_->impl()->zone());
reported_errors_end_++;
}
// Copies the error at position i of the list of reported errors, so that
// it becomes the last error reported for this classifier. Position i
// could be either after the existing errors of this classifier (i.e.,
// in an inner classifier) or it could be an existing error (in case a
// copy is needed).
V8_INLINE void Copy(int i) {
DCHECK_LT(i, reported_errors_->length());
if (reported_errors_end_ != i)
reported_errors_->at(reported_errors_end_) = reported_errors_->at(i);
reported_errors_end_++;
}
private:
#ifdef DEBUG
V8_INLINE void CheckErrorPositions(ExpressionClassifier<Types>* const inner) {
DCHECK_EQ(inner->reported_errors_, this->reported_errors_);
DCHECK_EQ(inner->reported_errors_begin_, this->reported_errors_end_);
DCHECK_EQ(inner->reported_errors_end_, this->reported_errors_->length());
}
#endif
V8_INLINE void RewindErrors(ExpressionClassifier<Types>* const inner) {
this->reported_errors_->Rewind(this->reported_errors_end_);
inner->reported_errors_begin_ = inner->reported_errors_end_ =
this->reported_errors_end_;
}
void AccumulateErrorImpl(ExpressionClassifier<Types>* const inner,
unsigned productions, unsigned errors) {
// Traverse the list of errors reported by the inner classifier
// to copy what's necessary.
for (int i = inner->reported_errors_begin_; errors != 0; i++) {
int mask = 1 << this->reported_errors_->at(i).kind;
if ((errors & mask) != 0) {
errors ^= mask;
this->Copy(i);
}
}
RewindErrors(inner);
}
private:
ZoneList<Error>* reported_errors_;
// The uint16_t for reported_errors_begin_ and reported_errors_end_ will
// not be enough in the case of a long series of expressions using nested
// classifiers, e.g., a long sequence of assignments, as in:
// literals with spreads, as in:
// var N=65536; eval("var x;" + "x=".repeat(N) + "42");
// This should not be a problem, as such things currently fail with a
// stack overflow while parsing.
uint16_t reported_errors_begin_;
uint16_t reported_errors_end_;
friend BaseClassType;
};
template <typename Types>
class ExpressionClassifierEmptyErrorTracker
: public ExpressionClassifierBase<
Types, ExpressionClassifierEmptyErrorTracker<Types>> {
public:
using BaseClassType =
ExpressionClassifierBase<Types,
ExpressionClassifierEmptyErrorTracker<Types>>;
using typename BaseClassType::Error;
using typename BaseClassType::ErrorKind;
using TP = typename BaseClassType::TargetProduction;
explicit ExpressionClassifierEmptyErrorTracker(typename Types::Base* base)
: BaseClassType(base) {}
protected:
V8_INLINE const Error& reported_error(ErrorKind kind) const {
static Error none;
return none;
}
V8_INLINE void Add(TP production, const Error& e) {
this->invalid_productions_ |= production;
}
private:
#ifdef DEBUG
V8_INLINE void CheckErrorPositions(ExpressionClassifier<Types>* const inner) {
}
#endif
V8_INLINE void AccumulateErrorImpl(ExpressionClassifier<Types>* const inner,
unsigned productions, unsigned errors) {}
friend BaseClassType;
};
template <typename Types>
class ExpressionClassifier
: public std::conditional<
Types::ExpressionClassifierReportErrors,
ExpressionClassifierErrorTracker<Types>,
ExpressionClassifierEmptyErrorTracker<Types>>::type {
static constexpr bool ReportErrors = Types::ExpressionClassifierReportErrors;
public:
using BaseClassType = typename std::conditional<
Types::ExpressionClassifierReportErrors,
typename ExpressionClassifierErrorTracker<Types>::BaseClassType,
typename ExpressionClassifierEmptyErrorTracker<Types>::BaseClassType>::
type;
using typename BaseClassType::Error;
using typename BaseClassType::ErrorKind;
using TP = typename BaseClassType::TargetProduction;
explicit ExpressionClassifier(typename Types::Base* base)
: std::conditional<
Types::ExpressionClassifierReportErrors,
ExpressionClassifierErrorTracker<Types>,
ExpressionClassifierEmptyErrorTracker<Types>>::type(base),
previous_(base->classifier_) {
base->classifier_ = this;
}
V8_INLINE ~ExpressionClassifier() override {
if (this->base_->classifier_ == this) this->base_->classifier_ = previous_;
}
V8_INLINE const Error& expression_error() const {
return this->reported_error(ErrorKind::kExpressionProduction);
}
V8_INLINE const Error& formal_parameter_initializer_error() const {
return this->reported_error(
ErrorKind::kFormalParameterInitializerProduction);
}
V8_INLINE const Error& pattern_error() const {
return this->reported_error(ErrorKind::kPatternProduction);
}
V8_INLINE const Error& binding_pattern_error() const {
return this->reported_error(ErrorKind::kBindingPatternProduction);
}
V8_INLINE const Error& strict_mode_formal_parameter_error() const {
return this->reported_error(
ErrorKind::kStrictModeFormalParametersProduction);
}
V8_INLINE const Error& let_pattern_error() const {
return this->reported_error(ErrorKind::kLetPatternProduction);
}
V8_INLINE const Error& async_arrow_formal_parameters_error() const {
return this->reported_error(
ErrorKind::kAsyncArrowFormalParametersProduction);
}
V8_INLINE bool does_error_reporting() { return ReportErrors; }
void RecordExpressionError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(TP::ExpressionProduction,
Error(loc, message, ErrorKind::kExpressionProduction));
}
void RecordFormalParameterInitializerError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(
TP::FormalParameterInitializerProduction,
Error(loc, message, ErrorKind::kFormalParameterInitializerProduction));
}
void RecordPatternError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(TP::PatternProduction,
Error(loc, message, ErrorKind::kPatternProduction));
}
void RecordBindingPatternError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(TP::BindingPatternProduction,
Error(loc, message, ErrorKind::kBindingPatternProduction));
}
void RecordAsyncArrowFormalParametersError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(
TP::AsyncArrowFormalParametersProduction,
Error(loc, message, ErrorKind::kAsyncArrowFormalParametersProduction));
}
// Record a binding that would be invalid in strict mode. Confusingly this
// is not the same as StrictFormalParameterList, which simply forbids
// duplicate bindings.
void RecordStrictModeFormalParameterError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(
TP::StrictModeFormalParametersProduction,
Error(loc, message, ErrorKind::kStrictModeFormalParametersProduction));
}
void RecordLetPatternError(const Scanner::Location& loc,
MessageTemplate message) {
this->Add(TP::LetPatternProduction,
Error(loc, message, ErrorKind::kLetPatternProduction));
}
ExpressionClassifier* previous() const { return previous_; }
private:
ExpressionClassifier* previous_;
DISALLOW_COPY_AND_ASSIGN(ExpressionClassifier);
};
#undef ERROR_CODES
} // namespace internal
} // namespace v8
#endif // V8_PARSING_EXPRESSION_CLASSIFIER_H_

516
src/parsing/expression-scope.h Normal file
View File

@ -0,0 +1,516 @@
// Copyright 2018 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef V8_PARSING_EXPRESSION_SCOPE_H_
#define V8_PARSING_EXPRESSION_SCOPE_H_
#include "src/message-template.h"
#include "src/parsing/scanner.h"
namespace v8 {
namespace internal {
template <typename Types>
class ExpressionParsingScope;
template <typename Types>
class AccumulationScope;
template <typename Types>
class ArrowHeadParsingScope;
// ExpressionScope is used in a stack fashion, and is used to specialize
// expression parsing for the task at hand. It allows the parser to reuse the
// same code to parse destructuring declarations, assignment patterns,
// expressions, and (async) arrow function heads.
//
// One of the specific subclasses needs to be instantiated to tell the parser
// the meaning of the expression it will parse next. The parser then calls
// Record* on the expression_scope() to indicate errors. The expression_scope
// will either discard those errors, immediately report those errors, or
// classify the errors for later validation.
// TODO(verwaest): Record is a slightly odd name since it will directly throw
// for unambiguous scopes.
template <typename Types>
class ExpressionScope {
public:
typedef typename Types::Impl ParserT;
typedef typename Types::Expression ExpressionT;
enum ScopeType : uint8_t {
// Expression or assignment target.
kExpression,
// Declaration or expression or assignment target.
kMaybeArrowParameterDeclaration,
kMaybeAsyncArrowParameterDeclaration,
// Declarations.
kParameterDeclaration,
kVarDeclaration,
kLexicalDeclaration,
};
void ValidateAsPattern(ExpressionT expression, int begin, int end) {
if (!CanBeExpression()) return;
AsExpressionParsingScope()->ValidatePattern(expression, begin, end);
AsExpressionParsingScope()->ClearExpressionError();
}
// Record async arrow parameters errors in all ambiguous async arrow scopes in
// the chain up to the first unambiguous scope.
void RecordAsyncArrowParametersError(const Scanner::Location& loc,
MessageTemplate message) {
// Only ambiguous scopes (ExpressionParsingScope, *ArrowHeadParsingScope)
// need to propagate errors to a possible kAsyncArrowHeadParsingScope, so
// immediately return if the current scope is not ambiguous.
if (!CanBeExpression()) return;
AsExpressionParsingScope()->RecordAsyncArrowParametersError(loc, message);
}
// Record initializer errors in all scopes that can turn into parameter scopes
// (ArrowHeadParsingScopes) up to the first known unambiguous parameter scope.
void RecordParameterInitializerError(const Scanner::Location& loc,
MessageTemplate message) {
ExpressionScope* scope = this;
while (!scope->IsCertainlyParameterDeclaration()) {
if (!has_possible_parameter_in_scope_chain_) return;
if (scope->CanBeParameterDeclaration()) {
scope->AsArrowHeadParsingScope()->RecordDeclarationError(loc, message);
}
scope = scope->parent();
if (scope == nullptr) return;
}
Report(loc, message);
}
void RecordPatternError(const Scanner::Location& loc,
MessageTemplate message) {
// TODO(verwaest): Non-assigning expression?
if (IsCertainlyPattern()) {
Report(loc, message);
} else {
AsExpressionParsingScope()->RecordPatternError(loc, message);
}
}
void RecordStrictModeParameterError(const Scanner::Location& loc,
MessageTemplate message) {
DCHECK_IMPLIES(!has_error(), loc.IsValid());
if (!CanBeParameterDeclaration()) return;
if (IsCertainlyParameterDeclaration()) {
if (is_strict(parser_->language_mode())) {
Report(loc, message);
} else {
parser_->parameters_->set_strict_parameter_error(loc, message);
}
} else {
parser_->next_arrow_function_info_.strict_parameter_error_location = loc;
parser_->next_arrow_function_info_.strict_parameter_error_message =
message;
}
}
void RecordDeclarationError(const Scanner::Location& loc,
MessageTemplate message) {
if (!CanBeDeclaration()) return;
if (IsCertainlyDeclaration()) {
Report(loc, message);
} else {
AsArrowHeadParsingScope()->RecordDeclarationError(loc, message);
}
}
void RecordExpressionError(const Scanner::Location& loc,
MessageTemplate message) {
if (!CanBeExpression()) return;
// TODO(verwaest): Non-assigning expression?
// if (IsCertainlyExpression()) Report(loc, message);
AsExpressionParsingScope()->RecordExpressionError(loc, message);
}
void RecordLexicalDeclarationError(const Scanner::Location& loc,
MessageTemplate message) {
if (IsLexicalDeclaration()) Report(loc, message);
}
void RecordNonSimpleParameter() {
if (!IsArrowHeadParsingScope()) return;
AsArrowHeadParsingScope()->RecordNonSimpleParameter();
}
protected:
ParserT* parser() const { return parser_; }
ExpressionScope* parent() const { return parent_; }
void Report(const Scanner::Location& loc, MessageTemplate message) const {
parser_->ReportMessageAt(loc, message);
}
ExpressionScope(ParserT* parser, ScopeType type)
: parser_(parser),
parent_(parser->expression_scope_),
type_(type),
has_possible_parameter_in_scope_chain_(
CanBeParameterDeclaration() ||
(parent_ && parent_->has_possible_parameter_in_scope_chain_)) {
parser->expression_scope_ = this;
}
~ExpressionScope() {
DCHECK(parser_->expression_scope_ == this ||
parser_->expression_scope_ == parent_);
parser_->expression_scope_ = parent_;
}
ExpressionParsingScope<Types>* AsExpressionParsingScope() {
DCHECK(CanBeExpression());
return static_cast<ExpressionParsingScope<Types>*>(this);
}
#ifdef DEBUG
bool has_error() const { return parser_->has_error(); }
#endif
bool CanBeExpression() const {
return IsInRange(type_, kExpression, kMaybeAsyncArrowParameterDeclaration);
}
bool CanBeDeclaration() const {
return IsInRange(type_, kMaybeArrowParameterDeclaration,
kLexicalDeclaration);
}
bool IsCertainlyDeclaration() const {
return IsInRange(type_, kParameterDeclaration, kLexicalDeclaration);
}
private:
friend class AccumulationScope<Types>;
friend class ExpressionParsingScope<Types>;
ArrowHeadParsingScope<Types>* AsArrowHeadParsingScope() {
DCHECK(IsArrowHeadParsingScope());
return static_cast<ArrowHeadParsingScope<Types>*>(this);
}
bool IsArrowHeadParsingScope() const {
return IsInRange(type_, kMaybeArrowParameterDeclaration,
kMaybeAsyncArrowParameterDeclaration);
}
bool IsCertainlyPattern() const { return IsCertainlyDeclaration(); }
bool CanBeParameterDeclaration() const {
return IsInRange(type_, kMaybeArrowParameterDeclaration,
kParameterDeclaration);
}
bool IsCertainlyParameterDeclaration() const {
return type_ == kParameterDeclaration;
}
bool IsLexicalDeclaration() const { return type_ == kLexicalDeclaration; }
ParserT* parser_;
ExpressionScope<Types>* parent_;
ScopeType type_;
bool has_possible_parameter_in_scope_chain_;
DISALLOW_COPY_AND_ASSIGN(ExpressionScope);
};
// Used to parse var, let, const declarations and declarations known up-front to
// be parameters.
template <typename Types>
class DeclarationParsingScope : public ExpressionScope<Types> {
public:
typedef typename Types::Impl ParserT;
typedef typename ExpressionScope<Types>::ScopeType ScopeType;
DeclarationParsingScope(ParserT* parser, ScopeType type)
: ExpressionScope<Types>(parser, type) {
DCHECK(this->IsCertainlyDeclaration());
}
private:
DISALLOW_COPY_AND_ASSIGN(DeclarationParsingScope);
};
// Parsing expressions is always ambiguous between at least left-hand-side and
// right-hand-side of assignments. This class is used to keep track of errors
// relevant for either side until it is clear what was being parsed.
template <typename Types>
class ExpressionParsingScope : public ExpressionScope<Types> {
public:
typedef typename Types::Impl ParserT;
typedef typename Types::Expression ExpressionT;
typedef class ExpressionScope<Types> ExpressionScopeT;
typedef typename ExpressionScopeT::ScopeType ScopeType;
ExpressionParsingScope(ParserT* parser,
ScopeType type = ExpressionScopeT::kExpression)
: ExpressionScopeT(parser, type),
has_async_arrow_in_scope_chain_(
type == ExpressionScopeT::kMaybeAsyncArrowParameterDeclaration ||
(this->parent() && this->parent()->CanBeExpression() &&
this->parent()
->AsExpressionParsingScope()
->has_async_arrow_in_scope_chain_)) {
DCHECK(this->CanBeExpression());
clear(kExpressionIndex);
clear(kPatternIndex);
}
void RecordAsyncArrowParametersError(const Scanner::Location& loc,
MessageTemplate message) {
for (ExpressionScopeT* scope = this; scope != nullptr;
scope = scope->parent()) {
if (!has_async_arrow_in_scope_chain_) break;
if (scope->type_ ==
ExpressionScopeT::kMaybeAsyncArrowParameterDeclaration) {
scope->AsArrowHeadParsingScope()->RecordDeclarationError(loc, message);
}
}
}
~ExpressionParsingScope() { DCHECK(this->has_error() || verified_); }
ExpressionT ValidateAndRewriteReference(ExpressionT expression, int beg_pos,
int end_pos) {
if (V8_LIKELY(this->parser()->IsAssignableIdentifier(expression))) {
this->mark_verified();
return expression;
} else if (V8_LIKELY(expression->IsProperty())) {
ValidateExpression();
return expression;
}
this->mark_verified();
return this->parser()->RewriteInvalidReferenceExpression(
expression, beg_pos, end_pos, MessageTemplate::kInvalidLhsInFor,
kSyntaxError);
}
void RecordExpressionError(const Scanner::Location& loc,
MessageTemplate message) {
Record(kExpressionIndex, loc, message);
}
void RecordPatternError(const Scanner::Location& loc,
MessageTemplate message) {
Record(kPatternIndex, loc, message);
}
void ValidateExpression() { Validate(kExpressionIndex); }
void ValidatePattern(ExpressionT expression, int begin, int end) {
Validate(kPatternIndex);
if (expression->is_parenthesized()) {
ExpressionScopeT::Report(Scanner::Location(begin, end),
MessageTemplate::kInvalidDestructuringTarget);
}
}
void ClearExpressionError() {
DCHECK(verified_);
#ifdef DEBUG
verified_ = false;
#endif
clear(kExpressionIndex);
}
protected:
bool is_verified() const {
#ifdef DEBUG
return verified_;
#else
return false;
#endif
}
void ValidatePattern() { Validate(kPatternIndex); }
private:
friend class AccumulationScope<Types>;
enum ErrorNumber : uint8_t {
kExpressionIndex = 0,
kPatternIndex = 1,
kNumberOfErrors = 2,
};
void clear(int index) {
messages_[index] = MessageTemplate::kNone;
locations_[index] = Scanner::Location::invalid();
}
bool is_valid(int index) const { return !locations_[index].IsValid(); }
void Record(int index, const Scanner::Location& loc,
MessageTemplate message) {
DCHECK_IMPLIES(!this->has_error(), loc.IsValid());
if (!is_valid(index)) return;
messages_[index] = message;
locations_[index] = loc;
}
void Validate(int index) {
DCHECK(!this->is_verified());
if (!is_valid(index)) Report(index);
this->mark_verified();
}
void Report(int index) const {
ExpressionScopeT::Report(locations_[index], messages_[index]);
}
// Debug verification to make sure every scope is validated exactly once.
void mark_verified() {
#ifdef DEBUG
verified_ = true;
#endif
}
void clear_verified() {
#ifdef DEBUG
verified_ = false;
#endif
}
#ifdef DEBUG
bool verified_ = false;
#endif
MessageTemplate messages_[kNumberOfErrors];
Scanner::Location locations_[kNumberOfErrors];
bool has_async_arrow_in_scope_chain_;
DISALLOW_COPY_AND_ASSIGN(ExpressionParsingScope);
};
// This class is used to parse multiple ambiguous expressions and declarations
// in the same scope. E.g., in async(X,Y,Z) or [X,Y,Z], X and Y and Z will all
// be parsed in the respective outer ArrowHeadParsingScope and
// ExpressionParsingScope. It provides a clean error state in the underlying
// scope to parse the individual expressions, while keeping track of the
// expression and pattern errors since the start. The AccumulationScope is only
// used to keep track of the errors so far, and the underlying ExpressionScope
// keeps being used as the expression_scope(). If the expression_scope() isn't
// ambiguous, this class does not do anything.
template <typename Types>
class AccumulationScope {
public:
typedef typename Types::Impl ParserT;
static const int kNumberOfErrors =
ExpressionParsingScope<Types>::kNumberOfErrors;
explicit AccumulationScope(ExpressionScope<Types>* scope) : scope_(nullptr) {
if (!scope->CanBeExpression()) return;
scope_ = scope->AsExpressionParsingScope();
for (int i = 0; i < kNumberOfErrors; i++) {
// If the underlying scope is already invalid at the start, stop
// accumulating. That means an error was found outside of an
// accumulating path.
if (!scope_->is_valid(i)) {
scope_ = nullptr;
break;
}
copy(i);
}
}
// Merge errors from the underlying ExpressionParsingScope into this scope.
// Only keeps the first error across all accumulate calls, and removes the
// error from the underlying scope.
void Accumulate() {
if (scope_ == nullptr) return;
DCHECK(!scope_->is_verified());
for (int i = 0; i < kNumberOfErrors; i++) {
if (!locations_[i].IsValid()) copy(i);
scope_->clear(i);
}
}
// This is called instead of Accumulate in case the parsed member is already
// known to be an expression. In that case we don't need to accumulate the
// expression but rather validate it immediately. We also ignore the pattern
// error since the parsed member is known to not be a pattern. This is
// necessary for "{x:1}.y" parsed as part of an assignment pattern. {x:1} will
// record a pattern error, but "{x:1}.y" is actually a valid as part of an
// assignment pattern since it's a property access.
void ValidateExpression() {
if (scope_ == nullptr) return;
DCHECK(!scope_->is_verified());
scope_->ValidateExpression();
DCHECK(scope_->is_verified());
scope_->clear(ExpressionParsingScope<Types>::kPatternIndex);
#ifdef DEBUG
scope_->clear_verified();
#endif
}
~AccumulationScope() {
if (scope_ == nullptr) return;
for (int i = 0; i < kNumberOfErrors; i++) copy_back(i);
}
private:
void copy(int entry) {
messages_[entry] = scope_->messages_[entry];
locations_[entry] = scope_->locations_[entry];
}
void copy_back(int entry) {
if (!locations_[entry].IsValid()) return;
scope_->messages_[entry] = messages_[entry];
scope_->locations_[entry] = locations_[entry];
}
ExpressionParsingScope<Types>* scope_;
MessageTemplate messages_[2];
Scanner::Location locations_[2];
DISALLOW_COPY_AND_ASSIGN(AccumulationScope);
};
// The head of an arrow function is ambiguous between expression, assignment
// pattern and declaration. This keeps track of the additional declaration
// error and allows the scope to be validated as a declaration rather than an
// expression or a pattern.
template <typename Types>
class ArrowHeadParsingScope : public ExpressionParsingScope<Types> {
public:
typedef typename Types::Impl ParserT;
typedef typename ExpressionScope<Types>::ScopeType ScopeType;
ArrowHeadParsingScope(ParserT* parser, ScopeType type)
: ExpressionParsingScope<Types>(parser, type) {
DCHECK(this->CanBeDeclaration());
DCHECK(!this->IsCertainlyDeclaration());
}
void ValidateExpression() {
// Turns out this is not an arrow head. Clear any possible tracked strict
// parameter errors.
this->parser()->next_arrow_function_info_.ClearStrictParameterError();
ExpressionParsingScope<Types>::ValidateExpression();
}
void ValidateDeclaration() {
DCHECK(!this->is_verified());
if (declaration_error_location.IsValid()) {
ExpressionScope<Types>::Report(declaration_error_location,
declaration_error_message);
}
this->ValidatePattern();
}
void RecordDeclarationError(const Scanner::Location& loc,
MessageTemplate message) {
DCHECK_IMPLIES(!this->has_error(), loc.IsValid());
declaration_error_location = loc;
declaration_error_message = message;
}
bool has_simple_parameter_list() const { return has_simple_parameter_list_; }
void RecordNonSimpleParameter() { has_simple_parameter_list_ = false; }
private:
Scanner::Location declaration_error_location = Scanner::Location::invalid();
MessageTemplate declaration_error_message = MessageTemplate::kNone;
bool has_simple_parameter_list_ = true;
DISALLOW_COPY_AND_ASSIGN(ArrowHeadParsingScope);
};
} // namespace internal
} // namespace v8
#endif // V8_PARSING_EXPRESSION_SCOPE_H_

743
src/parsing/parser-base.h
View File

File diff suppressed because it is too large Load Diff

21
src/parsing/parser.cc
View File

@ -758,12 +758,13 @@ FunctionLiteral* Parser::DoParseFunction(Isolate* isolate, ParseInfo* info,
SetLanguageMode(scope, info->language_mode());
scope->set_start_position(info->start_position());
ExpressionClassifier formals_classifier(this);
ParserFormalParameters formals(scope);
// The outer FunctionState should not contain destructuring assignments.
DCHECK_EQ(0,
function_state.destructuring_assignments_to_rewrite().size());
{
DeclarationParsingScope formals_scope(
this, ExpressionScope::kParameterDeclaration);
// Parsing patterns as variable reference expression creates
// NewUnresolved references in current scope. Enter arrow function
// scope for formal parameter parsing.
@ -774,6 +775,7 @@ FunctionLiteral* Parser::DoParseFunction(Isolate* isolate, ParseInfo* info,
Expect(Token::RPAREN);
} else {
// BindingIdentifier
ParameterParsingScope scope(impl(), &formals);
ParseFormalParameter(&formals);
DeclareFormalParameters(&formals);
}
@ -1131,10 +1133,8 @@ Statement* Parser::ParseExportDefault() {
default: {
int pos = position();
ExpressionClassifier classifier(this);
AcceptINScope scope(this, true);
Expression* value = ParseAssignmentExpression();
ValidateExpression();
SetFunctionName(value, ast_value_factory()->default_string());
const AstRawString* local_name =
@ -2319,6 +2319,17 @@ Statement* Parser::DesugarLexicalBindingsInForStatement(
return outer_block;
}
void ParserFormalParameters::ValidateDuplicate(Parser* parser) const {
if (has_duplicate()) {
parser->ReportMessageAt(duplicate_loc, MessageTemplate::kParamDupe);
}
}
void ParserFormalParameters::ValidateStrictMode(Parser* parser) const {
if (strict_error_loc.IsValid()) {
parser->ReportMessageAt(strict_error_loc, strict_error_message);
}
}
void Parser::AddArrowFunctionFormalParameters(
ParserFormalParameters* parameters, Expression* expr, int end_pos) {
// ArrowFunctionFormals ::
@ -2941,8 +2952,6 @@ void Parser::ParseFunction(
bool is_wrapped = function_type == FunctionLiteral::kWrapped;
ExpressionClassifier formals_classifier(this);
int expected_parameters_end_pos = parameters_end_pos_;
if (expected_parameters_end_pos != kNoSourcePosition) {
// This is the first function encountered in a CreateDynamicFunction eval.
@ -2971,6 +2980,8 @@ void Parser::ParseFunction(
} else {
// For a regular function, the function arguments are parsed from source.
DCHECK_NULL(arguments_for_wrapped_function);
DeclarationParsingScope formals_scope(
this, ExpressionScope::kParameterDeclaration);
ParseFormalParameterList(&formals);
if (expected_parameters_end_pos != kNoSourcePosition) {
// Check for '(' or ')' shenanigans in the parameter string for dynamic

24
src/parsing/parser.h
View File

@ -113,13 +113,23 @@ struct ParserFormalParameters : FormalParametersBase {
Parameter* const* next() const { return &next_parameter; }
};
Scanner::Location duplicate_location() const { return duplicate_loc; }
void set_strict_parameter_error(const Scanner::Location& loc,
MessageTemplate message) {
strict_error_loc = loc;
strict_error_message = message;
}
bool has_duplicate() const { return duplicate_loc.IsValid(); }
void ValidateDuplicate(Parser* parser) const;
void ValidateStrictMode(Parser* parser) const;
explicit ParserFormalParameters(DeclarationScope* scope)
: FormalParametersBase(scope) {}
base::ThreadedList<Parameter> params;
Scanner::Location duplicate_loc = Scanner::Location::invalid();
Scanner::Location strict_error_loc = Scanner::Location::invalid();
MessageTemplate strict_error_message = MessageTemplate::kNone;
};
template <>
@ -155,8 +165,6 @@ struct ParserTypes<Parser> {
typedef v8::internal::SourceRangeScope SourceRangeScope;
typedef ParserTarget Target;
typedef ParserTargetScope TargetScope;
static constexpr bool ExpressionClassifierReportErrors = true;
};
class V8_EXPORT_PRIVATE Parser : public NON_EXPORTED_BASE(ParserBase<Parser>) {
@ -192,8 +200,8 @@ class V8_EXPORT_PRIVATE Parser : public NON_EXPORTED_BASE(ParserBase<Parser>) {
private:
friend class ParserBase<Parser>;
friend class v8::internal::ExpressionClassifierErrorTracker<
ParserTypes<Parser>>;
friend struct ParserFormalParameters;
friend class i::ExpressionScope<ParserTypes<Parser>>;
friend bool v8::internal::parsing::ParseProgram(ParseInfo*, Isolate*);
friend bool v8::internal::parsing::ParseFunction(
ParseInfo*, Handle<SharedFunctionInfo> shared_info, Isolate*);
@ -915,7 +923,6 @@ class V8_EXPORT_PRIVATE Parser : public NON_EXPORTED_BASE(ParserBase<Parser>) {
}
V8_INLINE void DeclareFormalParameters(ParserFormalParameters* parameters) {
ValidateFormalParameterInitializer();
bool is_simple = parameters->is_simple;
DeclarationScope* scope = parameters->scope;
if (!is_simple) scope->SetHasNonSimpleParameters();
@ -955,11 +962,6 @@ class V8_EXPORT_PRIVATE Parser : public NON_EXPORTED_BASE(ParserBase<Parser>) {
void SetFunctionNameFromIdentifierRef(Expression* value,
Expression* identifier);
V8_INLINE ZoneList<typename ExpressionClassifier::Error>*
GetReportedErrorList() const {
return function_state_->GetReportedErrorList();
}
V8_INLINE void CountUsage(v8::Isolate::UseCounterFeature feature) {
++use_counts_[feature];
}

21
src/parsing/preparser.cc
View File

@ -96,6 +96,14 @@ PreParser::PreParseResult PreParser::PreParseProgram() {
return kPreParseSuccess;
}
void PreParserFormalParameters::ValidateDuplicate(PreParser* preparser) const {
if (has_duplicate_) preparser->ReportUnidentifiableError();
}
void PreParserFormalParameters::ValidateStrictMode(PreParser* preparser) const {
if (strict_parameter_error_) preparser->ReportUnidentifiableError();
}
PreParser::PreParseResult PreParser::PreParseFunction(
const AstRawString* function_name, FunctionKind kind,
FunctionLiteral::FunctionType function_type,
@ -131,12 +139,12 @@ PreParser::PreParseResult PreParser::PreParseFunction(
FunctionState function_state(&function_state_, &scope_, function_scope);
PreParserFormalParameters formals(function_scope);
std::unique_ptr<ExpressionClassifier> formals_classifier;
// Parse non-arrow function parameters. For arrow functions, the parameters
// have already been parsed.
if (!IsArrowFunction(kind)) {
formals_classifier.reset(new ExpressionClassifier(this));
DeclarationParsingScope formals_scope(
this, ExpressionScope::kParameterDeclaration);
// We return kPreParseSuccess in failure cases too - errors are retrieved
// separately by Parser::SkipLazyFunctionBody.
ParseFormalParameterList(&formals);
@ -291,13 +299,16 @@ PreParser::Expression PreParser::ParseFunctionLiteral(
}
FunctionState function_state(&function_state_, &scope_, function_scope);
ExpressionClassifier formals_classifier(this);
Expect(Token::LPAREN);
int start_position = position();
function_scope->set_start_position(start_position);
PreParserFormalParameters formals(function_scope);
ParseFormalParameterList(&formals);
{
DeclarationParsingScope formals_scope(
this, ExpressionScope::kParameterDeclaration);
ParseFormalParameterList(&formals);
}
Expect(Token::RPAREN);
int formals_end_position = scanner()->location().end_pos;
@ -406,7 +417,7 @@ bool PreParser::IdentifierEquals(const PreParserIdentifier& identifier,
PreParserExpression PreParser::ExpressionFromIdentifier(
const PreParserIdentifier& name, int start_position, InferName infer) {
VariableProxy* proxy = nullptr;
DCHECK_EQ(name.string_ == nullptr, has_error());
DCHECK_IMPLIES(name.string_ == nullptr, has_error());
if (name.string_ == nullptr) return PreParserExpression::Default();
proxy = scope()->NewUnresolved(factory()->ast_node_factory(), name.string_,
start_position, NORMAL_VARIABLE);

27
src/parsing/preparser.h
View File

@ -832,21 +832,28 @@ class PreParserFactory {
Zone* zone_;
};
class PreParser;
class PreParserFormalParameters : public FormalParametersBase {
public:
explicit PreParserFormalParameters(DeclarationScope* scope)
: FormalParametersBase(scope) {}
Scanner::Location duplicate_location() const { UNREACHABLE(); }
bool has_duplicate() const { return has_duplicate_; }
void set_has_duplicate() { has_duplicate_ = true; }
bool has_duplicate() { return has_duplicate_; }
void ValidateDuplicate(PreParser* preparser) const;
void set_strict_parameter_error(const Scanner::Location& loc,
MessageTemplate message) {
strict_parameter_error_ = loc.IsValid();
}
void ValidateStrictMode(PreParser* preparser) const;
private:
bool has_duplicate_ = false;
bool strict_parameter_error_ = false;
};
class PreParser;
class PreParserTarget {
public:
PreParserTarget(ParserBase<PreParser>* preparser,
@ -935,8 +942,6 @@ struct ParserTypes<PreParser> {
typedef PreParserSourceRangeScope SourceRangeScope;
typedef PreParserTarget Target;
typedef PreParserTargetScope TargetScope;
static constexpr bool ExpressionClassifierReportErrors = false;
};
@ -954,7 +959,6 @@ struct ParserTypes<PreParser> {
// it is used) are generally omitted.
class PreParser : public ParserBase<PreParser> {
friend class ParserBase<PreParser>;
friend class v8::internal::ExpressionClassifier<ParserTypes<PreParser>>;
public:
typedef PreParserIdentifier Identifier;
@ -1016,6 +1020,8 @@ class PreParser : public ParserBase<PreParser> {
}
private:
friend class i::ExpressionScope<ParserTypes<PreParser>>;
friend class PreParserFormalParameters;
// These types form an algebra over syntactic categories that is just
// rich enough to let us recognize and propagate the constructs that
// are either being counted in the preparser data, or is important
@ -1677,14 +1683,12 @@ class PreParser : public ParserBase<PreParser> {
V8_INLINE void DeclareFormalParameters(
const PreParserFormalParameters* parameters) {
ValidateFormalParameterInitializer();
if (!parameters->is_simple) parameters->scope->SetHasNonSimpleParameters();
}
V8_INLINE void DeclareArrowFunctionFormalParameters(
PreParserFormalParameters* parameters, const PreParserExpression& params,
const Scanner::Location& params_loc) {
ValidateFormalParameterInitializer();
if (params.variables_ != nullptr) {
Scope* scope = parameters->scope;
for (auto variable : *params.variables_) {
@ -1709,11 +1713,6 @@ class PreParser : public ParserBase<PreParser> {
const PreParserExpression& value, const PreParserExpression& identifier) {
}
V8_INLINE ZoneList<typename ExpressionClassifier::Error>*
GetReportedErrorList() const {
return function_state_->GetReportedErrorList();
}
V8_INLINE void CountUsage(v8::Isolate::UseCounterFeature feature) {
if (use_counts_ != nullptr) ++use_counts_[feature];
}

5
test/message/regress/fail/regress-903874.out
View File

@ -1,5 +0,0 @@
*%(basename)s:6: RangeError: Maximum call stack size exceeded
eval(code);
^
RangeError: Maximum call stack size exceeded
at *%(basename)s:6:6

5
test/mjsunit/es8/async-arrow-default-function-await.js Normal file
View File

@ -0,0 +1,5 @@
// Copyright 2018 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
async(o = (function(await) {})) => 0

2
test/message/regress/fail/regress-903874.js → test/mjsunit/regress/regress-903874.js
View File

@ -3,4 +3,4 @@
// found in the LICENSE file.
var code = "function f(" + ("{o(".repeat(10000));
eval(code);
assertThrows(code, SyntaxError);