AuroraMiddleware/v8
1
0
Fork 0
Code Issues 2 Pull Requests Releases Wiki Activity
b7a7e2fb89
v8/test/unittests/strings/char-predicates-unittest.cc

191 lines
6.7 KiB
C++
Raw Normal View History

Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Copyright 2014 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
Move string-related files to src/strings Bug: v8:9247 Change-Id: I9bcf2694b449f79cdbe03f5fde59cb21b8cad418 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1619758 Commit-Queue: Yang Guo <yangguo@chromium.org> Reviewed-by: Michael Lippautz <mlippautz@chromium.org> Reviewed-by: Andreas Haas <ahaas@chromium.org> Reviewed-by: Georg Neis <neis@chromium.org> Cr-Commit-Position: refs/heads/master@{#61676}
2019-05-21 06:38:38 +00:00
#include "src/strings/char-predicates.h"
#include "src/strings/char-predicates-inl.h"
#include "src/strings/unicode.h"
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
#include "testing/gtest/include/gtest/gtest.h"
namespace v8 {
namespace internal {
Conform to the unittest naming convention. R=bmeurer@chromium.org Review URL: https://codereview.chromium.org/644973002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24513 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 08:06:21 +00:00
TEST(CharPredicatesTest, WhiteSpace) {
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsWhiteSpace(0x0009));
EXPECT_TRUE(IsWhiteSpace(0x000B));
EXPECT_TRUE(IsWhiteSpace(0x000C));
EXPECT_TRUE(IsWhiteSpace(' '));
EXPECT_TRUE(IsWhiteSpace(0x00A0));
EXPECT_TRUE(IsWhiteSpace(0x1680));
EXPECT_TRUE(IsWhiteSpace(0x2000));
EXPECT_TRUE(IsWhiteSpace(0x2007));
EXPECT_TRUE(IsWhiteSpace(0x202F));
EXPECT_TRUE(IsWhiteSpace(0x205F));
EXPECT_TRUE(IsWhiteSpace(0x3000));
EXPECT_TRUE(IsWhiteSpace(0xFEFF));
EXPECT_FALSE(IsWhiteSpace(0x180E));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
}
Conform to the unittest naming convention. R=bmeurer@chromium.org Review URL: https://codereview.chromium.org/644973002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24513 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 08:06:21 +00:00
TEST(CharPredicatesTest, WhiteSpaceOrLineTerminator) {
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x0009));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x000B));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x000C));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(' '));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x00A0));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x1680));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x2000));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x2007));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x202F));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x205F));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0xFEFF));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Line terminators
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x000A));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x000D));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x2028));
EXPECT_TRUE(IsWhiteSpaceOrLineTerminator(0x2029));
EXPECT_FALSE(IsWhiteSpaceOrLineTerminator(0x180E));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
}
Conform to the unittest naming convention. R=bmeurer@chromium.org Review URL: https://codereview.chromium.org/644973002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24513 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 08:06:21 +00:00
TEST(CharPredicatesTest, IdentifierStart) {
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierStart('$'));
EXPECT_TRUE(IsIdentifierStart('_'));
EXPECT_TRUE(IsIdentifierStart('\\'));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// http://www.unicode.org/reports/tr31/
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// curl http://www.unicode.org/Public/UCD/latest/ucd/PropList.txt |
// grep 'Other_ID_Start'
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Other_ID_Start
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierStart(0x1885));
EXPECT_TRUE(IsIdentifierStart(0x1886));
EXPECT_TRUE(IsIdentifierStart(0x2118));
EXPECT_TRUE(IsIdentifierStart(0x212E));
EXPECT_TRUE(IsIdentifierStart(0x309B));
EXPECT_TRUE(IsIdentifierStart(0x309C));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Issue 2892:
// \u2E2F has the Pattern_Syntax property, excluding it from ID_Start.
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_FALSE(IsIdentifierStart(0x2E2F));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
#ifdef V8_INTL_SUPPORT
// New in Unicode 8.0 (6,847 code points)
// [:ID_Start:] & [[:Age=8.0:] - [:Age=7.0:]]
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierStart(0x08B3));
EXPECT_TRUE(IsIdentifierStart(0x0AF9));
EXPECT_TRUE(IsIdentifierStart(0x13F8));
EXPECT_TRUE(IsIdentifierStart(0x9FCD));
EXPECT_TRUE(IsIdentifierStart(0xAB60));
EXPECT_TRUE(IsIdentifierStart(0x10CC0));
EXPECT_TRUE(IsIdentifierStart(0x108E0));
EXPECT_TRUE(IsIdentifierStart(0x2B820));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// New in Unicode 9.0 (7,177 code points)
// [:ID_Start:] & [[:Age=9.0:] - [:Age=8.0:]]
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierStart(0x1C80));
EXPECT_TRUE(IsIdentifierStart(0x104DB));
EXPECT_TRUE(IsIdentifierStart(0x1E922));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
#endif
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
}
Conform to the unittest naming convention. R=bmeurer@chromium.org Review URL: https://codereview.chromium.org/644973002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24513 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 08:06:21 +00:00
TEST(CharPredicatesTest, IdentifierPart) {
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart('$'));
EXPECT_TRUE(IsIdentifierPart('_'));
EXPECT_TRUE(IsIdentifierPart('\\'));
EXPECT_TRUE(IsIdentifierPart(0x200C));
EXPECT_TRUE(IsIdentifierPart(0x200D));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
#ifdef V8_INTL_SUPPORT
// New in Unicode 8.0 (6,847 code points)
// [:ID_Start:] & [[:Age=8.0:] - [:Age=7.0:]]
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart(0x08B3));
EXPECT_TRUE(IsIdentifierPart(0x0AF9));
EXPECT_TRUE(IsIdentifierPart(0x13F8));
EXPECT_TRUE(IsIdentifierPart(0x9FCD));
EXPECT_TRUE(IsIdentifierPart(0xAB60));
EXPECT_TRUE(IsIdentifierPart(0x10CC0));
EXPECT_TRUE(IsIdentifierPart(0x108E0));
EXPECT_TRUE(IsIdentifierPart(0x2B820));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// [[:ID_Continue:]-[:ID_Start:]] & [[:Age=8.0:]-[:Age=7.0:]]
// 162 code points
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart(0x08E3));
EXPECT_TRUE(IsIdentifierPart(0xA69E));
EXPECT_TRUE(IsIdentifierPart(0x11730));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// New in Unicode 9.0 (7,177 code points)
// [:ID_Start:] & [[:Age=9.0:] - [:Age=8.0:]]
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart(0x1C80));
EXPECT_TRUE(IsIdentifierPart(0x104DB));
EXPECT_TRUE(IsIdentifierPart(0x1E922));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// [[:ID_Continue:]-[:ID_Start:]] & [[:Age=9.0:]-[:Age=8.0:]]
// 162 code points
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart(0x08D4));
EXPECT_TRUE(IsIdentifierPart(0x1DFB));
EXPECT_TRUE(IsIdentifierPart(0xA8C5));
EXPECT_TRUE(IsIdentifierPart(0x11450));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
#endif
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// http://www.unicode.org/reports/tr31/
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// curl http://www.unicode.org/Public/UCD/latest/ucd/PropList.txt |
// grep 'Other_ID_(Continue|Start)'
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Other_ID_Start
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart(0x1885));
EXPECT_TRUE(IsIdentifierPart(0x1886));
EXPECT_TRUE(IsIdentifierPart(0x2118));
EXPECT_TRUE(IsIdentifierPart(0x212E));
EXPECT_TRUE(IsIdentifierPart(0x309B));
EXPECT_TRUE(IsIdentifierPart(0x309C));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Other_ID_Continue
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierPart(0x00B7));
EXPECT_TRUE(IsIdentifierPart(0x0387));
EXPECT_TRUE(IsIdentifierPart(0x1369));
EXPECT_TRUE(IsIdentifierPart(0x1370));
EXPECT_TRUE(IsIdentifierPart(0x1371));
EXPECT_TRUE(IsIdentifierPart(0x19DA));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
// Issue 2892:
// \u2E2F has the Pattern_Syntax property, excluding it from ID_Start.
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_FALSE(IsIdentifierPart(0x2E2F));
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
}
[intl] Reorganize code - Split out code for Intl objects into src/objects/ - Rename i18n to intl (except for the name of the build flag) - Use build system more broadly to turn on/off Intl code - Delete a little bit of dead code Bug: v8:5751 Change-Id: I41bf2825a5cb0df20824922b17c24cae637984da Reviewed-on: https://chromium-review.googlesource.com/481284 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Adam Klein <adamk@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Reviewed-by: Marja Hölttä <marja@chromium.org> Commit-Queue: Daniel Ehrenberg <littledan@chromium.org> Cr-Commit-Position: refs/heads/master@{#44801}
2017-04-21 08:35:12 +00:00
#ifdef V8_INTL_SUPPORT
Conform to the unittest naming convention. R=bmeurer@chromium.org Review URL: https://codereview.chromium.org/644973002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24513 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 08:06:21 +00:00
TEST(CharPredicatesTest, SupplementaryPlaneIdentifiers) {
Allow identifier code points from supplementary multilingual planes. ES5.1 section 6 ("Source Text"): "Throughout the rest of this document, the phrase “code unit” and the word “character” will be used to refer to a 16-bit unsigned value used to represent a single 16-bit unit of text." This changed in ES6 draft section 10.1 ("Source Text"): "The ECMAScript code is expressed using Unicode, version 5.1 or later. ECMAScript source text is a sequence of code points. All Unicode code point values from U+0000 to U+10FFFF, including surrogate code points, may occur in source text where permitted by the ECMAScript grammars." This patch is to reflect this spec change. BUG=v8:3617 LOG=Y R=jochen@chromium.org Review URL: https://codereview.chromium.org/640193002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24510 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:13:46 +00:00
// Both ID_Start and ID_Continue.
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierStart(0x10403)); // Category Lu
EXPECT_TRUE(IsIdentifierPart(0x10403));
EXPECT_TRUE(IsIdentifierStart(0x1043C)); // Category Ll
EXPECT_TRUE(IsIdentifierPart(0x1043C));
EXPECT_TRUE(IsIdentifierStart(0x16F9C)); // Category Lm
EXPECT_TRUE(IsIdentifierPart(0x16F9C));
EXPECT_TRUE(IsIdentifierStart(0x10048)); // Category Lo
EXPECT_TRUE(IsIdentifierPart(0x10048));
EXPECT_TRUE(IsIdentifierStart(0x1014D)); // Category Nl
EXPECT_TRUE(IsIdentifierPart(0x1014D));
Allow identifier code points from supplementary multilingual planes. ES5.1 section 6 ("Source Text"): "Throughout the rest of this document, the phrase “code unit” and the word “character” will be used to refer to a 16-bit unsigned value used to represent a single 16-bit unit of text." This changed in ES6 draft section 10.1 ("Source Text"): "The ECMAScript code is expressed using Unicode, version 5.1 or later. ECMAScript source text is a sequence of code points. All Unicode code point values from U+0000 to U+10FFFF, including surrogate code points, may occur in source text where permitted by the ECMAScript grammars." This patch is to reflect this spec change. BUG=v8:3617 LOG=Y R=jochen@chromium.org Review URL: https://codereview.chromium.org/640193002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24510 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:13:46 +00:00
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
// New in Unicode 8.0
// [ [:ID_Start=Yes:] & [:Age=8.0:]] - [:Age=7.0:]
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_TRUE(IsIdentifierStart(0x108E0));
EXPECT_TRUE(IsIdentifierStart(0x10C80));
Use ICU for ID_START, ID_CONTINUE and WhiteSpace check Use ICU to check ID_Start, ID_Continue and WhiteSpace even for BMP when V8_INTL_SUPPORT is on (which is default). Change LineTerminator::Is() to check 4 code points from ES#sec-line-terminators instead of using tables and Lookup function. Remove Lowercase::Is(). It's not used anywhere. Update webkit/{ToNumber,parseFloat}.js to have the correct expectation for U+180E and the corresponding expected files. This is a follow-up to an earlier change ( https://codereview.chromium.org/2720953003 ). CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_win_dbg,v8_mac_dbg;master.tryserver.chromium.android:android_arm64_dbg_recipe CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng BUG=v8:5370,v8:5155 TEST=unittests --gtest_filter=CharP* TEST=webkit: ToNumber, parseFloat TEST=test262: built-ins/Number/S9.3*, built-ins/parse{Int,Float}/S15* TEST=test262: language/white-space/mong* TEST=test262: built-ins/String/prototype/trim/u180e TEST=mjsunit: whitespaces Review-Url: https://codereview.chromium.org/2331303002 Cr-Commit-Position: refs/heads/master@{#45957}
2017-06-14 20:32:49 +00:00
Allow identifier code points from supplementary multilingual planes. ES5.1 section 6 ("Source Text"): "Throughout the rest of this document, the phrase “code unit” and the word “character” will be used to refer to a 16-bit unsigned value used to represent a single 16-bit unit of text." This changed in ES6 draft section 10.1 ("Source Text"): "The ECMAScript code is expressed using Unicode, version 5.1 or later. ECMAScript source text is a sequence of code points. All Unicode code point values from U+0000 to U+10FFFF, including surrogate code points, may occur in source text where permitted by the ECMAScript grammars." This patch is to reflect this spec change. BUG=v8:3617 LOG=Y R=jochen@chromium.org Review URL: https://codereview.chromium.org/640193002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24510 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:13:46 +00:00
// Only ID_Continue.
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_FALSE(IsIdentifierStart(0x101FD)); // Category Mn
EXPECT_TRUE(IsIdentifierPart(0x101FD));
EXPECT_FALSE(IsIdentifierStart(0x11002)); // Category Mc
EXPECT_TRUE(IsIdentifierPart(0x11002));
EXPECT_FALSE(IsIdentifierStart(0x104A9)); // Category Nd
EXPECT_TRUE(IsIdentifierPart(0x104A9));
Allow identifier code points from supplementary multilingual planes. ES5.1 section 6 ("Source Text"): "Throughout the rest of this document, the phrase “code unit” and the word “character” will be used to refer to a 16-bit unsigned value used to represent a single 16-bit unit of text." This changed in ES6 draft section 10.1 ("Source Text"): "The ECMAScript code is expressed using Unicode, version 5.1 or later. ECMAScript source text is a sequence of code points. All Unicode code point values from U+0000 to U+10FFFF, including surrogate code points, may occur in source text where permitted by the ECMAScript grammars." This patch is to reflect this spec change. BUG=v8:3617 LOG=Y R=jochen@chromium.org Review URL: https://codereview.chromium.org/640193002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24510 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:13:46 +00:00
// Neither.
[cleanup] Make unicode predicate cache tables static Moves the unicode predicate cache tables out of the unicode cache, and turns them into generic predicates in char-predicates.h which use static constexpr tables. This drops the per-isolate cost of unicode caches, and removes the need for accessing the unicode cache from most files. It does remove the mutability of the cache, which means that there may be regressions when parsing non-ASCII identifiers. Most likely the benefits to ASCII identifiers/keywords will outweigh any non-ASCII costs. Change-Id: I9a7a8b7c9b22d3e9ede824ab4e27f133ce20a399 Reviewed-on: https://chromium-review.googlesource.com/c/1335564 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Toon Verwaest <verwaest@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Leszek Swirski <leszeks@chromium.org> Cr-Commit-Position: refs/heads/master@{#57506}
2018-11-14 14:14:58 +00:00
EXPECT_FALSE(IsIdentifierStart(0x10111)); // Category No
EXPECT_FALSE(IsIdentifierPart(0x10111));
EXPECT_FALSE(IsIdentifierStart(0x1F4A9)); // Category So
EXPECT_FALSE(IsIdentifierPart(0x1F4A9));
Allow identifier code points from supplementary multilingual planes. ES5.1 section 6 ("Source Text"): "Throughout the rest of this document, the phrase “code unit” and the word “character” will be used to refer to a 16-bit unsigned value used to represent a single 16-bit unit of text." This changed in ES6 draft section 10.1 ("Source Text"): "The ECMAScript code is expressed using Unicode, version 5.1 or later. ECMAScript source text is a sequence of code points. All Unicode code point values from U+0000 to U+10FFFF, including surrogate code points, may occur in source text where permitted by the ECMAScript grammars." This patch is to reflect this spec change. BUG=v8:3617 LOG=Y R=jochen@chromium.org Review URL: https://codereview.chromium.org/640193002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24510 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:13:46 +00:00
}
[intl] Reorganize code - Split out code for Intl objects into src/objects/ - Rename i18n to intl (except for the name of the build flag) - Use build system more broadly to turn on/off Intl code - Delete a little bit of dead code Bug: v8:5751 Change-Id: I41bf2825a5cb0df20824922b17c24cae637984da Reviewed-on: https://chromium-review.googlesource.com/481284 Reviewed-by: Yang Guo <yangguo@chromium.org> Reviewed-by: Adam Klein <adamk@chromium.org> Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Reviewed-by: Marja Hölttä <marja@chromium.org> Commit-Queue: Daniel Ehrenberg <littledan@chromium.org> Cr-Commit-Position: refs/heads/master@{#44801}
2017-04-21 08:35:12 +00:00
#endif // V8_INTL_SUPPORT
Allow identifier code points from supplementary multilingual planes. ES5.1 section 6 ("Source Text"): "Throughout the rest of this document, the phrase “code unit” and the word “character” will be used to refer to a 16-bit unsigned value used to represent a single 16-bit unit of text." This changed in ES6 draft section 10.1 ("Source Text"): "The ECMAScript code is expressed using Unicode, version 5.1 or later. ECMAScript source text is a sequence of code points. All Unicode code point values from U+0000 to U+10FFFF, including surrogate code points, may occur in source text where permitted by the ECMAScript grammars." This patch is to reflect this spec change. BUG=v8:3617 LOG=Y R=jochen@chromium.org Review URL: https://codereview.chromium.org/640193002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24510 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-10 07:13:46 +00:00
Update unicode to 7.0.0. And do not use code points with PATTERN_* property for identifier start. Maintain that \u180E is a white space character. BUG=v8:2892 LOG=Y R=dpino@igalia.com, mathias@qiwi.be Review URL: https://codereview.chromium.org/638643002 git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24473 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-08 14:55:03 +00:00
} // namespace internal
} // namespace v8
Reference in New Issue Copy Permalink