lrn@chromium.org
2d4aa6295b
Extract scanner base/JS/JSON and move base and JS to scanner-base.
...
Remove templates from prescanner.
Review URL: http://codereview.chromium.org/5136002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5854 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-11-18 13:07:02 +00:00
lrn@chromium.org
9902e43293
Add separate scanner only intended for preparsing.
...
Review URL: http://codereview.chromium.org/5063003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5837 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-11-17 12:00:22 +00:00
lrn@chromium.org
47c1870996
Move static scanner fields to scanner-base.h
...
Review URL: http://codereview.chromium.org/5026005
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5828 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-11-16 08:01:45 +00:00
lrn@chromium.org
f386f97476
Move part of scanner.* into scanner-base.* for reuse in preparser scanner.
...
Make checks.h not depend on flags.h or global.h (or anything else except
include/v8stdint.h). Only checks.cc has the dependencies (so another
implementation of checks.cc can be provided by the preparser).
Now files depending on checks.h (using ASSERT macros) can include it
directly without depending on all of v8.
Review URL: http://codereview.chromium.org/4576001
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5775 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-11-05 13:33:40 +00:00
lrn@chromium.org
a98baf9666
Remove unused variable from Scanner class.
...
BUG=None
TEST=compiles
Patch from Thiago Farina <tfarina@chromium.org>
Review URL: http://codereview.chromium.org/3413015
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5490 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-09-20 08:43:35 +00:00
erik.corry@gmail.com
d59bb82f1f
Add and use AsciiAlphaToLower. Move RemoveLast test. Add Clear test. This is a commit of http://codereview.chromium.org/3307003/show http://codereview.chromium.org/3312007/show and http://codereview.chromium.org/3341002/show for tfarina
...
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5409 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-09-03 12:59:52 +00:00
lrn@chromium.org
5628d3c482
Fix parsing of /**/--> on first line of input.
...
BUG=53548
TEST=
Review URL: http://codereview.chromium.org/3330001
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5400 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-09-02 07:36:30 +00:00
lrn@chromium.org
23d0abb040
Ensure that scanner state is correctly reset when an error is encountered.
...
Add a scope object to ensure that leaving a literal scanning prematurely
will clean up after itself.
Also reset the literal buffer if a scanner is reinitialized with a new
source code.
Review URL: http://codereview.chromium.org/3137037
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5327 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-08-24 12:29:50 +00:00
lrn@chromium.org
b1e27e2cd5
Created collector class and used it to collect identifiers during scanning.
...
The collector class automatically expands to hold the values added to it,
like a List, but doesn't ensure that the backing store is contiguous, which
allows it to avoid copying back and forth as the buffer grows.
This is in preparation for identifyng identical symbols during preparsing.
Review URL: http://codereview.chromium.org/3181036
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5325 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-08-24 10:53:44 +00:00
ager@chromium.org
ad5e73fb85
Fix preparsing from a source string that is not external.
...
This fixes issue 775.
Review URL: http://codereview.chromium.org/2959007
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@5050 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-07-13 10:29:31 +00:00
lrn@chromium.org
05fa888927
Made scanner follow coding style.
...
Review URL: http://codereview.chromium.org/2832018
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@4916 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-06-22 12:31:24 +00:00
sgjesse@chromium.org
fca7b2cefa
Refactor the scanner interface
...
No need to create sub strings for lazy compiles. The scanner will start from the start position provided.
Moved the creating of character streams into the scanner where possible. This uses a input buffer in the scanner class instead of a stack allocated one.
Added a UTF16 buffer for reading external ascii strings (by templating the external two byte string one) as all the source for the builtins are exposed as external ascii strings.
Review URL: http://codereview.chromium.org/661367
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@4007 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-03-03 13:16:10 +00:00
lrn@chromium.org
f03e42b133
Added validating JSON parser mode to parser.
...
Review URL: http://codereview.chromium.org/549207
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3752 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2010-02-01 10:31:55 +00:00
lrn@chromium.org
c1efaa01fe
Reapply 3246 and 3247 after fixing valgrind warning.
...
Review URL: http://codereview.chromium.org/386001
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3257 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-11-10 10:23:23 +00:00
erik.corry@gmail.com
95d550578c
Revert 3245 and 3246 because they cause valgrind failures.
...
TBR=lrn
Review URL: http://codereview.chromium.org/372059
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3254 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-11-09 17:05:51 +00:00
lrn@chromium.org
8ad022d544
Fix lint issue.
...
Review URL: http://codereview.chromium.org/379005
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3247 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-11-09 12:18:25 +00:00
lrn@chromium.org
cf0d74554c
Remove unnecessary buffer doubling and content copying.
...
Review URL: http://codereview.chromium.org/377006
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3246 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-11-09 11:52:18 +00:00
lrn@chromium.org
e8db709be1
Removed (now) unused Token-lookup code.
...
Review URL: http://codereview.chromium.org/361026
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3223 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-11-05 10:24:20 +00:00
lrn@chromium.org
273e860cc1
Changed keyword token recognition to be done inline in the identifier scanner.
...
Review URL: http://codereview.chromium.org/360048
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@3221 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-11-05 10:11:38 +00:00
sgjesse@chromium.org
77204cb36f
Fix an ASSERT in the scanner.
...
The assert when performing a push back on a two byte string was wrong.
Added a small regression test.
Review URL: http://codereview.chromium.org/173116
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@2735 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-08-21 06:30:59 +00:00
feng@chromium.org
1c70072f7f
Streamline the scanner for external two byte string input.
...
Review URL: http://codereview.chromium.org/165403
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@2703 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-08-18 07:14:02 +00:00
mikhail.naganov@gmail.com
30a0a7de43
Split nested namespaces declaration in two lines in accordance with C++ Style Guide.
...
This issue was raised by Brett Wilson while reviewing my changelist for readability. Craig Silverstein (one of C++ SG maintainers) confirmed that we should declare one namespace per line. Our way of namespaces closing seems not violating style guides (there is no clear agreement on it), so I left it intact.
Review URL: http://codereview.chromium.org/115756
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@2038 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-05-25 10:05:56 +00:00
kasperl@chromium.org
43cddf412d
Optimize the lexical scanner by selective inlining, and
...
by dealing with whitespace as part of the token scanning
instead of as a separate step before it.
Review URL: http://codereview.chromium.org/113336
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@1934 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-05-13 13:40:02 +00:00
lrn@chromium.org
396fa22799
Regular Expression literal flags may contain unicode escapes. If these escape any of the
...
valid flags, "gim", they should work.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@1253 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-02-12 09:09:28 +00:00
deanm@chromium.org
b7c1200462
Fix a bunch of spelling mistakes :\
...
Review URL: http://codereview.chromium.org/18094
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@1088 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2009-01-15 19:08:34 +00:00
olehougaard
ed7d9e9c5b
Reverting the BOM changes due to security concerns.
...
Review URL: http://codereview.chromium.org/14890
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@1011 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2008-12-19 13:28:56 +00:00
olehougaard
2c3b392c8c
Handling byte-order marks as specified in Ecmascript-262 and in compliance with Safari.
...
Review URL: http://codereview.chromium.org/15075
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@1006 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2008-12-19 12:06:11 +00:00
kasperl@chromium.org
d24556d1c5
Fix Chromium issue 176 by treating byte-order marks as whitespace.
...
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@765 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2008-11-17 06:56:28 +00:00
christian.plesner.hansen@gmail.com
9bed566bdb
Changed copyright header from google inc. to v8 project authors.
...
Added presubmit step to check copyright.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@242 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2008-09-09 20:08:45 +00:00
christian.plesner.hansen
43d26ecc35
Initial export.
...
git-svn-id: http://v8.googlecode.com/svn/trunk@2 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2008-07-03 15:10:15 +00:00