2002-10-22 00:09:32 +00:00
|
|
|
|
|
|
|
|
#*****************************************************************************
|
|
|
|
|
#
|
|
|
|
|
# Copyright (C) 2002, International Business Machines Corporation and others.
|
|
|
|
|
# All Rights Reserved.
|
|
|
|
|
#
|
|
|
|
|
#*****************************************************************************
|
|
|
|
|
#
|
|
|
|
|
# file: regexcst.txt
|
|
|
|
|
# ICU Regular Expression Parser State Table
|
|
|
|
|
#
|
|
|
|
|
# This state table is used when reading and parsing a regular expression pattern
|
|
|
|
|
# The pattern parser uses a state machine; the data in this file define the
|
|
|
|
|
# state transitions that occur for each input character.
|
|
|
|
|
#
|
|
|
|
|
# *** This file defines the regex pattern grammar. This is it.
|
|
|
|
|
# *** The determination of what is accepted is here.
|
|
|
|
|
#
|
|
|
|
|
# This file is processed by a perl script "regexcst.pl" to produce initialized C arrays
|
|
|
|
|
# that are then built with the rule parser.
|
|
|
|
|
#
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# Here is the syntax of the state definitions in this file:
|
|
|
|
|
#
|
|
|
|
|
#
|
|
|
|
|
#StateName:
|
|
|
|
|
# input-char n next-state ^push-state action
|
|
|
|
|
# input-char n next-state ^push-state action
|
|
|
|
|
# | | | | |
|
|
|
|
|
# | | | | |--- action to be performed by state machine
|
|
|
|
|
# | | | | See function RBBIRuleScanner::doParseActions()
|
|
|
|
|
# | | | |
|
|
|
|
|
# | | | |--- Push this named state onto the state stack.
|
|
|
|
|
# | | | Later, when next state is specified as "pop",
|
|
|
|
|
# | | | the pushed state will become the current state.
|
|
|
|
|
# | | |
|
|
|
|
|
# | | |--- Transition to this state if the current input character matches the input
|
|
|
|
|
# | | character or char class in the left hand column. "pop" causes the next
|
|
|
|
|
# | | state to be popped from the state stack.
|
|
|
|
|
# | |
|
|
|
|
|
# | |--- When making the state transition specified on this line, advance to the next
|
|
|
|
|
# | character from the input only if 'n' appears here.
|
|
|
|
|
# |
|
|
|
|
|
# |--- Character or named character classes to test for. If the current character being scanned
|
|
|
|
|
# matches, peform the actions and go to the state specified on this line.
|
|
|
|
|
# The input character is tested sequentally, in the order written. The characters and
|
|
|
|
|
# character classes tested for do not need to be mutually exclusive. The first match wins.
|
|
|
|
|
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# start state, scan position is at the beginning of the pattern.
|
|
|
|
|
#
|
|
|
|
|
start:
|
2002-11-11 18:49:49 +00:00
|
|
|
default term doPatStart
|
2002-10-22 00:09:32 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
2002-11-11 18:49:49 +00:00
|
|
|
# term. At a position where we can accept the start most items in a pattern.
|
2002-10-22 00:09:32 +00:00
|
|
|
#
|
|
|
|
|
term:
|
2003-01-25 18:57:42 +00:00
|
|
|
quoted n expr-quant doLiteralChar
|
|
|
|
|
rule_char n expr-quant doLiteralChar
|
2002-11-11 18:49:49 +00:00
|
|
|
'[' n expr-quant doScanUnicodeSet
|
|
|
|
|
'(' n open-paren
|
2002-10-22 00:09:32 +00:00
|
|
|
'.' n expr-quant doDotAny
|
2002-11-06 02:35:20 +00:00
|
|
|
'^' n term doCaret
|
|
|
|
|
'$' n term doDollar
|
2002-10-24 22:16:07 +00:00
|
|
|
'\' n backslash
|
2003-01-25 18:57:42 +00:00
|
|
|
'|' n term doOrOperator
|
|
|
|
|
')' n pop doCloseParen
|
2003-01-16 01:12:04 +00:00
|
|
|
eof term doPatFinish
|
2002-10-22 00:09:32 +00:00
|
|
|
default errorDeath doRuleError
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# expr-quant We've just finished scanning a term, now look for the optional
|
|
|
|
|
# trailing quantifier - *, +, ?, *?, etc.
|
|
|
|
|
#
|
|
|
|
|
expr-quant:
|
|
|
|
|
'*' n quant-star
|
|
|
|
|
'+' n quant-plus
|
2002-11-07 02:34:46 +00:00
|
|
|
'?' n quant-opt
|
2003-01-16 01:12:04 +00:00
|
|
|
'{' n interval-open doIntervalInit
|
2003-01-25 18:57:42 +00:00
|
|
|
'(' n open-paren-quant
|
2002-10-22 00:09:32 +00:00
|
|
|
default expr-cont
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# expr-cont Expression, continuation. At a point where additional terms are
|
2002-10-24 22:16:07 +00:00
|
|
|
# allowed, but not required. No Quantifiers
|
2002-10-22 00:09:32 +00:00
|
|
|
#
|
|
|
|
|
expr-cont:
|
|
|
|
|
'|' n term doOrOperator
|
|
|
|
|
')' n pop doCloseParen
|
2002-10-24 22:16:07 +00:00
|
|
|
default term
|
2002-10-22 00:09:32 +00:00
|
|
|
|
|
|
|
|
|
2003-01-25 18:57:42 +00:00
|
|
|
#
|
|
|
|
|
# open-paren-quant Special case handling for comments appearing before a quantifier,
|
|
|
|
|
# e.g. x(?#comment )*
|
|
|
|
|
# Open parens from expr-quant come here; anything but a (?# comment
|
|
|
|
|
# branches into the normal parenthesis sequence as quickly as possible.
|
|
|
|
|
#
|
|
|
|
|
open-paren-quant:
|
|
|
|
|
'?' n open-paren-quant2
|
|
|
|
|
default open-paren
|
|
|
|
|
|
|
|
|
|
open-paren-quant2:
|
|
|
|
|
'#' n paren-comment ^expr-quant
|
|
|
|
|
default open-paren-extended
|
|
|
|
|
|
|
|
|
|
|
2002-10-22 00:09:32 +00:00
|
|
|
#
|
|
|
|
|
# open-paren We've got an open paren. We need to scan further to
|
|
|
|
|
# determine what kind of quantifier it is - plain (, (?:, (?>, or whatever.
|
|
|
|
|
#
|
|
|
|
|
open-paren:
|
|
|
|
|
'?' n open-paren-extended
|
|
|
|
|
default term ^expr-quant doOpenCaptureParen
|
|
|
|
|
|
|
|
|
|
open-paren-extended:
|
|
|
|
|
':' n term ^expr-quant doOpenNonCaptureParen # (?:
|
|
|
|
|
'>' n term ^expr-quant doOpenAtomicParen # (?>
|
|
|
|
|
'=' n term ^expr-cont doOpenLookAhead # (?=
|
|
|
|
|
'!' n term ^expr-cont doOpenLookAheadNeg # (?!
|
|
|
|
|
'<' n open-paren-lookbehind
|
2003-01-25 18:57:42 +00:00
|
|
|
'#' n paren-comment ^term
|
2002-11-07 02:34:46 +00:00
|
|
|
'i' n paren-flag doMatchMode
|
|
|
|
|
'x' n paren-flag doMatchMode
|
|
|
|
|
's' n paren-flag doMatchMode
|
|
|
|
|
'm' n paren-flag doMatchMode
|
|
|
|
|
'-' n paren-flag doMatchMode
|
2003-01-24 02:05:03 +00:00
|
|
|
'(' n errorDeath doConditionalExpr
|
|
|
|
|
'{' n errorDeath doPerlInline
|
2002-10-22 00:09:32 +00:00
|
|
|
default errorDeath doBadOpenParenType
|
|
|
|
|
|
|
|
|
|
open-paren-lookbehind:
|
|
|
|
|
'=' n term ^expr-cont doOpenLookBehind # (?<=
|
|
|
|
|
'!' n term ^expr-cont doOpenLookBehindNeg # (?<!
|
|
|
|
|
default errorDeath doBadOpenParenType
|
|
|
|
|
|
|
|
|
|
|
2002-11-06 02:35:20 +00:00
|
|
|
#
|
|
|
|
|
# paren-comment We've got a (?# ... ) style comment. Eat pattern text till we get to the ')'
|
|
|
|
|
# TODO: should parens nest here? Check what perl does.
|
|
|
|
|
#
|
|
|
|
|
paren-comment:
|
2003-01-25 18:57:42 +00:00
|
|
|
')' n pop
|
|
|
|
|
eof errorDeath doMismatchedParenErr
|
2002-11-06 02:35:20 +00:00
|
|
|
default n paren-comment
|
2002-11-07 02:34:46 +00:00
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# paren-flag Scanned a (?ismx-ismx flag setting thing
|
|
|
|
|
# TODO: this is not fully implemented yet.
|
|
|
|
|
paren-flag:
|
|
|
|
|
'i' n paren-flag doMatchMode
|
|
|
|
|
's' n paren-flag doMatchMode
|
|
|
|
|
'm' n paren-flag doMatchMode
|
|
|
|
|
'x' n paren-flag doMatchMode
|
|
|
|
|
'-' n paren-flag doMatchMode
|
|
|
|
|
')' n term
|
|
|
|
|
':' n term ^expr-quant doOpenNonCaptureParen
|
|
|
|
|
default errorDeath
|
2002-11-06 02:35:20 +00:00
|
|
|
|
|
|
|
|
|
2002-10-22 00:09:32 +00:00
|
|
|
#
|
|
|
|
|
# quant-star Scanning a '*' quantifier. Need to look ahead to decide
|
|
|
|
|
# between plain '*', '*?', '*+'
|
|
|
|
|
#
|
|
|
|
|
quant-star:
|
|
|
|
|
'?' n expr-cont doNGStar # *?
|
|
|
|
|
'+' n expr-cont doPossesiveStar # *+
|
|
|
|
|
default expr-cont doStar
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# quant-plus Scanning a '+' quantifier. Need to look ahead to decide
|
|
|
|
|
# between plain '+', '+?', '++'
|
|
|
|
|
#
|
|
|
|
|
quant-plus:
|
|
|
|
|
'?' n expr-cont doNGPlus # *?
|
|
|
|
|
'+' n expr-cont doPossesivePlus # *+
|
|
|
|
|
default expr-cont doPlus
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# quant-opt Scanning a '?' quantifier. Need to look ahead to decide
|
|
|
|
|
# between plain '?', '??', '?+'
|
|
|
|
|
#
|
|
|
|
|
quant-opt:
|
2002-10-29 01:20:15 +00:00
|
|
|
'?' n expr-cont doNGOpt # ??
|
|
|
|
|
'+' n expr-cont doPossesiveOpt # ?+
|
|
|
|
|
default expr-cont doOpt # ?
|
2002-10-22 00:09:32 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# Interval scanning a '{', the opening delimiter for an interval specification
|
2003-01-16 01:12:04 +00:00
|
|
|
# {number} or {min, max} or {min, }
|
2002-10-22 00:09:32 +00:00
|
|
|
#
|
|
|
|
|
interval-open:
|
2003-01-16 01:12:04 +00:00
|
|
|
white_space n interval-open # TODO: is white space allowed here in non-free mode?
|
|
|
|
|
digit_char interval-lower
|
|
|
|
|
default errorDeath doIntervalError
|
|
|
|
|
|
|
|
|
|
interval-lower:
|
|
|
|
|
digit_char n interval-lower doIntevalLowerDigit
|
|
|
|
|
',' n interval-upper
|
|
|
|
|
'}' n interval-type doIntervalSame # {n}
|
|
|
|
|
default errorDeath doIntervalError
|
|
|
|
|
|
|
|
|
|
interval-upper:
|
|
|
|
|
digit_char n interval-upper doIntervalUpperDigit
|
|
|
|
|
'}' n interval-type
|
|
|
|
|
default errorDeath doIntervalError
|
|
|
|
|
|
|
|
|
|
interval-type:
|
|
|
|
|
'?' n expr-cont doNGInterval # {n,m}?
|
|
|
|
|
'+' n expr-cont doPossesiveInterval # {n,m}+
|
|
|
|
|
default expr-cont doInterval # {m,n}
|
2002-10-22 00:09:32 +00:00
|
|
|
|
|
|
|
|
|
2002-10-24 22:16:07 +00:00
|
|
|
#
|
|
|
|
|
# backslash # Backslash. Figure out which of the \thingies we have encountered.
|
|
|
|
|
# The low level next-char function will have preprocessed
|
|
|
|
|
# some of them already; those won't come here.
|
|
|
|
|
backslash:
|
|
|
|
|
'A' n term doBackslashA
|
2002-10-28 17:18:44 +00:00
|
|
|
'B' n term doBackslashB
|
|
|
|
|
'b' n term doBackslashb
|
2002-11-11 18:49:49 +00:00
|
|
|
'd' n expr-quant doBackslashd
|
2002-10-31 01:58:01 +00:00
|
|
|
'D' n expr-quant doBackslashD
|
2002-10-28 17:18:44 +00:00
|
|
|
'G' n term doBackslashG
|
2002-11-11 18:49:49 +00:00
|
|
|
'N' n expr-quant doNamedChar # \N{NAME} named char
|
|
|
|
|
'p' expr-quant doProperty # \p{Lu} style property
|
|
|
|
|
'P' expr-quant doProperty
|
2002-10-31 23:01:54 +00:00
|
|
|
'Q' n term doEnterQuoteMode
|
|
|
|
|
'S' n expr-quant doBackslashS
|
|
|
|
|
's' n expr-quant doBackslashs
|
2002-10-31 01:58:01 +00:00
|
|
|
'W' n expr-quant doBackslashW
|
|
|
|
|
'w' n expr-quant doBackslashw
|
|
|
|
|
'X' n expr-quant doBackslashX
|
2002-11-06 02:35:20 +00:00
|
|
|
'x' n expr-quant doBackslashx
|
2002-10-28 17:18:44 +00:00
|
|
|
'Z' n term doBackslashZ
|
|
|
|
|
'z' n term doBackslashz
|
2003-01-21 04:56:14 +00:00
|
|
|
'0' n expr-quant doOctal
|
2003-01-24 02:05:03 +00:00
|
|
|
digit_char n expr-quant doBackRef # Will scan multiple digits
|
|
|
|
|
eof errorDeath doEscapeError
|
|
|
|
|
default n expr-quant doLiteralChar # Escaped literal char.
|
2002-10-22 00:09:32 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#
|
|
|
|
|
# errorDeath. This state is specified as the next state whenever a syntax error
|
|
|
|
|
# in the source rules is detected. Barring bugs, the state machine will never
|
|
|
|
|
# actually get here, but will stop because of the action associated with the error.
|
|
|
|
|
# But, just in case, this state asks the state machine to exit.
|
|
|
|
|
errorDeath:
|
|
|
|
|
default n errorDeath doExit
|
|
|
|
|
|
|
|
|
|
|
