TokenStack

Cite.util. TokenStack

Create a TokenStack for parsing strings with complex escape sequences.

Constructor

(protected) new TokenStack(array)

Source:
Parameters:
Name Type Description
array Array.<String>

list of tokens

Methods

(protected, static) getMatchCallback(pattern) → {Cite.util.TokenStack~match}

Get a single callback to match a token against one or several patterns.

Source:
Parameters:
Name Type Description
pattern Cite.util.TokenStack~pattern

pattern

Returns:
Type:
Cite.util.TokenStack~match

Match callback

(protected, static) getPatternText(pattern) → {String}

Get string representation of pattern.

Source:
Parameters:
Name Type Description
pattern String | RegExp

pattern

Returns:
Type:
String

string representation

(protected) consume(patternopt, options) → {String}

Consumes all consecutive tokens matching pattern. Throws if number of matched tokens not within range min-max.

Source:
Parameters:
Name Type Attributes Default Description
pattern Cite.util.TokenStack~pattern <optional>
/^[\s\S]$/

pattern

options Object
Name Type Attributes Default Description
inverse Boolean <optional>
false

invert pattern

min Number <optional>
0

mininum number of consumed tokens

max Number <optional>
Infinity

maximum number of matched tokens

tokenMap Cite.util.TokenStack~tokenMap <optional>

map tokens before returning

tokenFilter Cite.util.TokenStack~tokenFilter <optional>

filter tokens before returning

Throws:
  • Not enough tokens

    Type
    SyntaxError
  • Too many tokens

    Type
    SyntaxError
Returns:
Type:
String

consumed tokens

(protected) consumeN(length) → {String}

Consume n tokens. Throws if not enough tokens left

Source:
Parameters:
Name Type Description
length Number

number of tokens

Throws:

Not enough tokens left

Type
SyntaxError
Returns:
Type:
String

consumed tokens

(protected) consumeSequence(sequence) → {String}

Consume a pattern spanning multiple tokens ('sequence').

Source:
Parameters:
Name Type Description
sequence Cite.util.TokenStack~sequence

sequence

Throws:

Expected sequence, got tokens

Type
SyntaxError
Returns:
Type:
String

consumed tokens

(protected) consumeToken(patternopt, options) → {String}

Consume a single token if possible, and throw if not.

Source:
Parameters:
Name Type Attributes Default Description
pattern Cite.util.TokenStack~pattern <optional>
/^[\s\S]$/

pattern

options Object
Name Type Attributes Default Description
inverse Boolean <optional>
false

invert pattern

spaced Boolean <optional>
true

allow leading and trailing whitespace

Throws:

Unexpected token at index: Expected pattern, got token

Type
SyntaxError
Returns:
Type:
String

token

(protected) consumeWhitespace(patternopt, options) → {String}

Consume a single token if possible, and throw if not.

Source:
Parameters:
Name Type Attributes Default Description
pattern Cite.util.TokenStack~pattern <optional>
/^\s$/

whitespace pattern

options Object
Name Type Attributes Default Description
optional Boolean <optional>
true

allow having no whitespace

Throws:

Unexpected token at index: Expected whitespace, got token

Type
SyntaxError
Returns:
Type:
String

matched whitespace

(protected) matches(pattern) → {Boolean}

Match current token against pattern.

Source:
Parameters:
Name Type Description
pattern Cite.util.TokenStack~pattern

pattern

Returns:
Type:
Boolean

match

(protected) matchesSequence(pattern) → {Boolean}

Match current token against pattern.

Source:
Parameters:
Name Type Description
pattern Cite.util.TokenStack~sequence

pattern

Returns:
Type:
Boolean

match

(protected) tokensLeft() → {Number}

Get a number representing the number of tokens that are left.

Source:
Returns:
Type:
Number

tokens left

Type Definitions

match(token, index, stack) → {Boolean}

Source:
Parameters:
Name Type Description
token String

token

index Number

token index

stack Array.<String>

token stack

Returns:
Type:
Boolean

match or not

pattern

TokenStack pattern

Source:
Type:

sequence

TokenStack pattern sequence

Source:
Type:

tokenFilter(token) → {Boolean}

Source:
Parameters:
Name Type Description
token String

token

Returns:
Type:
Boolean

keep or not

tokenMap(token) → {String}

Source:
Parameters:
Name Type Description
token String

token

Returns:
Type:
String

new token