new StringTokenizer()
- Source:
Methods
tokenize(string) → {Array.<String>}
- Source:
Parameters:
| Name | Type | Description |
|---|---|---|
string |
String
|
the string to be splitted into multiple tokens |
Returns:
- Type:
-
Array.<String>
an array of tokens. the
Examples
const tokens = tokenizer.tokenize('@amy import(abc/def.html).with(abc.d.e.f.g)')
// tokens[0] => @amy
// tokens[1] => import(abc/def.html).with(abc.d.e.f.g)const tokens = tokenizer.tokenize('@amy import abc/def.html with abc.d.e.f.g ')
// tokens[0] => @amy
// tokens[1] => import
// tokens[2] => abc/def.html
// tokens[3] => with
// tokens[4] => abc.d.e.f.g