# Regular expressions

This chapter covers the regular expressions or regex syntax and programming use. The regex model is an original implementation with its own syntax and execution model.

## Regular expression syntax

A regular expression is defined with a special `Regex` object. A regular expression can be built implicitly or explicitly with the use of the `Regex` object. The regex syntax uses the `[` and `]` characters as block delimiters. When used in a source file, the parser automatically recognizes a regex and built the object accordingly. The following example shows two equivalent methods for the same regex expression.

# syntax built-in regex (== [$d+] 2000) # true # explicit built-in regex (== (Regex "$d+") 2000) # true

In its first form, the `[` and `]` characters are used as syntax delimiters. The lexical analyzer automatically recognizes this token as a regex and built the equivalent `Regex` object. The second form is the explicit construction of the `Regex` object. Note also that the `[` and `]` characters are also used as regex block delimiters.

## Regex characters and meta-characters

Any character, except the one used as operators can be used in a regex. The `$` character is used as a meta-character -- or control character -- to represent a particular set of characters. For example, `[hello world]` is a regex which match only the `"hello world"` string. The `[$d+]` regex matches one or more digits. The following meta characters are built-in in the regex engine.

## Regular expression meta-characters

Character | Description
--- | ---
$a | matches any letter or digit
$b | matches any blank characters
$c | matches any combining alphanumeric
$d | matches any digit
$e | matches eol, cr and eos
$l | matches any lower case letter
$n | matches eol or cr
$s | matches any letter
$u | matches any upper case letter
$v | matches any valid afnix constituent
$w | matches any word constituent
$x | matches any hexadecimal characters

The uppercase version is the complement of the corresponding lowercase character set.

A character which follows a `$` character and that is not a meta character is treated as a normal character. For example `$[` is the `[` character. A quoted string can be used to define character matching which could otherwise be interpreted as control characters or operator. A quoted string also interprets standard escaped sequences but not meta characters.

(== [$d+] 2000) # true (== ["$d+"] 2000) # false

Combining alphanumerical characters can generate surprising result when used with Unicode string. Combining alphanumeric characters are alphanumeric characters and non spacing combining mark as defined by the Unicode consortium. In practice, the combining marks are the diacritics used with regular letter, such like the string "noël". The regular expression in this case, must be designed to find such complex character.

(== [$c+] "noël") # true (== [$a+] "noël") # false

Because the writing system uses a canonical decomposition for representing the Unicode string, it turns out that the string "noël" is in fact represented internally with 5 five characters, where the character 'ë' is broken into two characters, the letter 'e' and the diaeresis diacritic. This is also a good example of a 4 letters printable string with 5 character glyphs. As a consequence, the regular expression must be designed in such way that the diacritic is matched with the other characters.

## Regex character set

A character set is defined with the `<` and `>` characters. Any enclosed character defines a character set. Note that meta characters are also interpreted inside a character set. For example, `<$d+->` represents any digit or a plus or minus. If the first character is the `^` character in the character set, the character set is complemented with regards to its definition.

## Regex blocks and operators

The `[` and `]` characters are the regex sub-expressions delimiters. When used at the top level of a regex definition, they can identify an implicit object. Their use at the top level for explicit construction is optional. The following example is strictly equivalent.

# simple real number check const real-1 (Regex "$d*.$d+") # another way with [] characters const real-2 (Regex "[$d*.$d+]")

Sub-expressions can be nested -- that's their role -- and combined with operators. There is no limit in the nesting level.

# pair of digit testing (== [$d$d[$d$d]+] 2000) # true (== [$d$d[$d$d]+] 20000) # false

The following unary operators can be used with single character, control characters and sub-expressions.

## Regular expression operators

Operator | Description
--- | ---
* | match 0 or more times
+ | match 1 or more times
? | match 0 or 1 time
| | alternation

Alternation is an operator which work with a secondary expression. Care should be taken when writing the right sub-expression. For example the following regex `[$d|hello]` is equivalent to `[[$d|h]ello]`. In other word, the minimal first sub-expression is used when compiling the regex.

## Grouping

Groups of sub-expressions are created with the `(` and `)` characters. When a group is matched, the resulting sub-string is placed on a stack and can be used later. In this respect, the regex engine can be used to extract sub-strings. The following example extracts the month, day and year from a particular date format: `[($d$d):($d$d):($d$d$d$d)]`. This regex assumes a date in the form mm:dd:yyyy.

if (== (const re [($d$d):($d$d)]) "12:31") { trans hr (re:get 0) trans mn (re:get 1) }

Grouping is the mechanism to retrieve sub-strings when a match is successful. If the regex is bound to a symbol, the `get` method can be used to get the sub-string by index.

## Regex object

Although a regex can be built implicitly, the `Regex` object can also be used to build a new regex. The argument is a string which is compiled during the object construction.

## Literal object

A `Regex` object is a literal object. This means that the `to-string` method is available and that a call to the `println` special form will work directly.

const re (Regex "$d+") println re # $d+ println re:to-string # [$d+]

## Regex operators

The `==` and `!=` operators are the primary operators to perform a regex match. The `==` operator returns `true` if the regex matches the string argument from the beginning to the end of string. Such operator implies the begin and end of string anchoring. The `<` operator returns true if the regex matches the string or a sub-string or the string argument.

## Regex methods

The primary regex method is the `get` method which returns by index the sub-string when a group has been matched. The `length` method returns the number of group match.

if (== (const re [($d$d):($d$d)]) "12:31") { re:length # 2 re:get 0 # 12 re:get 1 # 31 }

The `match` method returns the first string which is matched by the regex.

const regex [$d+] regex:match "Happy new year 2000" # 2000

The `replace` method any occurrence of the matching string with the string argument.

const regex [$d+] regex:replace "Hello year 2000" "3000" # hello year 3000

## Argument conversion

The use of the `Regex` operators implies that the arguments are evaluated as literal object. For this reason, an implicit string conversion is made during such operator call. For example, passing the integer `12` or the string `"12"` is strictly equivalent. Care should be taken when using this implicit conversion with real numbers.