How to use Lexical Analysis

The following code fragments are taken from an example that implements a simple Reverse Polish Notation calculation engine.

A TLex object may be constructed as an empty object, from another TLex, or from an existing string as below.

TInt RPNCalc(const TDesC& aCommand, TReal& aReturnValue)
    {
    ...
    TLex input (aCommand) ;
    ...

Code can then proceed to move through the TLex data to:

• mark positions for rolling back to later

• delimit the start of lexical tokens

• delete parts of the string held by the TLex object.

...
input.Mark() ; // Remember where we are.
input.SkipCharacters() ; // Move to end of character token.
...
_LIT(KTextMemset,"MEMSET");
if ( input.TokenLength() != 0 ) // if valid potential token
    {
    TPtrC token = input.MarkedToken() ; // then extract token
    if ( token.CompareF(KTextMemset == 0) // and test.
        {
        ...
        }
    ...
    }
 ...

Analysis can also be done by character using functions that move through the TLex data, extracting, returning and jumping specified character lengths. For example:

...
// ensure we are looking at a digit or sign
if (!(input.Peek().IsDigit() || (input.Peek() == '.') ) )
    {
    return KErrNotFound ;
    }
...
// deal with sign
if (input.Peek() == '+')
    {
    input.Inc();
    }

Additionally, numeric conversion functions permit a variety of numeric formats to be extracted from the TLex data, with provision for conversion using the most common number systems (radixes).

...
if (input.Val(extractUint) == KErrNone)
    {
    stack.Push(TReal(extractUint));
    }
else if (input.Val(extractReal) == KErrNone)
        {
        stack.Push(extractReal);
        }
...

where stack, is an instance of a class implementing a stack.