INTERFACE M3Scanner;

IMPORT Rd, M3Token;

TYPE
  T = OBJECT
    token        : M3Token.T := M3Token.T.Comment;
    start        : CARDINAL  := 0;
    length       : CARDINAL  := 0;
    offset       : CARDINAL  := 0;
    line         : CARDINAL  := 0;
    column       : CARDINAL  := 0;
    msg          : TEXT      := NIL;
    buffer       : Buf       := NIL;
  METHODS
    next ();
  END;

  Buf = REF ARRAY OF CHAR;

TYPE
  Default <: T OBJECT METHODS
    initFromRd  (source        : Rd.T;
                 skip_comments := TRUE;
                 split_pragmas := TRUE): T;
    initFromBuf (buf           : Buf;
                 skip_comments := TRUE;
                 split_pragmas := TRUE): T;
  END;

END M3Scanner.

An M3Scanner.T, or scanner, parses a stream of characters and returns a stream of Modula-3 tokens. If s is a scanner, each call s.next() sets the values of s's fields to correspond to the next token in the stream.

The fields of a scanner are not to be modified by its client.

s.token is the class of the token.

In s.buffer[s.start .. s.start+s.length-1] are the characters that comprise the token.

s.offset is the character offset of the token relative to the beginning of the stream. The first character of the stream is at offset zero.

s.line is the line where the token occured relative to the beginning of the stream. The first line is one.

s.column is the character offset of the beginning of the token within the line that contains it. The first column is zero.

s.buffer contains the source being scanned. Modifying its contents may perturb the token stream or cause a checked runtime error.

s.msg describes the error that caused the Error token to be returned.

The scanner returned by NEW(Default).initFromRd(rd) will read the entire contents of rd into its buffer and initialize the scanner as a zero-length comment at offset zero.

The scanner returned by NEW(Default).initFromBuf(buf) will use buf as its buffer and initialize the scanner as a zero-length comment at offset zero.

If skip_comments is TRUE, outer-level comments will be returned as tokens. Otherwise, comments are ignored.

If split_pragmas is TRUE, the contents of pragmas will be scanned and returned as a stream of tokens between Begin_pragma and End_pragma tokens. If split_pragmas is false, the entire pragma is returned in a single Begin_pragma token.

• m3scan/src/M3Scanner.m3
• m3scanner/src/M3Scanner.m3

• m3scan/src/M3Scanner.m3
• m3scanner/src/M3Scanner.m3