Expression Lexing
-
Introduce the topic of lexical analysis in a programming language such as
Java
. -
Develop a robust lexer that is successful regardless of the whitespace.
-
Should be able to parse both
"(6 * a) + (b / 4)"
and"(6*a)+(b/4)"
.
-
-
Create a project called
W9L1
-
Import
ExpressionLexing-starter.jar
into yourW9L1
project.
Your directory should look like this:
W9L1/ └── src ├── Lexer.java └── TestCode.java
Lexical analysis is the first phase of a compiler. It involves taking a series of words and breaking them down into tokens by removing whitespace and comments.
The Lexer has several different versions of a lexing method for identifying tokens within an expression.
-
The first method is called
scannerLexer
, and uses aScanner
object. -
The second method is called
splitLexer
, which uses the methodString.split()
. -
The third method is call
tokenizerLexer
and is based on aStringTokenizer
object.
Use the javadoc to implement the methods is the Lexer
class.
HINT: After a token is returned from each of the different lexing methods,
call the String.trim()
method to remove extra whitespace from the beginning
and end of the string. If the token is empty, do not add it to the
ArrayList
.
Answer the questions at the bottom of Lexer.java
Use the following resources to learn more about Regular Expressions
and to answer the attached questions.
-
Regex 101. Breaks down and explains a given expression. (Select python)
Submission
To receive credit for this recitation show your TA or helper that your program
passes the TestCode and answers the questions in Lexer.java
.