CS 453 Programming Assignment #3 — Lexical Analysis
Due Friday, February 25th (by 6:00pm)
Preliminary Subversion Log
Due Monday, February 20 (by 11:59pm)
Sit down and take a deep breath. This project is the first step in
building a compiler for
MiniJava! You will use a tool called JLex to construct a lexical analyzer
that recognizes all of the tokens in MiniJava. You will describe
the tokens in MiniJava to JLex using regular expressions.
We provide you with PA3-start.tar,
which includes the JLex and JavaCUP packages, a driver you will
use for your lexer, and the start of a minijava.lex file for specifying
the MiniJava tokens. After downloading PA3-start.tar, do the following:
JLex/ contains implementation of JLex tool.
java-cup-11a-runtime.jar contains definition for the Symbol class, which is what we will eventually be using to talk to the parser generated by JavaCUP.
For more information see a
JLex Tutorial and do a google search on JLex.
- Untar the start directory.
% tar xf PA3-start.tar
It should unpack into the following:
- Rename MJLexerStart to MJLexer-groupname, and start a subversion repository for MJLexer-groupname.
- Start a new eclipse project with the existing files in MJLexer-groupname.
File -> New -> Project
Use project specific JRE (1.5 or 5.0)
Build project from existing source files (MJLexer-groupname)
Remove TestCases from the buildpath
- Put java-cup-11a-runtime.jar in CLASSPATH.
Eclipse -> Preferences -> Java -> Build Path -> Classpath Variables
Path: put in full path for java-cup-11a-runtime.jar,
which is in the mjparser/ subdirectory
Project -> Properties -> Libraries -> Add Variable
- Generate parser.java and sym.java from minijava.cup. See the notes in src/mjparser/minijava.cup.
- Move the JLex directory into the mjparser/ subdirectory.
% cd PA3-start
% mv JLex MJLexer-groupname/src/mjparser/
- Do a refresh in the Eclipse project. Remove JLex from the project build in Eclipse by right clicking on it in Eclipse and
- See the minijava.lex file for instructions for generating the Yylex.java file.
- Run LexDriver on the TestCases/test.file to observe the following output:
[1, 0] AND
[1, 2] BOOLEAN
[1, 9] SEMI
The MJLexerStart code includes a starting lexer specification in
the minijava.lex file.
An abbreviated JLex tutorial can be found at http://pages.cs.wisc.edu/~bodik/cs536/NOTES/2a.JLEX.html.
We recommend you specify one token at a time and then unit test each token
with a file that contains instances of that token.
Create a lexical description for MiniJava in minijava.lex,
and verify that it is correct
by feeding it to JLex and testing the constructed lexical analyzer.
The BNF grammar for MiniJava can be found
here. From the grammar, you can determine which tokens are required
by the language, and describe them formally. (The context free
how to string these tokens together to construct a legal program. You
won't be specifying an order for the tokens yet — just what the
tokens are. e.g., "main /* hi */ System.out.println" is a legal
input file at this point.) You will probably find this information from the MiniJava language reference
useful as well. It describes identifiers and other items in more
detail. It says that you should be able to handle nested
/*...*/ style comments, but I'm giving you a break: You need
to handle both // and /*...*/ style comments, but you
may assume that a comment (of any style) never appears within another
Preliminary Subversion Log
Submit assignment using checkin utility
Make sure you test your lexer thoroughly — ensure that it
recognizes all legal tokens, and that it rejects illegal tokens.
You should test your finished lexer on many sample programs and some files
with random orderings of legal tokens.
- Create a jar file that contains all the source code, the .class files, the minijava.lex and minijava.cup files, a README file, and a subversion.txt file. We should be able to use the jar file to run the LexDriver class.
- Create a mainClass.txt file in MJLexer-groupname/src/ with the following contents:
- Also put the subversion.txt and README files into the src/ subdirectory.
- While in the src/ subdirectory, do the following commands:
% javac -classpath .:mjparser/java-cup-11a-runtime.jar LexDriver.java
% jar cmf mainClass.txt MJLexer-groupname.jar *.class */*.class */*/*.class mjparser/minijava.* README subversion.txt
The README file should include your username(s), emails(s), and any
information you want the TA to have before grading your assignment.
e.g., this part is not working due to..., a favorite quote or joke
to amuse the TA, etc.
The subversion.txt file should include output from the following:
svnlook tree REPOSITORY_PATH svnlook must be issued on a department machine
~cs453/bin/checkin PA3 MJLexer-groupname.jar
Sanity Check (procedure TA will use to run your assignment):
% java -classpath java-cup-11a-runtime.jar -jar MJLexer-groupname.jar Test.java
% mkdir MJLexer-groupname/src/
% cd MJLexer-groupname/src/
% cp ../../MJLexer-groupname.jar .
% jar xf MJLexer-groupname.jar
Note that you need to have a copy of the java-cup-11a-runtime.jar file in the same directory as the MJLexer-groupname.jar file. We will provide our own copy of the runtime jar file for testing. We will be running your lexer on multiple test files. Also, the TA will be looking at the source files and possibly running JLex on the minijava.lex file.
Late assignments will be accepted up to 24 hours past the due date for a deduction of
20% and will not accepted past this period.
.... February 16, 2007
Originally written by Brad Richards 2006,
modified with permission by Michelle Strout and Dave Rostron 2006.
Project changed to use a new tool, Michelle Strout 2008.