CS453 Colorado State University ============================================== Midterm Review ============================================== ---------------------------------- Finishing up the AST notes ------------------- Examples to test understanding - put example on board and ask what numbering we get, preorder or postorder? public class Test1 extends DepthFirstVisitor { private HashMap mNodeToNum = new HashMap(); int count = 0; ... public void defaultIn(Node node) { mNodeTNum.put(node,count); count++; } - what is the sequence of calls? [This will not be on midterm] Suggested exercise: how would we write a visitor that reprints the expressions with parentheses around all of the sub-expressions so as to indicate the full expression evaluation order? --------------------------------------------------------------------- Evaluating byte and integer expressions in the code generator visitor Map expression nodes to an integer value. Look up child values, perform operation, and then map computed value to current node. [Have students suggest some code based on previous example and then show them all of the below] // Snippets from concrete visitor. class AVRgenVisitor extends DepthFirstVisitor { HashMap mExpVal = new HashMap(); public void outPlusExp(PlusExp node) { // Look up child values, Integer lexpval = mExpVal.get(node.getLExp()); Integer rexpval = mExpVal.get(node.getRExp()); // perform operation, Integer value = lexpval + rexpval; // and then map computed value to current node. mExpVal.put(node, value); } public void outByteCast(ByteCast node) { // Look up child value, Integer expval = mExpVal.get(node.getExp()); // perform operation, Integer value = expval; // and then map computed value to current node. mExpVal.put(node, value); } } ---------------------------------- General approach for midterm study -While studying put together a 1 sided 8.5x11 study sheet for use while taking the midterm -Be prepared for questions based on the assignments. Everyone should understand how the programs work. For example, Given the following set of statements, (1) draw the parse tree, (2) associate values with expressions to show expression evaluation, and (3) write the AVR code for the setPixel statements. -Look at questions from old midterms, but keep in mind that the material changes somewhat from year to year. -Do suggested exercises in notes and check your answers with someone else or in lab hours, during review, or set up a meeting. -Redo examples from the quizzes and hws -Redo any examples we did in class -Review all posted notes and slides and notes you took in class -Reread assigned parts of the book ----------------------------- Lexical Analysis or scanning Terminology Regular expressions DFA NFA Techniques Creating a scanner for a set of tokens -creating a DFA for lexing a set of tokens ids that must start with a letter and have letters or numbers of that the key word "in" the C equality operator 1) first specify regular expressions ID=LETTER(LETTER|DIGIT)* IN="in" EQ="==" 2) then create NFAs 3) then create DFA 4) then implement DFA ------------------------------ Syntactic Analysis or Parsing Terminology and concepts context-free grammars terminal, non-terminal, symbol syntax-directed translation actions attributes parse trees abstract syntax trees ambiguity derivation LL(k) LR(1) top-down and bottom-up parsing precedence associativity left recursion in a grammar left factoring pre and post order depth first traversal Techniques Make a grammar LL(1) by left refactoring and removing left recursion. Calculate FIRST and FOLLOW sets and a predictive parsing table. Write a predictive parser with error handling. Write syntax-directed definitions, or actions Draw implicit parse tree and AST for an example string given a grammar and a list of AST nodes. Use JavaCUP precedence and associativity to make an expression grammar unambiguous. Rewrite a grammar to make an expression grammar unambiguous. Perform a shift-reduce parse with or without an LR parsing table. ------------------------------ Code generation and AVR How many bits are there in each register? What is the calling convention that avr-gcc uses? How are parameters passed? How is the return value passed from callee to caller? ------------------------------ Visitor Design Pattern Tell what a visitor does by looking at visitor code. How to use the in and out methods provided by DepthFirstVisitor. [Not doing this on midterm] Show the sequence of methods being called when applying a visitor to a data structure. [Not doing this on midterm] ------------------------------ Example Problems ------------------------------ - Create regular expressions, NFAs, and combined DFA for a set of tokens. while, when, and identifiers Regular Expressions while when LET (LET | DIG)* assuming the regular definitions LET=[A..Za..z] and DIG=[0..9] NFAs and DFA (see pdf linked to progress page for today) Implementation in code Assume the following routines have been defined: private int nextchar() { // returns the next character in the stream } private void match(int c) throws ParseException, IOException { int next = nextChar(); if (c != next) { throw new ParseException("Expected character '" + (char) c + "' and got '" + (char) next + "'.", line, pos); } } //=========== Straight forward DFA recognizer except handle //=========== missing edges out of accept states differently. public Token scan() { int c = nextChar(); int state = 0; // states are named after the first state in the set of states while ( c != EOF ) { switch (state) { case 0: if (c=='w') { state = 2; } else if (Character.isLetter(c)) { state = 10; } else { /* error */ } break; case 2: if (c=='h') { state = 3; } else if (Character.isLetter(c)) { state = 10; } else { return Token.ID } break; case 3: if (c=='i') { state = 4; } else if (c=='e') { state = 8; } else if (Character.isLetter(c)) { state = 10; } else { return Token.ID } break; case 4: if (c=='l') { state = 5; } else if (Character.isLetter(c)) { state = 10; } else { return Token.ID } break; case 5: if (c=='e') { state = 6; } else if (Character.isLetter(c)) { state = 10; } else { return Token.ID } break; case 6: if (Character.isLetter(c)) { state = 10; } else { return Token.WHILE } break; case 8: if (c=='n') { state = 11; } else if (Character.isLetter(c)) { state = 10; } else { return Token.ID } break; case 11: if (Character.isLetter(c)) { state = 10; } else { return Token.WHEN } break; case 10: if (Character.isLetter(c)) { state = 10; } else { return Token.ID } break; } } // =========== Another possible implementation, similar to what // =========== was suggested in the book. public Lexer(PushbackReader fb) { filebuf = fb; // reserve all of the keywords this.reserve("while", Token.Tag.KW_WHILE); this.reserve("when", Token.Tag.KW_WHEN); } public Token scan() { if (Character.isLetter(c)) { retval = getWordToken(); } else { // error } return retval; } private Token getWordToken() { int first_char_pos = this.pos; // get all letters and numbers and look up the keyword while (Character.isLetterOrDigit(this.nextChar())); // find token tag for specified word and create a Word token Token.Tag toktype = this.lookup(this.getLexeme()); if (null == toktype) { return TokenID; } else { return toktype; } } ------------------------------ - Write a syntax directed definition (or actions) that would create a linked list if given the following grammar: non terminal LinkedList exp_list; non terminal IExp exp; ... exp_list ::= exp | exp_list COMMA exp ------------------------------ - Assume a small scripting language that manipulates strings. To concatenate strings s1 and s2, use the + operator: s1 + s2 The power operator concatenates the same string the specified number of times. s1 ^ 2 a) Write a grammar for this language and use the JavaCUP precedence keywords to give s1 the power operator higher precedence than the concatenation operator. b) Associate actions with the grammar rules that print out the final string to standard out. Assume that the type for the string tokens is the Java String type. ------------------------------ For the following grammar, construct a recursive-descent parser. grammar from 2.4.1c in book S -> 0 S 1 | 0 1 What did we have to do to fix this grammar? ----------- Given an expression grammar, write the JavaCUP actions that would print the postfix version of the expression to standard out. E -> E:l + E:r {: System.out.print(" + "); :} | E:l * E:r {: System.out.print(" * "); :} | NUM:n {: System.out.print(n); :} ----------- Perform a shift reduce parse by showing the stack, input, and action configurations given an input string, a grammar, and a set of JavaCUP precedence declarations. precedence left + precedence right ^ E -> E ^ E | E + E | ( E ) | NUM ---------- Perform a shift reduce parse by showing the stack, input, and action configurations given an input string, a grammar, and an LR parse table. [0] S -> E [1] E -> E or B [2] | B [3] B -> t [4] | f t f or $ || S E B ---------------------------------------------- 0 s3 s4 || g1 g2 1 s5 acc || 2 r[2] r[2] || 3 r[3] r[3] || 4 r[4] r[4] || 5 s3 s4 || g6 6 r[1] || We did "t or f" on the board. ---------- Fix the precedence in a grammar WITHOUT using the JavaCup precedence keywords. Instead modify the grammar. precedence left + precedence right ^ E -> E ^ E | E + E | ( E ) | NUM ------------------------ mstrout@cs.colostate.edu, 3/3/11