CS453 Colorado State University ========================================== Syntax-Directed Evaluation and Shift-Reduce Parsing, Addendum ========================================== ------------- Announcements Show testing framework in recitation tomorrow morning regression script driver that opens input file and puts something into output file Quiz 3 is due tonight ------------- Plan for today [TODO fix] Shift-reduce parsing of some ambiguous grammars Syntax-directed code generation for PA3 Summary on precedence and associativity in JavaCUP ------------------------------------------------- Shift-reduce parsing of some ambiguous grammars (slides 10 and 11) For each of the below examples do the following: - first do the right-most derivation - then do the shift-reduce parse by following the right-most derivation in reverse - note how precedence and associativity declarations can resolve shift-reduce conflicts ------------------- Ambiguous example 1 // This time without the $ in the grammar. (1) S’ -> E (2) E -> E + E (3) E -> E * E (4) E -> ( E ) (5) E -> NUM Input 42 + 6 * 7 Rightmost derivation S' => E => E + E => E + E * E => E + E * 7 => E + 6 * 7 => 42 + 6 * 7 Order of reductions based on reverse of rightmost derivation E => 42 E => 6 E => 7 E => E * E E => E + E S' => E Stack Input Action $ 42 + 6 * 7 $ shift token NUM(42) $ 42 + 6 * 7 $ reduce using production (5) $ E + 6 * 7 $ shift + and NUM(6) $ E + 6 * 7 $ reduce using production (5) $ E + E * 7 $ shift * NUM(7) $ E + E * 7 $ reduce using (5) $ E + E * E $ reduce using E -> E*E $ E + E $ reduce using E -> E+E $ E $ reduce S' -> E $ S' $ accept ------------------- Ambiguous example 2 (1) S’ -> E (2) E -> E + E (3) E -> ( byte ) E (4) E -> NUM Input ( byte ) 3 + 4 Rightmost derivation S' => E => E + E => E + NUM(4) => ( byte ) E + 4 => (byte) 3 + 4 Order of reductions based on reverse of rightmost derivation E -> 3 E -> (byte) E E -> 4 E -> E+ E Stack Input Action $ ( byte ) 3 + 4 $ shift ( byte ) 3 $ (byte) 3 + 4 $ reduce 3 to E $ (byte) E + 4 $ reduce E -> (byte) E $ E + 4 $ shift + 4 $ E + 4 $ reduce 4 to E $ E + E $ reduce E + E to E $ E $ accept [Switch to writing on board] --------------------------------------------------- Summary on precedence and associativity in JavaCUP Tokens and productions are given precedence. Tokens are given precedence level in precedence declarations. Productions are given precedence level of last token on right-hand side. When in shift-reduce conflict, shift if look ahead token has higher precedence than reduce production reduce if reduce production has higher precedence than the look ahead token Within the same precedence level associativity determines whether to shift or reduce. left associativity results in a reduce right associativity results in a shift --------------------------------------------------- Precedence declarations for ambiguous example 2 (1) S’ -> E (2) E -> E + E (3) E -> ( byte ) E (4) E -> NUM ------------------------- Process for handling expressions for PA3-PA7 1. look up precedence for Java 2. assign precedence to tokens and productions where needed ---------------------------------------------------- What about (E)? Do we need to declare a precedence or associativity for E -> ( E )? Try some examples and say why or why not? Input examples: ( 3 + 4 ) * 5 5 - ( 2 - 3 ) Precedence declarations for ambiguous example 1 (1) S’ -> E (2) E -> E + E (3) E -> E * E (4) E -> ( E ) (5) E -> NUM ----------------------- Other shift-reduce parser conflicts ----------------- shift-reduce conflict due to dangling else Stack Input $ ... if E then S else ... $ ----------------- reduce-reduce conflict example Latex example (thanks Wim!) E -> E _ E | E ^ E | E _ E ^ E Assume _ and ^ have the same precedence and are right associative. Stack Input $ E _ E ^ E ... $ [Don't know how to fix this yet.] ----------------------- LR parse tables (slide 13) - LR(k) parser is a kind of shift-reduce parser - left-to-right parse - rightmost derivation - k-token look ahead - You can find another good example in the book and on wikipedia entry for LR parser. Doing examples is important to aid in understanding and study for the midterm and final. - rules for the parse table Look at state at top of stack and input symbol to find action in table shift(n): advance input, push state n on stack - Putting input on stack as well to help us humans keep track. As is described in the book, computers just need the state numbers. reduce(k): pop rhs of grammar rule k look up state on top of stack and lhs for goto n push lhs(k) and n onto stack accept: stop and success error: stop and fail - Parsing the string ((a)) using the LR parse table generate by by JavaCUP. - See slide for grammar and LR parse table. Stack Input Action $0 ((a))$ shift 3 $0(3 (a))$ shift 3 $0(3(3 a))$ shift 1 $0(3(3a1 ))$ reduce by [3] S -> ID, goto 4 $0(3(3S4 ))$ shift 5 $0(3(3S4)5 )$ reduce by [0] S -> ( S ), goto 4 $0(3S4 )$ shift 5 $0(3S4)5 $ reduce by [0] S -> ( S ), goto 2 $0S2 $ accept -------------------------------------------------------------- Syntax-directed expression evaluation and shift-reduce parsing The values associated with nonterminals can be stored on the stack. ------------------------------------------------ Have all you need for PA3, Suggested Progression (1) Set up testing framework. Will initially fail on all test cases. - Kiley showed you how to do driver in recitation. - Use provided Makefile to generate a MJPA3.jar. - See mailing list for two ways to handle input and output files. (2) Specify and test the lexer. // construct the lexer, // the lexer will be the same for all of the parsers Yylex lexer = new Yylex(new FileReader(filename)); // print out all of the tokens for the file java_cup.runtime.Symbol token = lexer.next_token(); while (token.sym != sym.EOF) { System.out.println(token.toString()); token = lexer.next_token(); } System.exit(0); (3) Specify and test parser with only byte casts of int literals and color literals for expressions. (4) Implement and test with testing framework code gen with only byte casts of int literals and color literals for expressions. (5) Put in all expressions for PA3 and test with testing framework. ------------------------ mstrout@cs.colostate.edu, 2/24/11