Code Coverage Lab                

Philosophy                

Writing code is hard, and it’s never right the first time. It’s best to find the problems in our code before the GTA/boss/customers find them. That’s why we write test cases. However, the question arises: Did we test all of our code? People have a natural tendency to test only the parts of the code that they’re confident about, and shy away from the questionable parts.                 

If only there were some way to know if we’d tested it all!                 

Description                

gcov is a code coverage tool. It tells you how many times each line in your code has been executed. This tells you:                 

1. whether all of the code is being tested
2. which parts of the code are taking up the most time

Granted, #2 is only an approximation, since it only tells you how many times the line is being executed. Not all lines of code take the same amount of time—consider the assignment of an integer, as opposed to calculating a cosine. However, it’s often a good-enough approximation.                 

In this lab, we will use gcov to:

1. see if we’ve tested some code adequately
2. make the code more efficient

Tasks                

1. Copy the directory ~cs253/Lab/Coverage to a convenient location in your home directory.
2. Look at code.cc. Take a minute or two to look at the code, including the test cases in main().
3. Build the code:
   make
   Note the interesting g++ options.
4. Run the code:
   ./code
5. Create coverage data:
   gcov code.cc
   Ignore the confusing output.
6. Look at the coverage data:
   more code.cc.gcov
   Each line contains the number of times that it was executed (or ##### for never executed, or - for a non-executable line), a line number, and the source code for that line.
7. Find the lines that start with #####. Those lines weren’t tested.
8. Add test cases to main() for those untested lines.
9. Recompile (make rerun (./code), and recreate the coverage data (gcov code.cc).
10. Look at the coverage data, and verify that all code is now tested.
11. Show your results to the TA.
12. It’s time to look for inefficient code. Look at code.cc.gcov, and find the lines that start with the largest number. It should be the routine days_per_month(). Resist the urge to optimize that code. Instead, find out who’s calling it tens of millions of times. The culprit should be line 112 or so.
13. Observe the code above line 112 that’s commented out with #if 0. Try making that code active, recompile, rerun, recreate coverage data, and see if it helped.
14. Look at the coverage data for the function leap(). Observe how it almost always calculates the modulus (%) three times for each year. Since most years are not leap years, this is silly. Make this code more efficient, without sacrificing clarity, and prove it using gcov data. Before you do that, however, run code and save the output. After you make your improvements to leap(), run code again and diff the output, so you’ll know if you broke it or not.
15. Show your results to the TA.

For Fame & Glory                

1. Instead of gcov code.cc, run gcov -b code.cc, for branch flow analysis.
2. Observe the annoying output due to calling assert().
3. Change the Makefile to compile with -DNDEBUG, which turns off assertions. Recompile, rerun, and see what gcov -b code.cc gives you.
4. Look at the branch coverage for leap() and see if it makes sense.
5. Look at the branch coverage for operator>>. Note that some branches are taken 0% of the time. Why? What still isn’t being tested?
6. Add tests to cover those cases.