Coverage Lab
Philosophy
Writing code is hard, and it’s never right the first time. It’s best to
find the problems in our code before the GTA/boss/customers find them.
That’s why we write test cases. However, the question arises: Did we
test all of our code? People have a natural tendency to test only the
parts of the code that they’re confident about, and shy away from the
questionable parts.
If only there were some way to know if we’d tested it all!
Description
gcov is a code coverage tool. It tells you how many times each line
in your code has been executed. This tells you:
- whether all of the code is being tested
- which parts of the code are taking up the most time
Granted, #2 is only an approximation, since it only tells
you how many times the line is being executed. Not all
lines of code take the same amount of time—consider the assignment
of an integer, as opposed to calculating a cosine. However, it’s
often a good-enough approximation.
In this lab, we will use gcov to:
- see if we’ve tested some code adequately
- make the code more efficient
Tasks
- Copy the directory
~cs253/Lab/Coverage
to a convenient location in your home directory.
It contains a Date class in Date.h
and Date.cc, and a test program in
main.cc.
- You will modify
Date.cc and main.cc, wrap them up in
a tar file, and turn them in for credit.
- Take a minute or two to look at the code,
including the test cases in main().
- Build the code; note the interesting g++ options used:
make
- Run the code:
./prog
- Create coverage data:
gcov -r Date.cc
- Look at the coverage data:
more Date.cc.gcov
Each line contains the number of times that it was executed
(or ##### for never executed, or - for a non-executable line),
a line number, and the source code for that line.
- Find the lines that contain
#####. Those lines weren’t tested.
- Add test cases to main() for those untested lines.
- Recompile (make), rerun (
./prog),
and recreate the coverage data (gcov -r Date.cc).
- Look at the coverage data, and verify that all code is now tested.
- It’s time to look for inefficient code. Look at
Date.cc.gcov,
and find the lines that start with the largest number.
It should be the routine days_per_month(). Resist the urge
to optimize that code. Instead, find out who’s calling it
hundreds of millions of times. The culprit should be line 87 or so.
- Observe the code above line 87 that’s commented out with
#if 0.
Try making that code active (change #if 0 to #if 1),
recompile, rerun, recreate coverage data, and see if it helped.
- Look at the coverage data for the function
leap(). Observe how it
almost always calculates the modulus (%) three times for each
year. Since most years are not leap years, this is silly. Assume
that the % operator, which has to perform division, is slow.
Make this code more efficient, without sacrificing clarity, and prove
it using gcov data. Before you do that, however, run
./prog and save the output. After you make your improvements to
leap(), run ./prog again and diff the output, so you’ll know
if you broke it or not.
For Fame & Glory (but no extra points)
- Instead of
gcov -r Date.cc, run gcov -rb Date.cc, for branch
flow analysis.
- Observe the annoying output due to calling assert().
- Change the
Makefile to compile with -DNDEBUG, which
turns off assertions. Recompile, rerun, and see what
gcov -rb Date.cc gives you.
- Look at the branch coverage for
leap() and see if it makes sense.
- Look at the branch coverage for
operator>>.
Note that some branches are taken 0% of the time. Why?
What still isn’t being tested?
- Add tests to cover those cases.
Create a tar file
Create results.tar, like this:
tar -cvf results.tar Makefile main.cc Date.h Date.cc data
How to submit your work:
In Canvas, check in the
file
results.tar to the assignment “Lab08”.
It’s due 10:00:00ᴘᴍ MT Saturday, with a 24-hour late period for a 25% penalty.
How to receive negative points:
Turn in someone else’s work.