CS253: Software Development with C++

Fall 2019

Testing

Show Lecture.Testing as a slide show.

CS253 Testing

Philosophy of testing

  1. Get over it
  2. Trust, but verify
  3. Test the typical cases
  4. Test invalid input
  5. Test the limits
  6. Automation

Get over it

Trust, but verify

Doubting Thomas (I call him “Testing Thomas”)

Sure, whoever programmed this is really smart (especially if they’re you). It’s probably correct.

Test the typical cases

Test invalid input

  • Arguments
    • too many/too few
    • bad (e.g., -z)
    • duplicate (--high --high)
    • conflicting (--high --low)
  • Files
    • don’t exist (/foo/bogus)
    • bad permissions (/etc/shadow)
    • contain bad data (/bin/sync)
  • Numbers
    • integer
    • real
    • ±0.0
    • NaN
    • ±∞
    • too small
    • too big
    • too negative
    • fish
    • 3fish

Test the limits

Case study

Consider a command, zot, which takes a number of files as arguments, and an option which determines which files should be output. Think of it as a conditional cat.

    zot [-f first-last] file ...

Where first and last are optional inclusive ordinal file indices.

How do we test this command?

Test cases (valid cases)

    zot alpha
    zot alpha beta gamma delta
    zot -f2-3 alpha beta gamma delta
    zot -f 2-3 alpha beta gamma delta
    zot -f 2-2 alpha beta gamma delta
    zot -f 2-  alpha beta
    zot -f  -2 alpha beta gamma

Test cases (invalid cases)

    zot
    zot -fx alpha
    zot -f3 alpha
    zot -f1 -f2 alpha beta
    zot -f0-1 alpha
    zot -f1-2 alpha
    zot -f2-1 alpha beta gamma
    zot -q alpha
    zot badfile

Automation

    #! /bin/bash
    (
        zot -fx alpha
        zot -f3 alpha
        zot -f1 -f2 alpha beta
        …
    ) >&out
    diff out known-good-output