Learning goals

You will learn the following concepts/skills:

1. unit testing – systematically test your code
   1. framework – create a simple framework for unit-testing future C code
   2. organization – learn how to organize a test suite
   3. test coverage – measure how thoroughly your tests exercise your code
2. C preprocessor − practice using #define symbols and macros with #ifdef.

Overview

Real-world software projects typically comprise tens, hundreds, or many thousands of files. Even in your ECE courses, your programming assignments will be getting bigger and more complex. Testing code by hand (i.e., by playing with it) is useless since there are so many components, and ways things can go wrong. Hand-testing effectively would be complex—and prohibitively slow.

Until now, we have used a very low-tech method of structuring our tests with the expected.txt and the diff command. That method is based on the more general principle of unit testing.

In unit testing, programmers write collections of small functions that test some aspect of a program. For example, in mintf(…) in HW04, you might have a unit test for small positive integers in base 10, another unit for small negative numbers, one for extreme bases, extreme positive values, extreme negative values, and so on. Each test is its own function.

As the number and complexity of tests grows, it becomes necessary to have a foundation of code for running and organizing them. This is called a unit test framework. There are unit test frameworks for every major programming language. For C, one example is CUnit. These typically have special-purpose macros for checking assumptions, similarly to the assert(…) macro, which we have already covered this semester, but with a unit test framework, we may allow the code to continue, even if an assumption is not met, since the goal is simply to check. In addition to the checking macros, real-world testing frameworks have mechanisms for setting up the needed environment that the program-in-test depends on (e.g., files, network connections, etc.), and sophisticated interfaces for visualizing the results and scheduling test runs.

In this homework, you will create a very simple unit test framework… let's call it a unit test library. You may use this to test future assignments. We will illustrate by example.

Using miniunit.h to test — minimal example

Suppose you have the following (very trivial) module for doing arithmetic calculations involving the number 3.

three.c
 1 #include "three.h"
 2
 3 int times_3(int n) {
 4     return n * 3;
 5 }
 6
 7 int divided_by_3(int n) {
 8     return n % 3;  // BUG
 9 }
10
11 int plus_3(int n) {
12     return n * 3;  // BUG
13 }
14
15 int minus_3(int n) {
16     return n - 3;
17 }

three.h
 1 int times_3(int n);
 2 int divided_by_3(int n);
 3 int plus_3(int n);
 4 int minus_3(int n);

Here is the general format for every unit test function (e.g., in test_three.c). You will typically have several of these in your test_▒▒▒.c.

int test_▒▒▒() {
    mu_start();  // set up the test
    // …
    mu_check(condition);
    // …
    mu_check(condition);
    // …
    mu_end();    // finish the test
}

Then, to call all of your unit test functions, you will have a runner function (e.g., main(…)) like this:

int main(int argc, char* argv[]) {
    mu_run(test_▒▒▒);
    mu_run(test_▒▒▒);
    mu_run(test_▒▒▒);
    mu_run(test_▒▒▒);
    return EXIT_SUCCESS;
}

Using the above format, we create the following unit test suite (group of tests) for our three-arithmetic module.

// test_three.c

#include <stdio.h>
#include <stdlib.h>
#include "miniunit.h"
#include "three.h"

int test_plus_3() {
    mu_start();
    mu_check(plus_3(1) == 4);  // will FAIL
    mu_end();
}

int test_minus_3() {
    mu_start();
    mu_check(minus_3(1) == -2);
    mu_end();
}

int test_times_3() {
    mu_start();
    mu_check(times_3(1) == 3);
    mu_end();
}

int test_divided_by_3() {
    mu_start();
    mu_check(divided_by_3(1) == 0);  // will FAIL
    mu_end();
}

int main(int argc, char* argv[]) {
    mu_run(test_plus_3);        // will FAIL
    mu_run(test_minus_3);
    mu_run(test_times_3);
    mu_run(test_divided_by_3);  // will FAIL
    return EXIT_SUCCESS;
}

Your unit test library, using #define macros will convert that seemingly simple test code into a colored summary, including the name of the test function and the line where it failed. Here is the expected out put for the test above:

$ gcc -o test_three three.c test_three.c

$ ./test_three
Test failed: test_plus_3 at line 8
Test passed: test_minus_3
Test passed: test_times_3
Test failed: test_divided_by_3 at line 26

$

1. Set up your homework directory.

You will be creating a file called miniunit.h from scratch (i.e., starting with a blank file).

The only starter code is a (deliberately) buggy program called . You will get two files: count_words.c and count_words. You will use your miniunit.h to write a test file (test_count_words.c) for testing . This will let you verify that your miniunit.h is working, and practice using it.

Get the starter code and cd into your HW06 directory.

2. Create your miniunit.h.

Create a file called miniunit.h from scratch. Then, create the five #define macros as specified in the Requirements table.

3. Use miniunit.h to improve test_count_words.c

This part will teach you how to use miniunit.h in practice for future assignments. We are giving you the code for free, so it shouldn't take more than about 15 minutes to type and test.

1. At the top of test_count_words.c, include miniunit.h.
2. Add three test functions above main(…): int _test_empty(), int _test_simple(), and int _test_hard().
3. Inside each test function, add mu_start() at the beginning, and mu_end(…) just before the end.
4. mu_check(…) should be invoked at least once in between mu_start() and mu_end(…).
5. At the top of main(…), add mu_run(_test_empty), mu_run(_test_simple), and mu_run(_test_hard).
6. The result may look something like this:

   #include <stdio.h>
   #include <stdlib.h>
   #include "count_words"
   #include "miniunit.h"
   #include "log_macros.h"
   
   static int _test_empty() {
     mu_start();
     //-------------------------------
     mu_check(count_words("") == 0);
     //-------------------------------
     mu_end();
   }
   
   static int _test_simple() {
     mu_start();
     //-------------------------------
     mu_check(count_words("apple") == 1);
     mu_check(count_words("boring boxes") == 2);
     mu_check(count_words("apple banana") == 2);
     mu_check(count_words("apple banana cherry") == 3);
     //-------------------------------
     mu_end();
   }
   
   static int _test_hard() {
     mu_start();
     //-------------------------------
     mu_check(count_words("famigerate fiddle-faddle") == 2);
     mu_check(count_words("Mary's mongoose") == 2);
     mu_check(count_words("plumbers' platitudes") == 2);
     //-------------------------------
     mu_end();
   }
   
   int main(int argc, char* argv[]) {
     mu_run(_test_empty);
     mu_run(_test_simple);
     mu_run(_test_hard);
   
     log_int(count_words("My apples are sweet."));
     log_int(count_words("My friend's apples are sweeter."));
     log_int(count_words("A pear is sweeter than an apple.."));
   
     return EXIT_SUCCESS;
   }
   

   Note: Due to the intentially planted bugs in count_words.c, _test_simple(…) will fail on "apple banana" (due to 'a'), and _test_hard(…) will fail on "familgerate fiddle-faddle" (due to '-').
Reminder: count_words(…) has intentionally planted bugs. It is okay that some of the results are obviously incorrect and the tests fail. The purpose of tests is to detect flaws. If the flaws are detected, then the test worked correctly.
We are giving you some code for free. You are welcome to make changes, or hand-copy as is, as long as you understand what it is doing. If not, please ask.
7. You must submit your test_count_words.c, even if it is identical to the code above.

Printing in color

To print in color on the console—i.e., in an SSH window or terminal window—we use ANSI codes. These are instructions—mixed in with the text you want to display—that tell the terminal how to display the text.

Let's look at one ANSI code. To tell the terminal to start printing in green, you send "\x1b[31m".

By “send,” we really just mean print. So you could start printing in green using this function call:

printf("\x1b[31m")

After that, everything you print will be in green.

Let's break down that ANSI code to see what the characters mean.

\x1b is the escape character. It has ASCII value 27 (hexadecimal: 0x1b). In C, you can specify any character by its ASCII value as "\x▒▒", where ▒▒ is the ASCII value in expressed in hexadecimal. Sending the escape character ("\x1b") is equivalent to pressing the escape key on a keyboard. Every ANSI code must begin with the escape character.

[32m is a command that means, “set the foreground text color to green.” Using other numbers, you can set the foreground color to other colors, or set the background color and other aspects of the text formatting.

For example, to start printing all text in red, you would use this:

printf("\x1b[31m")

To reset (i.e., go back to the default colors), do the same thing, but with 0, like this:

printf("\x1b[0m")

You can put these all together in one call to printf(…). For example, …

printf("Roses are \x1b[31mRED\x1b[0m.\n");     // "\x1b[31m"→red;  "\x1b[0m"→reset.
printf("Violets are \x1b[34mBLUE\x1b[0m.\n");  // "\x1b[34m"→blue

… would print …

Roses are RED.
Violets are BLUE.

That code can be more readable if you put each code in a parameter of the printf(…) like this:

char* ansi_red   = "\x1b[31m";
char* ansi_blue  = "\x1b[34m";
char* ansi_reset = "\x1b[0m";
printf("Roses are %sRED%s.\n", ansi_red, ansi_reset);
printf("Violets are %sBLUE%s.\n", ansi_blue, ansi_reset);

Copying from this page

You may hand-copy any code snippets you find in this homework description into your HW06 submission.

Copying from this page is not necessary. We are allowing this only because there is some unfamiliar syntax, and we think this flexibility may help you. You are responsible for checking and adapting anything you copy, and ensuring the proper functioning of the code you submit.

⚠ Do not use copy-paste. You learn more from hand-copying unfamiliar syntax. Expect problems if you ignore this.

How much work is this?

#define MINIUNIT_BONUS_SELECTIVE_TESTING

#define MINIUNIT_BONUS_INLINE_FUNCTIONS

Requirements

Submit

To submit HW06 from within your hw06 directory, type 264submit HW06 miniunit.h test_count_words.c

Q&A

1. Should I have a semicolon at the end of the RHS of a #define macro

   No. The person using the macro will normally include the semicolon.

2. How can I continue the RHS of a #define macro definition onto the next line

   Put a backslash ("\") at the end of the line. (But keep reading for more on how to do this right.)

   #define profess_love_for_food(food) printf("I love %s", \
           food)

3. Can I have a macro with multiple C statements

   Yes. In theory, you could just have the two statements separated by a semicolon like this:

   // BAD
   #define profess_love_for_two_foods(food1, food2) \
         printf("I love %s", food1);                \
         printf("I love %s", food2)

   However, that would lead to surprising results if someone who doesn't follow the code quality standards calls that macro in an if statement like this:

   if(1 == 0)
       profess_love_for_two_foods("soap", "poison");

   Only the first statment would be connected to the if statement.

   if(1 == 0)
       printf("I love %s", "soap");
   
   printf("I love %s", "poison");

   It is tempting to just put curly braces around the two statements, but that also causes problems.

   // BAD
   #define profess_love_for_two_foods(food1, food2) { \
       printf("I love %s", food1); \
       printf("I love %s", food2); \
   }

   The problem comes back to uncivilized oafs who write if statements without curly braces, like this:

   if(age >= 30)
       profess_love_for_two_foods("chocolate", "pizza");
   else
       profess_love_for_two_foods("spinach", "broccoli");

   The above example would result in this:

   if(age >= 30) {
       printf("I love %s", "chocolate");
       printf("I love %s", "pizza");
   };  ← PROBLEM
   else {
       printf("I love %s", "spinach");
       printf("I love %s", "broccoli");
   };

   The standard solution is to wrap the statements in a do { … } while(false) block. Because the do…while requires a semicolon, this actually works out like we want.
   Yes, it is ugly. Hacks like this are not something the instructor would normally condone, but it is standard practice because there are very few truly versatile options for this.

   // USE THIS WAY
   #define profess_love_for_two_foods(food1, food2) do { \
       printf("I love %s", food1);                       \
       printf("I love %s", food2);                       \
   } while(false)

4. Why does miniunit.h need mu_start() and mu_end(…)

   These give a relatively clean and consistent form to your unit tests, so you can focus on the code that matters for each particular test.
   Without mu_start() and mu_end(…), one might resort to a naïve approach, like this:

   YUK!
   int test_count_words_▒▒▒▒▒() {
       int __mu_first_failure_line_number_or_0 = __MU_SUCCESS;
       mu_check(▒▒▒▒▒▒▒▒▒▒);
       mu_check(▒▒▒▒▒▒▒▒▒▒);
       mu_check(▒▒▒▒▒▒▒▒▒▒);
       return __mu_first_failure_line_number_or_0;
   }

   The programmer should not have to know the specific internal names used by miniunit.h! If they get it wrong, then it won't work properly with mu_check(…).
   Of course, they could avoid the problem of consistency if they do the whole thing without any macros—i.e., without miniunit.h—but then the above skeleton would be even messier!

   YUK! YUK! YUK! YUK! YUK!
   int test_count_words_▒▒▒▒▒() {
       const int __MU_SUCCESS = 0;
       int __mu_first_failure_line_number_or_0 = __MU_SUCCESS;
       if(!(▒▒▒▒▒▒▒▒▒▒) && __mu_first_failure_line_number_or_0 == __MU_SUCCESS) {
           __mu_first_failure_line_number_or_0 = __LINE__;
       }
       if(!(▒▒▒▒▒▒▒▒▒▒) && __mu_first_failure_line_number_or_0 == __MU_SUCCESS) {
           __mu_first_failure_line_number_or_0 = __LINE__;
       }
       if(!(▒▒▒▒▒▒▒▒▒▒) && __mu_first_failure_line_number_or_0 == __MU_SUCCESS) {
           __mu_first_failure_line_number_or_0 = __LINE__;
       }
       return __mu_first_failure_line_number_or_0;
   }

   With mu_start() and mu_end(…)—and adding a divider comment (optional)—you get this:

   GOOD
   int test_count_words_▒▒▒▒▒() {
       mu_start();
       //────────────────────────────────────────
       mu_check(▒▒▒▒▒▒▒▒▒▒);
       mu_check(▒▒▒▒▒▒▒▒▒▒);
       mu_check(▒▒▒▒▒▒▒▒▒▒);
       //────────────────────────────────────────
       mu_end();
   }

   mu_start() and mu_end(…) also make your unit testing library more extensible. If you wanted to change how your tests are organized and/or reported, you could do so without changing all of your test code that uses it.

5. What does #x do in a #define macro

   It expands to the text of the expression, instead of its value. This is easiest to see if you test using the /usr/bin/cpp command.
   Here is an example, which uses the log_int(…) snippet given in the Requirements table.

   // demonstrate_hash.c
   #include <stdio.h>
   #include <stdlib.h>
   
   #define log_int(n) printf("%s == %d\n", (#n), (n))
   
   int main(int argc, char* argv[]) {
       log_int(3 + 3);
       return EXIT_SUCCESS;
   }
   

   If we process that with the preprocessor directly (instead of via gcc), we can see what it becomes.
   you@eceprog ~/HW06 $ /usr/bin/cpp demonstrate_hash.c -DDEBUG | indent -kr

   … int main(int argc, char* argv[]) { printf("%s == %d\n", ("3 + 3"), (3 + 3)); return 0; }
   Notice that the third argument to fprintf(…) is a string literal, "3 + 3"—the text of the argument that was passed to log_int(…). That is different from the fourth argument, which is the value of that parameter, 3 + 3 (= 6).

6. How do I make mu_run(…) print the success/failure messages in color?

   Define a helper macro for printing in color outside the #ifdef/#endif. You can use the one in the Q&A of miniunit_1 or make your own. You will need to define the ANSI codes (#define ANSI_RED …, etc.) outside the #ifdef/#endif, as well.

7. GCC: “error: ‘true’ undeclared” ⋯???
   GCC: “error: ‘false’ undeclared” ⋯??

   The true and false constants are defined in a standard header called stdbool.h; you need to include it (i.e., #include <stdbool.h>) in any file where you use them. This was communicated in the Requirements table, as well as the Code Quality Standards, which have more about bool, true, and false.

8. GCC: “__mu_first_failure_line_number_or_0 is not defined” ⋯ Why

   If you get that within main(…), make sure you are not trying to use the variable declared in mu_start() in your mu_run(…). Your mu_start() is expanded in the context of a _test_▒▒▒(…) function, while mu_run(…) is expanded in the context of main(…).

9. CPP: "Unexpected EOF (end-of-file)" (or similar). Why?

   You probably have unmatched braces somewhere in your miniunit.h. First, fix your indents. It will make the errors more obvious. Then, be sure to use /usr/bin/cpp test_count_words.c | indent -kr or /usr/bin/cpp test_count_words.c -DDEBUG | indent -kr to make the output readable.

10. How do I start? Can I use TDD?

    Try this progression:
    1. Make it work for an empty test function (i.e., no calls to mu_check(…)).
    2. Make it work for tests that always pass. In other words, assume the condition is true.
    3. Make it work for test functions with one test that fails.
    4. Make it work for test functions with any number of tests that fail.

Updates