Learning goals

You will learn the following concepts/skills:

1. Build systems – Create make files for compilation and testing.

Overview of the Miniunit series (HW06, HW07, HW08)

This assignment is part 1 of a 3-part series collectively referred to as miniunit.

Real-world software projects typically comprise tens, hundreds, or many thousands of files. Even in your ECE courses, your programming assignments will be getting bigger and more complex. Testing code by hand (i.e., by playing with it) is useless since there are so many components, and ways things can go wrong. Hand-testing effectively would be complex—and prohibitively slow.

Most serious software projects use two tools: a unit testing framework (e.g., CUnit) to organize and coordinate tests, and a build system (e.g., make) to manage the process of compiling, testing, and deploying the project. In this homework, you will get a light introduction to both. In addition, you will learn to use console logging effectively (as a complement to gdb), without littering your code with printf(…) statements.

In the Miniunit Series, you will create the following:

• clog.h – your own library for smarter printf-style debugging and logging. From now on, instead of debugging with printf(…) directly, you will use function-like macros to print values in various formats and colors to make the output easier to view. You will use preprocessor directives to ensure that your debugging code does not interfere with your tests, or show up inadvertently when others are testing your code.
• Makefile – input file for the make build system. You will be able to build, test, submit, and/or pre-test your code with one command from bash or directly from Vim (and other editors).
• miniunit.h – your own simple unit test library. You can use this to test future assignments in this class, or for any other project you do in C (or C++) beyond ECE 264. This consists of four #define macros that you can use in your test code. The most important macro you will create is mu_check(…), which is somewhat similar to assert(…).

Starter code

You will create your Makefile from scratch (i.e., starting with a blank file).

About make and the Makefile

The make tool is used to build software projects and perform other operations (e.g., running tests, cleaning up intermediate files, etc.). It is widely used by real-world projects in industry and open source projects.

Instructions for how to build a project and perform other actions are given in a file called Makefile. That file contains instructions for how to build the executable, as well as rules for how to build or generate any intermediate files (e.g., object files).

Structure of a Makefile. A Makefile is a text file named “Makefile” (exactly). It may contain variables, rules, and phony targets.

# VARIABLES
  LHS=RHS
  LHS=RHS
  …

# RULES
target: prerequisites…
    action
    action
    …

# .PHONY TARGET - declare targets that do not correspond directly to an output file
.PHONY: target_name target_name …

Variables. The variables section (at the top) contains filenames, and other information that you will use in the commands that build and manage your project. In this homework, we have specified exactly what variables you need in the Requirements table.

Rules. A rule consists of a target, ≥0 prerequisites, and ≥0 actions. Normally, most rules in make files describe how to compile (or generate) a particular file. We will also use so-called phony targets, which are essentially just scripts for performing actions related to your project (e.g., submit, run tests, etc.).

Targets. The target of a rule is usually a file that will be created by the actions in that rule. For example, the test_count_words rule will create the test_count_words executable file. For phony targets, the target is the name of the command.

Prerequisites. The prerequisites of a rule are the targets that it The prerequisites… are a list of targets upon which that rule depends. For example, test_count_words depends on count_words.c, count_words.h. If you were to use your new console log macros in your count_words.c, then clog.h would also be a prerequisite of test_count_words.

Actions. The actions in each rule consist of commands that could be entered in your shell (bash). Your bash aliases will not work here. That means simply typing gcc would not include the flags we require for the class. Therefore, you will create some variables to make sure you get those flags. (These are given in the Requirements table below.)

⚠Actions must be indented with a tab character, not four spaces. If you are using Vim with the .vimrc we provided, this is taken care of for you.

Phony targets. At the end of your Makefile you should have a rule called .PHONY. The prerequisites of .PHONY should be every target that is not a real file target.

Running make. From bash, type make target to build that target. For example, typing make test_count_words would compile the count_words project to create or update the executable file test_count_words. If the rule has any prerequisites—e.g., files must be up-to-date or commands to be run before creating test_count_words—those are built first.

Default target. The first rule in a Makefile is the default. Simply running make (from bash) will build the first target. For example, if your first target in your Makefile is test_count_words, then running make (from bash) would be equivalent to running make test_count_words .

1. Set up your homework directory.

There is no starter code, though you will use your code from HW07.

Create a directory for HW08.

Copy your HW07 code into your HW08 directory.

The most important file you will create in HW08 is Makefile.

2. Create your Makefile

You will create your Makefile from scratch. However, you will be using it with your code from HW06 and HW07.

Your Makefile will have instructions for how to not only compile a program, but also run your tests, submit, run the pretester, and even run a code coverage tester to search for areas of your code that your tests may not be covering.

A Makefile must be specific to a particular project. For HW08, you will create a Makefile for the count_words project that is included in the starter. For the rest of this semester, you will simply change a few variables at the top of your Makefile to make it work for each future assignment.

The specifics on what your Makefile must contain are in the requirements table.

Reminder: You may hand-copy snippets from this homework description (HW08), but do not copy-paste. You won't learn as well, and it won't work anyway.)

Instructions

1. Create a very simple Makefile to just compile test_count_words
   1. Create a blank file called Makefile.
      you@ecegrid-thin1 ~/264/hw08 $ vim Makefile
   2. Add one rule to compile test_count_words as follows.
      test_count_words: count_words.c test_count_words.c count_words.h miniunit.h clog.h gcc -o test_count_words count_words.c test_count_words.c -g -std=c11 -Wall -Wshadow -Wvla -Werror -pedantic -Wno-unused-function
2. Test the Makefile from bash.
   1. From bash:
      you@ecegrid-thin1 ~/hw08 $ make
   2. From Vim:
      :make
   Since those commands do not specify a target, the first rule in your Makefile is used.
3. Modify your Makefile to use the CC variable.
   1. Set the CC variable to gcc at the top of Makefile like this:
      CC=gcc
   2. Replace gcc with $(CC) in your test_count_words rule.
   3. Test that everything works so far.
4. Modify your Makefile to use the CFLAGS variable.
   1. Set the CFLAGS variable to -g -std=c11 -Wall -Wshadow -Wvla -Werror -pedantic -Wno-unused-function at the top of Makefile like this:
      CFLAGS=-g -std=c11 -Wall -Wshadow -Wvla -Werror -pedantic -Wno-unused-function
      ⚠ CFLAGS should not include -DDEBUG or -DNDEBUG.
   2. Replace -g -std=c11 -Wall -Wshadow -Wvla -Werror -pedantic -Wno-unused-function with $(CFLAGS) in your test_count_words rule.
   3. Test that everything works so far.
5. Modify your Makefile to set and use the other variables.
   1. Set the BASE_NAME variable to count_words at the top of Makefile like this:
      BASE_NAME=count_words
   2. Set the SRC_C variable to count_words.c using $(BASE_NAME) to minimize duplication, like this:
      SRC_C=$(BASE_NAME).c
   3. Set the TEST_C variable to test_count_words.c using $(SRC_C) to minimize duplication, like this:
      TEST_C=test_$(SRC_C)
   4. Add all of the rest of the variables specified in the Requirements table. Wherever possible, use existing variables to minimize duplication.
   5. Use the variables you defined in your test_count_words rule. At this point, there should be no mention of …count_words… anywhere in the rule.
   6. Test that everything works so far.
6. Add a phony rule to submit your code
   1. Add a rule called submit that submits the assignment. Use the variables you defined. You can add more if needed. Your submit rule does not require need any prerequisites. However, if you want make submit to check that all required files are present, then you may optionally add $(SUBMIT_FILES) as a prerequisite. It is your choice.
   2. Add the following line to the bottom of your Makefile: .PHONY: submit
   3. Type make submit in bash or :make submit in Vim to test your rule for the submit phony target. . You may submit to it to test your Makefile, but will not be scored and will not affect your grade.
7. Add a phony rule to run your tests.
   1. Add a rule called test. Hand-copy the following code. We will be adding a bit more to this rule later in HW08.
      test: $(EXECUTABLE) $(TEST_EXPECTED) @# If actual output matches expected output then count it as a success @if diff -a -B <("./$(EXECUTABLE)") $(TEST EXPECTED) &> /dev/null ; then \ echo "Test passed: output of $(EXECUTABLE) matches $(TEST_EXPECTED)" ; \ else \ echo "Test failed: output of $(EXECUTABLE) does NOT match $(TEST_EXPECTED)" ; \ fi
   2. Add test to your .PHONY rule.
   3. Type make test in bash or :make test in Vim to test your rule for the test phony target.
8. Add a phony rule to run the pretester.
   1. Add a rule called pretest that runs the pretester. It should call your submit rule first (using a prerequisite) and then call 264test. Use the variables you created (including ASG_NICKNAME) in the actions for this rule. ⚠ Do not enter “HW55”, “count_words.c”, “test_count_words.c”, or “expected.txt” directly.
   2. Add pretest to your .PHONY rule.
   3. Type make pretest in bash or :make pretest in Vim to test your rule for the pretest phony target. It is a dummy pretester that gives everyone 100%.
9. Add a phony rule called clean that deletes the executable files (test_count_words, test_count_words_dbg, test_count_words_gcov). The command to delete a file(s) is rm files…. When working at the terminal, we have that command set to prompt for confirmation. To ensure that make does not prompt for confirmation, you add the -f flag. Thus, the whole command will be rm -f files….
   make clean should delete the executables and any data files created by the code coverage check (if any). To delete file(s), use rm -f ▒▒▒. The -f tells the rm command not to prompt for confirmation or print an error if no such file exists.
   ⚠ Be careful not to delete your .c or .h files, or your Makefile.
10. Add the remaining rules, as specified in the Requirements table. Some additional hints are below. Be sure all rules except for $(EXECUTABLE) are listed in your .PHONY: line.

3. Test coverage checking

Code coverage testing—including statement coverage—can help you find bugs in your code by discovering aspects of the functionality that your tests are not exercising. In this section, you will learn how to use test coverage checking, and integrate that functionality into your Makefile.

We will be using one kind of coverage, called line coverage. It runs your tests and monitors which statements in your implementation code were executed. If any statements were not executed, it typically indicates that either your tests are not exercising all of the functionality. It is also possible that you have some dead code. Either way, it should be rectified.

We will be using a tool called GCOV, which works in conjunction with GCC. To check test coverage using GCOV, use this:

# Compile a special version of executable with coverage checking
gcc -o executable source files... -ftest-coverage -fprofile-arcs -DNDEBUG

# Run the executable to record coverage statistics
./executable

# Print a coverage summary
gcov -f one source file

Detailed information, such as which lines were executed, can be found by opening the count_words.c.gcov in your editor.

Instructions

1. Try running GCOV from bash. Later, you will be adding this to your Makefile but it is important to first make sure it works for you, and make sure you understand what it does.
   You could do this with the included count_words program, but to show this in context of something more familiar, we will demonstrate using and implementation of HW05.

   you@ecegrid-thin1 ~/HW08
   $ gcc -o test_mintf_gcov mintf.c test_mintf.c -fprofile-arcs -ftest-coverage -DNDEBUG
   you@ecegrid-thin1 ~/HW08
   $ ./test_mintf_gcov    
   ▒▒▒▒▒▒▒▒▒▒
   ▒▒▒▒▒▒▒▒▒▒
   you@ecegrid-thin1 ~/HW08
   $ gcov -f mintf.c    
   Function 'mintf'
   Lines executed:61.29% of 31
   
   Function 'print_integer'
   Lines executed:75.00% of 8
   
   Function '_print_integer_digit'
   Lines executed:81.82% of 11
   
   File 'mintf.c'
   Lines executed:68.00% of 50
   Creating 'mintf.c.gcov'

   The example above is what you would see if you have not tested adequately. In this case, only 68% of the lines in count_words.c were executed at all. None of the three functions was tested thoroughly. Hopefully, the stats for your code will report 100% coverage.
   This metric excludes blank lines, comments, and lines containing only braces, so 100% line coverage should be attainable.
2. Add a rule called coverage to your Makefile to check code coverage. Use the variables you defined (e.g., CC, CFLAGS, etc.) wherever possible. Have the GCC command Hint: It should have two .c files as prerequisites, and three actions. Don't forget to add coverage to the list of phony targets (i.e., .PHONY: submit test ▒▒▒▒▒▒▒▒▒▒▒▒▒).
3. Type make coverage in bash or :make coverage in Vim to test your rule for the coverage phony target. Verify that it created five new files: count_words.c.gcov, count_words.gcda, count_words.gcno, test_count_words.gcda, test_count_words.gcno.
4. Modify the clean rule in your Makefile to remove the data files created by GCOV. You should use wildcards (*.c.gcov *.gcda *.gcno).
5. Type make clean in bash or :make clean in Vim to test your rule for the clean phony target. Verify that the five files created by GCOV (count_words.c.gcov, count_words.gcda, count_words.gcno, test_count_words.gcda, test_count_words.gcno) were deleted.

Requirements

Submit

To submit HW08 from within your hw08 directory, type 264submit HW08 Makefile miniunit.h clog.h test_count_words.c

Pre-tester ●

The pre-tester for HW08 has not yet been released. As soon as it is ready, this note will be changed.

Q&A

1. Shouldn't a Makefile refer to .o files

   We are using makefiles in a simplified manner that does not require awareness of .o files.

2. What is HW55? Is that a typo

   HW55 is a dummy assignment that you will use to test the rules in your Makefile that deal with assignment submission and pretesting. It will not be scored and will not affect your grade.
   HW08 is the “real” assignment you are doing here.

3. What is the purpose of the debug rule in the Makefile

   Compiling with make debug will enable your log_▒▒▒(…) macros by passing the -DDEBUG flag to GCC (i.e., to define the DEBUG symbol). With just make (or make test_count_words) DEBUG will not be defined, so your log_▒▒▒(…) macros will be silent. How you call make determines whether they are on or off.
   After HW08, you are welcome to modify this (e.g., if you want your log_▒▒▒(…) macros to be enabled by default.)

4. Make: “No rule to make target `miniunit.h', needed by `debug'”

   You do not have a file called miniunit.h, but it is listed as a prerequisite of that file.

5. After editing miniunit.h or clog.h, nothing changes when I run make test

   Make sure your make test rule depends on debug.

6. Should make coverage depend on $(EXECUTABLE)?

   No.

7. Should make debug depend on $(EXECUTABLE)?

   No.

8. Do I call make $(EXECUTABLE) or make test_count_words?

   Call make test_count_words (from bash).

9. I used a helper (e.g., __mu_log_color(…)) in clog.h. Can I reuse that in miniunit.h but have the messages go to stderr?

   Not easily. Make a new helper (e.g., __mu_log_color_stderr(…)) in miniunit.h. It may be similar to your original __mu_log_color_stderr(…) but using stderr instead of stdout, and STDERR_FILENO instead of STDOUT_FILENO. It can still refer to the ANSI color constants which will be in clog.h, to avoid duplicating those.

10. Can my Makefile refer to the ANSI color constants in clog.h?

    No. They must be duplicated in Makefile. The make tool cannot read C code directly.

Updates