tests: improve code coverage support

Fix the exclusion path for lcov; it should exclude the directory
with source code, not object files.

Use the COV environment variable to
* control whether we build for coverage or not
* select the output directory

Add a separate target for generating the report, so we can get a
report for all of the tests together or just a single test.

Add documentation.

Signed-off-by: Paul Fagerburg <pfagerburg@google.com>
Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a
Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>

2 files changed, 58 insertions, 0 deletions

diff --git a/Documentation/technotes/2021-05-code-coverage.md b/Documentation/technotes/2021-05-code-coverage.md
new file mode 100644
index 0000000000..ec7f4cb5f9
--- /dev/null
+++ b/Documentation/technotes/2021-05-code-coverage.md
@@ -0,0 +1,56 @@
+# Unit Test Code Coverage
+
+Code coverage for the coreboot unit tests allows us to see what lines of
+code in the coreboot library are covered by unit tests, and allows a test
+author to see where they need to add test cases for additional coverage.
+
+Enable code coverage in your unit test build by setting the environment
+variable `COV` to 1; either `export COV=1` in your shell, or add it to your
+`make` command, e.g. `COV=1 make unit-tests`.
+
+The build output directory is either `build/tests` or `build/coverage`,
+depending on whether `COV=1` is set in the environment.
+
+All of the unit test targets are available with and without `COV=1`
+* `clean-unit-tests`
+* `build-unit-tests`
+* `run-unit-tests`
+* `unit-tests` (which is just `build-unit-tests` followed by `run-unit-tests`)
+
+There are two new `make` targets:
+* `coverage-report` generates a code coverage report from all of the
+GCOV data (`*.gcda` and `*.gcno` files) in the build directory. To view the
+coverage report, open `build/coverage/coverage_reports/index.html` in your web
+browser.
+* `clean-coverage-report` deletes just the coverage report.
+
+The `coverage-report` and `clean-coverage-report` targets automatically set
+`COV=1` if it is not already set in the environment.
+
+
+## Examples
+
+`COV=1 make unit-tests coverage-report` builds all of the unit tests with code
+coverage, runs the unit tests, and generates the code coverage report.
+
+`COV=1 make build-unit-tests` builds all of the unit tests with code coverage.
+
+`COV=1 make run-unit-tests` runs the unit tests, building them with code
+coverage if they are out-of-date.
+
+`COV=1 make coverage-report` creates the code coverage report. This
+target does not explicitly depend on the tests being built and run; it gathers
+the code coverage data from the output directory, which it assumes already
+exists.
+
+`COV=1 make tests/lib/uuid-test coverage-report` builds the uuid test
+with code coverage, runs it, and generates a code coverage report just for
+that test.
+
+As a demonstration that building with and without coverage uses different
+output directories:
+1. `make build-unit-tests` builds unit tests without code coverage into
+`build/tests`.
+2. `COV=1 make clean-unit-tests` cleans `build/coverage`
+3. `make build-unit-tests` doesn't need to build anything in `build/tests`,
+because those files weren't affected by the previous `clean-unit-tests`.
diff --git a/Documentation/tutorial/part3.md b/Documentation/tutorial/part3.md
index 7ccee87754..6d147ff9a9 100644
--- a/Documentation/tutorial/part3.md
+++ b/Documentation/tutorial/part3.md
@@ -3,6 +3,8 @@
 ## Introduction
 General thoughts about unit testing coreboot can be found in
 [Unit testing coreboot](../technotes/2020-03-unit-testing-coreboot.md).
+Additionally, [code coverage](../technotes/2021-05-code-coverage.md) support
+is available for unit tests.
 
 This document aims to guide developers through the process of adding and writing
 unit tests for coreboot modules.