Unit testing in Fortran
last modified November 14, 2026
A unit test is a small program that calls one piece of code with known arguments and compares the result with the value it should have. The tests are kept beside the code, run after every change, and their exit status is the answer: zero when everything passed, nonzero when something did not. Fortran has no testing framework in the standard library, but a harness of three comparison routines and two counters is enough for most projects.
This article writes a small statistics module, a harness with
check_true, check_int and check_close, a
suite that passes, a test that fails and shows how the report identifies it, a
suite about floating-point tolerances, a suite about edge cases, and a shell
script that runs several test programs and returns a single status to the build
system.
A module worth testing
The subject of the tests is a module with three functions: the arithmetic mean, the median of an array and the sample standard deviation. The mean is the sum divided by the count, the median sorts a copy and takes the middle value or the average of the two middle values, and the standard deviation subtracts the mean from every value, squares the differences and divides by one less than the count. The last part matters: an array of one value has no sample deviation at all.
module stats
use iso_fortran_env, only: real64
implicit none
private
public :: mean, median, stddev
contains
pure real(real64) function mean(x) result(m)
real(real64), intent(in) :: x(:)
m = sum(x) / real(size(x), real64)
end function mean
pure real(real64) function median(x) result(m)
real(real64), intent(in) :: x(:)
real(real64) :: sorted(size(x)), swap
integer :: i, j, n
n = size(x)
sorted = x
do i = 2, n
swap = sorted(i)
j = i - 1
do while (j >= 1)
if (sorted(j) <= swap) exit
sorted(j + 1) = sorted(j)
j = j - 1
end do
sorted(j + 1) = swap
end do
if (mod(n, 2) == 1) then
m = sorted((n + 1) / 2)
else
m = 0.5_real64 * (sorted(n / 2) + sorted(n / 2 + 1))
end if
end function median
pure real(real64) function stddev(x) result(s)
real(real64), intent(in) :: x(:)
real(real64) :: m
integer :: i
m = mean(x)
s = 0.0_real64
do i = 1, size(x)
s = s + (x(i) - m)**2
end do
s = sqrt(s / real(size(x) - 1, real64))
end function stddev
end module stats
The module is pure throughout, so the functions can be called from
array expressions and cannot change their arguments. The median copies the
input into a local array before sorting, which is what makes it a function of
its argument rather than a rearranger of the caller's data.
program stats_demo
use iso_fortran_env, only: real64
use stats
implicit none
real(real64), parameter :: data(5) = [4.0_real64, 1.0_real64, &
3.0_real64, 2.0_real64, 5.0_real64]
print '(a, f0.3)', 'the mean: ', mean(data)
print '(a, f0.3)', 'the median: ', median(data)
print '(a, f0.4)', 'the standard deviation: ', stddev(data)
print '(a, f0.3)', 'the median of the first four: ', median(data(1:4))
end program stats_demo
The demo shows the values that the tests will assert: a mean and a median of 3 for the numbers one to five, a standard deviation of 1.5811, and a median of 2.5 for the first four, because an even count averages the two middle values. Running the demo by eye is what a test replaces with a comparison that a machine can check.
$ gfortran -std=f2018 -Wall -Wextra -fcheck=all stats.f90 stats_demo.f90 -o stats_demo $ ./stats_demo the mean: 3.000 the median: 3.000 the standard deviation: 1.5811 the median of the first four: 2.500
The harness
The harness keeps two counters in module variables and offers one routine per
kind of comparison. check_true records the name of a test and
whether a logical condition held; check_int compares two integers
and prints both when they differ; check_close compares two reals
within a tolerance. summary prints the counts and turns a failure
into a nonzero exit status with error stop 1.
module testing
use iso_fortran_env, only: real64
implicit none
private
public :: check_true, check_int, check_close, summary
integer :: passed = 0, failed = 0
contains
subroutine check_true(name, condition)
character(len=*), intent(in) :: name
logical, intent(in) :: condition
if (condition) then
passed = passed + 1
print '(a, a)', 'ok ', name
else
failed = failed + 1
print '(a, a)', 'FAIL ', name
end if
end subroutine check_true
subroutine check_int(name, actual, expected)
character(len=*), intent(in) :: name
integer, intent(in) :: actual, expected
call check_true(name, actual == expected)
if (actual /= expected) print '(a, i0, a, i0)', ' expected ', &
expected, ', got ', actual
end subroutine check_int
subroutine check_close(name, actual, expected, tolerance)
character(len=*), intent(in) :: name
real(real64), intent(in) :: actual, expected, tolerance
call check_true(name, abs(actual - expected) <= tolerance)
if (abs(actual - expected) > tolerance) print '(a, es12.4)', &
' the difference was ', abs(actual - expected)
end subroutine check_close
subroutine summary()
print '(a, i0, a, i0, a, i0)', 'passed ', passed, ', failed ', &
failed, ', tests in total ', passed + failed
if (failed > 0) then
print '(a)', 'the test suite failed'
error stop 1
end if
print '(a)', 'the test suite passed'
end subroutine summary
end module testing
Each check prints one line, prefixed with ok or FAIL,
so that a failing run can be read from the bottom upwards: the summary gives the
counts and the failing lines give the names. Keeping the counters in the module
means the test program does not have to thread them through every call, at the
price of state that a test cannot reset; for a suite that runs once per process
that is the right trade.
$ gfortran -std=f2018 -Wall -Wextra -fcheck=all -c testing.f90 -o testing.o
A suite that passes
The test program uses the harness and the module, calls one check per property
and ends with summary. The properties are the interesting part:
the mean and median of an odd count, the median of an even count, a standard
deviation computed by hand, and a mean of negative values, which is where a
sign mistake would show.
program test_stats
use iso_fortran_env, only: real64
use testing
use stats
implicit none
real(real64), parameter :: data(5) = [4.0_real64, 1.0_real64, &
3.0_real64, 2.0_real64, 5.0_real64]
call check_close('the mean of 1 to 5', mean(data), 3.0_real64, 1.0e-12_real64)
call check_close('the median of 1 to 5', median(data), 3.0_real64, &
1.0e-12_real64)
call check_close('the median of 1 to 4', median(data(1:4)), 2.5_real64, &
1.0e-12_real64)
call check_close('the standard deviation', stddev(data), 1.58113883_real64, &
1.0e-8_real64)
call check_close('the mean of negative values', &
mean([-3.0_real64, -5.0_real64]), -4.0_real64, 1.0e-12_real64)
call summary
end program test_stats
Five lines of ok and a summary of five passed, zero failed. The
program exits with status zero, and the numbers in the output are the ones the
demo printed, now checked rather than read. The names of the tests are the
documentation of what the module promises, and they are the first thing a reader
sees when a change breaks something.
$ gfortran -std=f2018 -Wall -Wextra -fcheck=all test_stats.f90 stats.f90 testing.f90 -o test_stats $ ./test_stats; echo "exit: $?" ok the mean of 1 to 5 ok the median of 1 to 5 ok the median of 1 to 4 ok the standard deviation ok the mean of negative values passed 5, failed 0, tests in total 5 the test suite passed exit: 0
A test that fails
A test is only useful if its failure is unmistakable. The next program contains one wrong expectation on purpose: it asks for a median of 2.0 where the correct value is 2.5, and it also checks that the median is below the mean, which is false for an even count because the median is the average of the middle pair. The suite reports both failures and exits with status 1.
program test_fail
use iso_fortran_env, only: real64
use testing
use stats
implicit none
real(real64), parameter :: data(4) = [4.0_real64, 1.0_real64, &
3.0_real64, 2.0_real64]
call check_close('the mean of 1 to 4', mean(data), 2.5_real64, 1.0e-12_real64)
call check_close('the median of 1 to 4', median(data), 2.0_real64, &
1.0e-12_real64)
call check_true('the median is below the mean', median(data) < mean(data))
call summary
end program test_fail
The failing check_close prints the difference it found, 0.5, next
to the expected and the actual value, and the failing logical check prints only
its name. The summary says one passed and two failed, and
error stop 1 ends the program with the status a build system reads.
The compile line adds -fno-backtrace so that the failure is reported
without a page of addresses, exactly as the error handling article describes.
$ gfortran -std=f2018 -Wall -Wextra -fcheck=all -fno-backtrace test_fail.f90 stats.f90 testing.f90 -o test_fail
$ ./test_fail; echo "exit: $?"
ok the mean of 1 to 4
FAIL the median of 1 to 4
the difference was 5.0000E-01
FAIL the median is below the mean
passed 1, failed 2, tests in total 3
the test suite failed
ERROR STOP 1
exit: 1
Comparing real numbers
Two real numbers that come from different paths rarely compare equal, even when the mathematics says they should. The classic example is one tenth plus two tenths, which is not three tenths in binary: the stored values differ in the seventeenth digit, and an equality test fails. A test for real results therefore compares the absolute difference with a tolerance, or a relative one when the values are large.
program test_close
use iso_fortran_env, only: real64
use testing
implicit none
real(real64) :: x, y, difference
x = 0.1_real64 + 0.2_real64
y = 0.3_real64
difference = abs(x - y)
print '(a, f0.20)', 'x = 0.1 + 0.2 = ', x
print '(a, f0.20)', 'y = 0.3 = ', y
print '(a, es12.4)', 'the difference: ', difference
call check_true('the exact comparison fails', .not. (x == y))
call check_close('x and y agree to 1e-16', x, y, 1.0e-16_real64)
call check_close('x and y agree to 1e-12', x, y, 1.0e-12_real64)
call check_true('the difference is below epsilon', difference < &
epsilon(1.0_real64))
call summary
end program test_close
The printed values show the source of the problem: 0.30000000000000004441
against 0.29999999999999998890, a difference of 5.55e-17, which is a little
less than the epsilon of a double. The exact comparison is
therefore false, the tolerance of 1e-16 accepts the pair, and the compiler even
warns about the equality test with -Wcompare-reals. All four checks
pass, and the warning in the compile output is the compiler agreeing with the
first line of the test.
$ gfortran -std=f2018 -Wall -Wextra -fcheck=all -fno-backtrace test_close.f90 testing.f90 -o test_close
test_close.f90:15:57:
15 | call check_true('the exact comparison fails', .not. (x == y))
| 1
Warning: Equality comparison for REAL(8) at (1) [-Wcompare-reals]
$ ./test_close; echo "exit: $?"
x = 0.1 + 0.2 = .30000000000000004441
y = 0.3 = .29999999999999998890
the difference: 5.5511E-17
ok the exact comparison fails
ok x and y agree to 1e-16
ok x and y agree to 1e-12
ok the difference is below epsilon
passed 4, failed 0, tests in total 4
the test suite passed
exit: 0
Edge cases
The ordinary inputs are the ones the author had in mind; the bugs live at the edges. A suite should include an array of one element, an array of two, an array whose values are all equal, a set of large values with a small spread, and a set that mixes signs. The program below tests all of them, including the standard deviation of four values around one billion, where the naive formula that subtracts the squares loses most of its digits.
program test_edges
use iso_fortran_env, only: real64
use testing
use stats
implicit none
real(real64), parameter :: single(1) = [7.0_real64]
real(real64), parameter :: pair(2) = [4.0_real64, 8.0_real64]
real(real64), parameter :: flat(4) = [2.0_real64, 2.0_real64, &
2.0_real64, 2.0_real64]
real(real64), parameter :: shifted(4) = [1.0e9_real64 + 1.0_real64, &
1.0e9_real64 + 2.0_real64, 1.0e9_real64 + 3.0_real64, &
1.0e9_real64 + 4.0_real64]
real(real64), parameter :: mixed(5) = [3.0_real64, -1.0_real64, &
4.0_real64, 1.0_real64, -2.0_real64]
call check_close('the median of one element', median(single), 7.0_real64, &
1.0e-12_real64)
call check_close('the median of two elements', median(pair), 6.0_real64, &
1.0e-12_real64)
call check_close('the median of equal elements', median(flat), 2.0_real64, &
1.0e-12_real64)
call check_close('the deviations of equal elements', stddev(flat), &
0.0_real64, 1.0e-12_real64)
call check_close('the deviation with a large offset', stddev(shifted), &
1.29099444_real64, 1.0e-6_real64)
call check_int('the values above the mean', count(mixed > mean(mixed)), 2)
call check_close('the mean of mixed signs', mean(mixed), 1.0_real64, &
1.0e-12_real64)
call summary
end program test_edges
All seven checks pass, and the one at the large offset passes because the module computes the deviations from the mean instead of subtracting two huge sums of squares; the two-pass form keeps about seven digits where the naive one would keep none. An edge case that passes is as valuable as an ordinary one: it records that the behaviour was considered, so a later rewrite that breaks it is caught immediately.
$ gfortran -std=f2018 -Wall -Wextra -fcheck=all -fno-backtrace test_edges.f90 stats.f90 testing.f90 -o test_edges $ ./test_edges; echo "exit: $?" ok the median of one element ok the median of two elements ok the median of equal elements ok the deviations of equal elements ok the deviation with a large offset ok the values above the mean ok the mean of mixed signs passed 7, failed 0, tests in total 7 the test suite passed exit: 0
Running the suite
A project has several test programs, and the build needs one answer from all of them. A shell loop over the executables does the job: it runs each one with its output discarded, prints a line per program and remembers a failure in a single status variable. The names are arguments, so the same script can run everything or a selection while a single test is being debugged.
#!/bin/sh
status=0
for test in "$@"; do
if ./"$test" > /dev/null 2>&1; then
echo "PASS $test"
else
echo "FAIL $test"
status=1
fi
done
if [ "$status" -eq 0 ]; then
echo "all tests passed"
else
echo "some tests failed"
fi
exit "$status"
The first run selects the three programs that should pass, and the script
reports three passes and exits with zero. The second includes the failing test,
which prints nothing because its output is redirected, and the script reports
the failure and exits with 1. That number is what a continuous integration
server or a make test target checks, so the suite needs no other
protocol.
$ ./run_tests.sh test_stats test_close test_edges; echo "exit: $?" PASS test_stats PASS test_close PASS test_edges all tests passed exit: 0 $ ./run_tests.sh test_stats test_fail; echo "exit: $?" PASS test_stats FAIL test_fail some tests failed exit: 1
Next steps
The compiling and debugging
article covers the flags and the exit statuses that a test run depends on, and
the style guide explains the naming that
makes a test name read as a sentence. The
floating-point article explains why one
tenth plus two tenths is not three tenths, the
modules article covers the
.mod files that the harness and the module produce, and the
error handling article describes
error stop in more detail.
Source
Fortran 2018 standard (J3/18-007r1)
In this article we have written a statistics module with a mean, a median and a standard deviation, built a harness that counts passing and failing checks and turns a failure into exit status 1, run a suite of five properties, watched a deliberately wrong expectation produce a readable failure, compared real numbers with tolerances and seen why equality fails, tested the edge cases of an array of one, of two and of four equal values, and collected several test programs into a script that returns one status. The previous article, Compiling and debugging Fortran, supplies the compiler flags that every transcript here uses.
Author
List all Fortran tutorials.