6 .Nd write tests that implement the Test Anything Protocol
12 library provides functions for writing test scripts that produce output
13 consistent with the Test Anything Protocol. A test harness that parses
14 this protocol can run these tests and produce useful reports indicating
15 their success or failure.
17 In the descriptions that follow, for any function that takes as the
19 .Dq Fa const char * , Fa ...
20 it can be assumed that the
24 -like format string, and the optional arguments are values to be placed
27 .Bl -tag -width indent
30 .Fn plan_tests "unsigned int"
34 .Fn plan_no_plan "void"
38 .Fn plan_skip_all "const char *" "..."
42 You must first specify a test plan. This indicates how many tests you
43 intend to run, and allows the test harness to notice if any tests were
44 missed, or if the test program exited prematurely.
48 The function will cause your program to exit prematurely if you specify
51 In some situations you may not know how many tests you will be running, or
52 you are developing your test program, and do not want to update the
54 parameter every time you make a change. For those situations use
56 It indicates to the test harness that an indeterminate number
63 will cause your test program to exit prematurely with a diagnostic
64 message if they are called more than once.
66 If your test program detects at run time that some required functionality
67 is missing (for example, it relies on a database connection which is not
68 present, or a particular configuration option that has not been included
69 in the running kernel) use
71 passing as parameters a string to display indicating the reason for skipping
74 .Bl -tag -width indent
77 .Fn ok "expression" "const char *" "..."
85 .Fn pass "const char *" "..."
89 .Fn fail "const char *" "..."
93 Tests are implemented as expressions checked by calls to the
99 should evaluate to true if the test succeeded.
102 allows you to specify a name, or comment, describing the test which will
103 be included in the output.
105 is for those times when the expression to be tested is self
106 explanatory and does not need an associated comment. In those cases
107 the test expression becomes the comment.
109 These four calls are equivalent:
110 .Bd -literal -offset indent
113 ok(i == 5, "i equals 5"); /* Overly verbose */
114 ok(i == 5, "i equals %d", i); /* Just to demonstrate printf-like
115 behaviour of the test name */
116 ok(i == 5, "i == 5"); /* Needless repetition */
117 ok1(i == 5); /* Just right */
120 It is good practice to ensure that the test name describes the meaning
121 behind the test rather than what you are testing. Viz
122 .Bd -literal -offset indent
123 ok(db != NULL, "db is not NULL"); /* Not bad, but */
124 ok(db != NULL, "Database conn. succeeded"); /* this is better */
130 return 1 if the expression evaluated to true, and 0 if it evaluated to
131 false. This lets you chain calls from
135 to only produce diagnostic output if the test failed. For example, this
136 code will include diagnostic information about why the database connection
137 failed, but only if the test failed.
138 .Bd -literal -offset indent
139 if (!ok(db != NULL, "Database conn. succeeded")) {
140 diag("Database error code: %d", dberrno);
148 From the Test::More documentation:
149 .Bd -literal -offset indent
150 Sometimes you just want to say that the tests have passed.
151 Usually the case is you've got some complicated condition
152 that is difficult to wedge into an ok(). In this case,
153 you can simply use pass() (to declare the test ok) or fail
156 Use these very, very, very sparingly.
159 These are synonyms for ok(1, ...) and ok(0, ...).
161 .Bl -tag -width indent
164 .Fn skip "unsigned int" "const char *" "..."
167 .Fn skip_if "expression" "unsigned int" "const char *" "..."
171 Sets of tests can be skipped. Ordinarily you would do this because
172 the test can't be run in this particular testing environment.
174 For example, suppose some tests should be run as root. If the test is
175 not being run as root then the tests should be skipped. In this
176 implementation, skipped tests are flagged as being ok, with a special
177 message indicating that they were skipped. It is your responsibility
178 to ensure that the number of tests skipped (the first parameter to
180 is correct for the number of tests to skip.
182 One way of implementing this is with a
185 .Dq if( ) { } else { }
186 construct, to ensure that there are no additional side effects from the
188 .Bd -literal -offset indent
190 skip(1, "because test only works as root");
192 ok(do_something_as_root() == 0, "Did something as root");
196 A convenient macro is provided to assist with this. The previous example could
197 be re-written as follows.
198 .Bd -literal -offset indent
199 skip_if(getuid() != 0, 1, "because test only works as root") {
200 ok(do_something_as_root() == 0, "Did something as root");
203 .Ss MARKING TESTS AS Dq TODO
204 .Bl -tag -width indent
207 .Fn todo_start "const char *" "..."
215 Sets of tests can be flagged as being
217 These are tests that you expect to fail, probably because you haven't
218 fixed a bug, or finished a new feature yet. These tests will still be
219 run, but with additional output that indicates that they are expected
220 to fail. Should a test start to succeed unexpectedly, tools like
222 will indicate this, and you can move the test out of the todo
223 block. This is much more useful than simply commenting out (or
224 .Dq #ifdef 0 ... #endif )
226 .Bd -literal -offset indent
227 todo_start("dwim() not returning true yet");
229 ok(dwim(), "Did what the user wanted");
236 ever start succeeding you will know about it as soon as you run the
241 family, additional code between
248 From the Test::More documentation;
249 .Bd -literal -offset indent
250 If it's something the user might not be able to do, use SKIP.
251 This includes optional modules that aren't installed, running
252 under an OS that doesn't have some feature (like fork() or
253 symlinks), or maybe you need an Internet connection and one
256 If it's something the programmer hasn't done yet, use TODO.
257 This is for any code you haven't written yet, or bugs you have
258 yet to fix, but want to put tests in your testing script
259 (always a good idea).
261 .Ss DIAGNOSTIC OUTPUT
262 .Bl -tag -width indent
265 .Fn diag "const char *" "..."
269 If your tests need to produce diagnostic output, use
271 It ensures that the output will not be considered by the TAP test harness.
273 adds the necessary trailing
276 It returns the number of characters written.
277 .Bd -literal -offset indent
278 diag("Expected return code 0, got return code %d", rcode);
281 .Bl -tag -width indent
288 For maximum compatability your test program should return a particular
289 exit code. This is calculated by
291 so it is sufficient to always return from
294 .Dq return exit_status();
296 .Dq exit(exit_status());
301 directory in the source distribution contains numerous tests of
303 functionality, written using
305 Examine them for examples of how to construct test suites.
308 strives to be compatible with the Perl Test::More and Test::Harness
309 modules. The test suite verifies that
311 is bug-for-bug compatible with their behaviour. This is why some
312 functions which would more naturally return nothing return constant
317 is found at compile time,
320 be thread safe. Indications to the contrary (and test cases that expose
321 incorrect behaviour) are very welcome.
324 .Xr Test::Harness 1 ,
330 compiler. Some of the
332 functionality is implemented as variadic macros, and that functionality
333 was not formally codified until C99. Patches to use
335 with earlier compilers that have their own implementation of variadic
336 macros will be gratefully received.
339 was written to help improve the quality and coverage of the FreeBSD
340 regression test suite, and released in the hope that others find it
341 a useful tool to help improve the quality of their code.
343 .An "Nik Clayton" Aq nik@ngo.org.uk ,
347 would not exist without the efforts of
348 .An "Michael G Schwern" Aq schqern@pobox.com ,
349 .An "Andy Lester" Aq andy@petdance.com ,
350 and the countless others who have worked on the Perl QA programme.
352 Ideally, running the tests would have no side effects on the behaviour
353 of the application you are testing. However, it is not always possible
354 to avoid them. The following side effects of using
357 .Bl -bullet -offset indent
359 stdout is set to unbuffered mode after calling any of the