@settitle Emacs Lisp Regression Testing
@c %**end of header
-@dircategory Emacs
+@dircategory Emacs misc features
@direntry
-* ERT: (ert). Emacs Lisp Regression Testing.
+* ERT: (ert). Emacs Lisp regression testing tool.
@end direntry
@copying
-Copyright @copyright{} 2008, 2010-2011 Free Software Foundation, Inc.
+Copyright @copyright{} 2008, 2010--2013 Free Software Foundation, Inc.
@quotation
Permission is granted to copy, distribute and/or modify this document
any later version published by the Free Software Foundation; with no
Invariant Sections, with the Front-Cover texts being ``A GNU Manual,''
and with the Back-Cover Texts as in (a) below. A copy of the license
-is included in the section entitled ``GNU Free Documentation License''
-in the Emacs manual.
+is included in the section entitled ``GNU Free Documentation License''.
(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
-modify this GNU manual. Buying copies from the FSF supports it in
-developing GNU and promoting software freedom.''
-
-This document is part of a collection distributed under the GNU Free
-Documentation License. If you want to distribute this document
-separately from the collection, you can do so by adding a copy of the
-license to the document, as described in section 6 of the license.
+modify this GNU manual.''
@end quotation
@end copying
-@node Top, Introduction, (dir), (dir)
+@node Top
@top ERT: Emacs Lisp Regression Testing
ERT is a tool for automated testing in Emacs Lisp. Its main features
* How to Debug Tests:: What to do if a test fails.
* Extending ERT:: ERT is extensible in several ways.
* Other Testing Concepts:: Features not in ERT.
+* GNU Free Documentation License:: The license for this documentation.
@detailmenu
--- The Detailed Node Listing ---
* Mocks and Stubs:: Stubbing out code that is irrelevant to the test.
* Fixtures and Test Suites:: How ERT differs from tools for other languages.
+Appendix
+
+* GNU Free Documentation License:: The license for this documentation.
+
@end detailmenu
@end menu
-@node Introduction, How to Run Tests, Top, Top
+@node Introduction
@chapter Introduction
ERT allows you to define @emph{tests} in addition to functions,
macros, variables, and the other usual Lisp constructs. Tests are
-simply Lisp code --- code that invokes other code and checks whether
+simply Lisp code: code that invokes other code and checks whether
it behaves as expected.
ERT keeps track of the tests that are defined and provides convenient
will pass if the three calls to @code{equal} all return true
(non-nil).
-@code{should} is a macro with the same meaning as @code{assert} but
+@code{should} is a macro with the same meaning as @code{cl-assert} but
better error reporting. @xref{The @code{should} Macro}.
Each test should have a name that describes what functionality it tests.
-Test names can be chosen arbitrarily --- they are in a
-namespace separate from functions and variables --- but should follow
+Test names can be chosen arbitrarily---they are in a
+namespace separate from functions and variables---but should follow
the usual Emacs Lisp convention of having a prefix that indicates
which package they belong to. Test names are displayed by ERT when
reporting failures and can be used when selecting which tests to run.
Environment}.
-@node How to Run Tests, How to Write Tests, Introduction, Top
+@node How to Run Tests
@chapter How to Run Tests
You can run tests either in the Emacs you are working in, or on the
@end menu
-@node Running Tests Interactively, Running Tests in Batch Mode, How to Run Tests, How to Run Tests
+@node Running Tests Interactively
@section Running Tests Interactively
You can run the tests that are currently defined in your Emacs with
@example
Selector: t
-Passed: 31
-Failed: 2 (2 unexpected)
-Total: 33/33
+Passed: 31
+Skipped: 0
+Failed: 2 (2 unexpected)
+Total: 33/33
Started at: 2008-09-11 08:39:25-0700
Finished.
of the expression.
-@node Running Tests in Batch Mode, Test Selectors, Running Tests Interactively, How to Run Tests
+@node Running Tests in Batch Mode
@section Running Tests in Batch Mode
ERT supports automated invocations from the command line or from
files that it requires are on your @code{load-path}.
-@node Test Selectors, , Running Tests in Batch Mode, How to Run Tests
+@node Test Selectors
@section Test Selectors
Functions like @code{ert} accept a @emph{test selector}, a Lisp
@code{:causes-redisplay}.
-@node How to Write Tests, How to Debug Tests, How to Run Tests, Top
+@node How to Write Tests
@chapter How to Write Tests
ERT lets you define tests in the same way you define functions. You
* Useful Techniques:: Some examples.
@end menu
-@node The @code{should} Macro, Expected Failures, How to Write Tests, How to Write Tests
+@node The @code{should} Macro
@section The @code{should} Macro
Test bodies can include arbitrary code; but to be useful, they need to
check whether the code being tested (or @emph{code under test})
does what it is supposed to do. The macro @code{should} is similar to
-@code{assert} from the cl package
+@code{cl-assert} from the cl package
(@pxref{Assertions,,, cl, Common Lisp Extensions}),
but analyzes its argument form and records information that ERT can
display to help debugging.
@code{arith-error}. The @code{:type} argument to @code{should-error}
is optional; if absent, any type of error is accepted.
@code{should-error} returns an error description of the error that was
-signalled, to allow additional checks to be made. The error
+signaled, to allow additional checks to be made. The error
description has the format @code{(ERROR-SYMBOL . DATA)}.
There is no @code{should-not-error} macro since tests that signal an
@code{should} reports.
-@node Expected Failures, Tests and Their Environment, The @code{should} Macro, How to Write Tests
+@node Expected Failures
@section Expected Failures
Some bugs are complicated to fix, or not very important, and are left as
@end lisp
-@node Tests and Their Environment, Useful Techniques, Expected Failures, How to Write Tests
+@node Tests and Their Environment
@section Tests and Their Environment
+Sometimes, it doesn't make sense to run a test due to missing
+preconditions. A required Emacs feature might not be compiled in, the
+function to be tested could call an external binary which might not be
+available on the test machine, you name it. In this case, the macro
+@code{skip-unless} could be used to skip the test:
+
+@lisp
+(ert-deftest test-dbus ()
+ "A test that checks D-BUS functionality."
+ (skip-unless (featurep 'dbusbind))
+ ...)
+@end lisp
+
The outcome of running a test should not depend on the current state
of the environment, and each test should leave its environment in the
same state it found it in. In particular, a test should not depend on
Instead, it is better to use lower-level mechanisms with simple and
predictable semantics like @code{with-temp-buffer}, @code{insert} or
@code{insert-file-contents-literally}, and to activate any desired mode
-by calling the corresponding function directly --- after binding the
+by calling the corresponding function directly, after binding the
hook variables to nil. This avoids the above problems.
-@node Useful Techniques, , Tests and Their Environment, How to Write Tests
+@node Useful Techniques
@section Useful Techniques when Writing Tests
Testing simple functions that have no side effects and no dependencies
The reason why this test only checks the first line of the backtrace
is that the remainder of the backtrace is dependent on ERT's internals
as well as whether the code is running interpreted or compiled. By
-looking only at the first line, the test checks a useful property
---- that the backtrace correctly captures the call to @code{signal} that
-results from the call to @code{ert-fail} --- without being brittle.
+looking only at the first line, the test checks a useful property---that
+the backtrace correctly captures the call to @code{signal} that
+results from the call to @code{ert-fail}---without being brittle.
This example also shows that writing tests is much easier if the code
under test was structured with testing in mind.
well.
-@node How to Debug Tests, Extending ERT, How to Write Tests, Top
+@node How to Debug Tests
@chapter How to Debug Tests
This section describes how to use ERT's features to understand why
@end menu
-@node Understanding Explanations, Interactive Debugging, How to Debug Tests, How to Debug Tests
+@node Understanding Explanations
@section Understanding Explanations
Failed @code{should} forms are reported like this:
function registered. @xref{Defining Explanation Functions}.
-@node Interactive Debugging, , Understanding Explanations, How to Debug Tests
+@node Interactive Debugging
@section Interactive Debugging
Debugging failed tests essentially works the same way as debugging any
failed. This can be useful to figure out how far it got.
@item You can instrument tests for debugging the same way you instrument
-@code{defun}s for debugging --- go to the source code of the test and
+@code{defun}s for debugging: go to the source code of the test and
type @kbd{@kbd{C-u} @kbd{C-M-x}}. Then, go back to the ERT buffer and
re-run the test with @kbd{r} or @kbd{d}.
@item If you have been editing and rearranging tests, it is possible that
-ERT remembers an old test that you have since renamed or removed ---
+ERT remembers an old test that you have since renamed or removed:
renamings or removals of definitions in the source code leave around a
stray definition under the old name in the running process (this is a
common problem in Lisp). In such a situation, hit @kbd{D} to let ERT
@end itemize
-@node Extending ERT, Other Testing Concepts, How to Debug Tests, Top
+@node Extending ERT
@chapter Extending ERT
There are several ways to add functionality to ERT.
@end menu
-@node Defining Explanation Functions, Low-Level Functions for Working with Tests, Extending ERT, Extending ERT
+@node Defining Explanation Functions
@section Defining Explanation Functions
The explanation function for a predicate is a function that takes the
explanation function.
-@node Low-Level Functions for Working with Tests, , Defining Explanation Functions, Extending ERT
+@node Low-Level Functions for Working with Tests
@section Low-Level Functions for Working with Tests
Both @code{ert-run-tests-interactively} and @code{ert-run-tests-batch}
Contributions to ERT are welcome.
-@node Other Testing Concepts, , Extending ERT, Top
+@node Other Testing Concepts
@chapter Other Testing Concepts
For information on mocks, stubs, fixtures, or test suites, see below.
* Fixtures and Test Suites:: How ERT differs from tools for other languages.
@end menu
-@node Mocks and Stubs, Fixtures and Test Suites, Other Testing Concepts, Other Testing Concepts
+@node Mocks and Stubs
@section Other Tools for Emacs Lisp
Stubbing out functions or using so-called @emph{mocks} can make it
offers mocks for Emacs Lisp and can be used in conjunction with ERT.
-@node Fixtures and Test Suites, , Mocks and Stubs, Other Testing Concepts
+@node Fixtures and Test Suites
@section Fixtures and Test Suites
In many ways, ERT is similar to frameworks for other languages like
selector "^ert-" selects ERT's self-tests.
Other uses include grouping tests by their expected execution time,
-e.g. to run quick tests during interactive development and slow tests less
+e.g., to run quick tests during interactive development and slow tests less
often. This can be achieved with the @code{:tag} argument to
@code{ert-deftest} and @code{tag} test selectors.
+@node GNU Free Documentation License
+@appendix GNU Free Documentation License
+@include doclicense.texi
+
@bye
@c LocalWords: ERT JUnit namespace docstring ERT's
HCoop / bpt / emacs.git / blobdiff