Lightweight testing framework
Damien Diederen <dd@crosstwine.com>
Package testing (global prefix VedaTesting) provides support for automated testing of SKILL(++) packages. It is a port of the Go testing framework (http://golang.org/src/pkg/testing/testing.go) as provided by go1.0.2.
It is intended to select and automate the execution of any method of the form
(defmethod Test_Xxx ((context VedaTestingContext)) ...)
where Xxx can be any alphanumeric string (but the first letter must not be in [a-z]) and serves to identify the test routine. These Test_Xxx routines should be declared within the package they are testing, in files ending in -test.il or -test.ils.
This package does not yet implement the BenchmarkXxx or Example{F,T,T_M} features of the Go testing package.
(defclass VedaTestingContext () ...)
Context instances are passed to Test methods to manage test state and support test logs. Logs are accumulated during execution and dumped to a file or standard output depending on configuration options.
@initarg ?name @reader GetName
Name of the test being executed.
@reader GetFailed @writer SetFailed
A boolean indicating whether the test has failed or not. Defaults to nil.
@reader GetDuration
The duration of the text execution, in seconds, or nil if the test hasn't been run or timing could not be measured.
(defclass VedaTestingOptions () ...)
Class VedaTestingOptions holds the options that are used for running tests.
@initarg ?verbose @reader GetVerboseLevel @writer SetVerboseLevel
Verbosity level. Chatty above 4; defaults to 0.
@initarg ?reportFormat @reader GetReportFormat @writer SetReportFormat
Report format, one of the following symbols: (text xml void). Defaults to text.
@initarg ?reportFilename @reader GetReportFilename @writer SetReportFilename
Filename used by XML report writer. Defaults to nil, which maps to "./result.xml".
(defmethod VedaTestingError ((context VedaTestingContext) @rest args) ...)
Error is equivalent to (Log) followed by (Fail).
(defmethod VedaTestingErrorf ((context VedaTestingContext) format @rest args) ...)
Errorf is equivalent to (Logf) followed by (Fail).
(defmethod VedaTestingFail ((context VedaTestingContext)) ...)
Fail marks the function as having failed but continues execution.
(defmethod VedaTestingFailNow ((context VedaTestingContext)) ...)
FailNow marks the function as having failed and stops its execution. Execution will continue at the next test or benchmark.
(defmethod VedaTestingFatal ((context VedaTestingContext) @rest args) ...)
Fatal is equivalent to (Log) followed by (FailNow).
(defmethod VedaTestingFatalf ((context VedaTestingContext) format @rest args) ...)
Fatalf is equivalent to (Logf) followed by (FailNow).
(defmethod VedaTestingLog ((context VedaTestingContext) @rest args) ...)
Log formats its arguments using default formatting, and records the text in the error log.
(defmethod VedaTestingLogf ((context VedaTestingContext) format @rest args) ...)
Logf formats its arguments according to the format, analogous to printf, and records the text in the error log.
(defun RunTests (tests @key options) ...)
RunTests runs the test functions in tests, a list of funcallable symbols. Returns t if none of the tests failed, nil otherwise.
Test execution and report generation is controlled via ?options; if nil, a default-initialized instance of VedaTestingOptions is used.
tests – listoptions – (or VedaTestingOptions list)(or symbol list)(defun RunTestsFromFiles (files @key options) ...)
RunTestsFromFiles loads files, a list of filenames, while monitoring the set of methods having the following form:
(defmethod Test_Xxx ((context VedaTestingContext)) ...)
It then executes the newly-defined/redefined ones.
Cf. RunTests for more information about ?options.
files – listoptions – (or VedaTestingOptions list)(or symbol list)(defun RunTestsOnPackage (pkg @key options) ...)
RunTestsOnPackage finds the set of source and test files associated with package pkg, reloads all the source files, then loads and runs all the tests contained in the test files.
Cf. RunTests for more information about ?options.
pkg – stringoptions – (or VedaTestingOptions list)(or symbol list)