Atmel Software Framework

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Test Suite Framework

This module is the test suite framework, which provides a set of standard functions and macros for defining and running test suites.

A test suite consists of test cases. Each test case consists of a set of functions that are called in a specific order by the framework when the test suite is run. The runtime environment of a test is referred to as the test fixture. A setup function can be defined for the purpose of fulfilling preconditions that are needed for the test to run. To free up or reset resources which were used in the test case, a cleanup function can be defined.

The following three functions can be defined for each test case:

The setup and/or cleanup functions may be skipped if they are not needed.

A function may report an error or failure by use of the macros test_fail , test_assert_true and test_assert_false. Note that only the first macro allows for the result value to be specified, while the two latter will automatically return TEST_FAIL if the specified condition is false.

If any of the functions return a failure/error result, execution of that specific function is ended. However, depending on which function did not pass, execution of the test case may continue:

The result of the first function in the test case which does not pass determines the end result.

Example code for definition of a test suite:

void setup_1(const struct test_case *test);
void run_1(const struct test_case *test);
void cleanup_1(const struct test_case *test);
void run_2(const struct test_case *test);
...
DEFINE_TEST_CASE(test_1,
setup_1,
run_1,
cleanup_1,
"First test");
run_2,
"Second test");
DEFINE_TEST_ARRAY(my_tests) = {
&test_1,
&test_2,
};
DEFINE_TEST_SUITE(my_suite, my_tests, "Two-test suite");
test_suite_run(&my_suite);
...

Example code for definition of a test case:

void setup_1(const struct test_case *test)
{
void *resource = allocate_test_resource();
if (resource == NULL) {
"Could not allocate test resource");
}
test_set_data(resource);
}
void run_1(const struct test_case *test)
{
void *resource = test_get_data();
int8_t test_result;
...
// TEST CODE
...
test_assert_true(test, test_result > 0,
"Test result is not larger than 0: %i", test_result);
test_assert_false(test, test_result > 2,
"Test result is larger than 2: %i", test_result);
}
void cleanup_1(const struct test_case *test)
{
void *resource = test_get_data();
free_test_resource(resource);
}

Modules

 

Data Structures

struct  test_case
 A test case. More...
 
struct  test_suite
 A test suite. More...
 

Macros

#define ARRAY_LEN(a)   (sizeof(a) / sizeof((a)[0]))
 Convenience macro for counting elements in arrays. More...
 

Functions

static int test_call (void(*func)(const struct test_case *), const struct test_case *test)
 Call a test or fixture function. More...
 
void test_case_fail (const struct test_case *test, int result, const char *file, unsigned int line, const char *fmt,...)
 Report a failure and jump out of current test case function. More...
 
static int test_case_run (const struct test_case *test)
 Run a test case. More...
 
static void test_report_failure (const struct test_case *test, const char *stage, int result)
 Report a failing test stage. More...
 

Variables

struct test_casetest_case_ptr = NULL
 Pointer to current test case. More...
 
static jmp_buf test_failure_jmpbuf
 Context saved before executing a test or fixture function. More...
 
void * test_priv_data
 Data pointer for test cases. More...
 

Wrappers for printing debug information

Note
The test suite framework feeds its output to printf. It is left up to the test application to set up the stream.
#define dbg(__fmt_)   printf(__fmt_)
 
#define dbg_info(__fmt_,...)   printf(__fmt_, __VA_ARGS__)
 
#define dbg_error(_x,...)   printf(_x, __VA_ARGS__)
 
#define dbg_putchar(c)   putc(c, stdout)
 
#define dbg_vprintf_pgm(...)   vfprintf(stdout, __VA_ARGS__)
 

Test suite definition macros

#define DEFINE_TEST_CASE(_sym, _setup, _run, _cleanup, _name)
 Create a test case struct. More...
 
#define DEFINE_TEST_ARRAY(_sym)   const struct test_case *const _sym[]
 Create an array of test case pointers. More...
 
#define DEFINE_TEST_SUITE(_sym, _test_array, _name)
 Create a test suite. More...
 

Test data access

void * test_priv_data
 Data pointer for test cases. More...
 
static void test_set_data (void *data)
 Set private data pointer for the current test. More...
 
static void * test_get_data (void)
 Get the private data pointer for the current test. More...
 

Test case pointer access

struct test_casetest_case_ptr
 Pointer to current test case. More...
 
static void test_set_case (const struct test_case *test)
 Set pointer to current test. More...
 
static struct test_casetest_get_case (void)
 Get pointer to current test. More...
 

Test result reporting

enum  test_status {
  TEST_ERROR = -1,
  TEST_PASS = 0,
  TEST_FAIL = 1
}
 Status codes returned by test cases and fixtures. More...
 
#define test_fail(test, result,...)   test_case_fail(test, result, __FILE__, __LINE__, __VA_ARGS__)
 Fail the test. More...
 
#define test_assert_true(test, condition,...)
 Verify that condition is true. More...
 
#define test_assert_false(test, condition,...)   test_assert_true(test, !(condition), __VA_ARGS__)
 Verify that condition is false. More...
 

Test suite interaction

int test_suite_run (const struct test_suite *suite)
 Run a test suite. More...
 

#define ARRAY_LEN (   a)    (sizeof(a) / sizeof((a)[0]))

Convenience macro for counting elements in arrays.

#define dbg (   __fmt_)    printf(__fmt_)

Referenced by test_case_run().

#define dbg_error (   _x,
  ... 
)    printf(_x, __VA_ARGS__)
#define dbg_info (   __fmt_,
  ... 
)    printf(__fmt_, __VA_ARGS__)

Referenced by test_case_run(), and test_suite_run().

#define dbg_putchar (   c)    putc(c, stdout)

Referenced by test_case_fail().

#define dbg_vprintf_pgm (   ...)    vfprintf(stdout, __VA_ARGS__)

Referenced by test_case_fail().

#define DEFINE_TEST_ARRAY (   _sym)    const struct test_case *const _sym[]

Create an array of test case pointers.

Parameters
_symVariable name of the resulting array

Referenced by main().

#define DEFINE_TEST_CASE (   _sym,
  _setup,
  _run,
  _cleanup,
  _name 
)
Value:
static const char _test_str_##_sym[] = _name; \
static const struct test_case _sym = { \
.setup = _setup, \
.run = _run, \
.cleanup = _cleanup, \
.name = _test_str_##_sym, \
}
A test case.
Definition: suite.h:164
void(* setup)(const struct test_case *test)
Set up the environment in which test is to be run.
Definition: suite.h:172

Create a test case struct.

Parameters
_symVariable name of the resulting struct
_setupFunction which sets up a test case environment. Can be NULL.
_runTest function
_cleanupFunction which cleans up what was set up. Can be NULL.
_nameString describing the test case.

Referenced by main().

#define DEFINE_TEST_SUITE (   _sym,
  _test_array,
  _name 
)
Value:
static const char _test_str_##_sym[] = _name; \
const struct test_suite _sym = { \
.nr_tests = ARRAY_LEN(_test_array), \
.tests = _test_array, \
.name = _test_str_##_sym, \
}
A test suite.
Definition: suite.h:194
unsigned int nr_tests
The number of tests in this suite.
Definition: suite.h:196
#define ARRAY_LEN(a)
Convenience macro for counting elements in arrays.
Definition: suite.h:284

Create a test suite.

Parameters
_symVariable name of the resulting struct
_test_arrayArray of test cases, created with DEFINE_TEST_ARRAY()
_nameString describing the test suite.

Referenced by main().

#define test_assert_false (   test,
  condition,
  ... 
)    test_assert_true(test, !(condition), __VA_ARGS__)

Verify that condition is false.

If condition is true, fail the test with an error message indicating the condition which failed.

Parameters
testThe test case currently being run
conditionExpression to be validated
...Format string and arguments

Referenced by run_fifo_size_test().

#define test_assert_true (   test,
  condition,
  ... 
)
Value:
do { \
if (!(condition)) { \
test_fail(test, TEST_FAIL, __VA_ARGS__); \
} \
} while (0)
#define test_fail(test, result,...)
Fail the test.
Definition: suite.h:379
Test failure.
Definition: suite.h:353
if(Is_usb_id_transition()&&Is_usb_id_interrupt_enabled())
Definition: applications/uc3-audio-player/usb_task.c:395

Verify that condition is true.

If condition is false, fail the test with an error message indicating the condition which failed.

Parameters
testThe test case currently being run
conditionExpression to be validated
...Format string and arguments

Referenced by run_byte_access_test(), run_check_presence_test(), run_data_integrity_test(), run_fifo_flush_test(), run_fifo_overflow_test(), run_fifo_push_pull_test(), run_fifo_size_test(), run_fifo_underflow_test(), run_generic_clock_test(), run_memory_check_test(), run_memory_range_check_test(), run_multiple_sector_access_test(), run_osc32_test(), run_osc_test(), run_pll_dfll_test(), run_sd_mmc_init_test(), run_sd_mmc_rw_test(), run_sd_mmc_sdio_rw_test(), run_sdio_rw_test(), run_sector_access_test(), run_sync_clock_test(), run_test_stop_uhc(), run_usb_cdc_config_test(), run_usb_keyboard_led_test(), run_usb_keyboard_test(), run_usb_msc_read_test(), run_usb_msc_test(), run_usb_resume_test(), run_usb_sof_test(), run_usb_suspend_test(), run_usb_vbus_test(), and sd_mmc_spi_read_multiple_sector_callback().

#define test_fail (   test,
  result,
  ... 
)    test_case_fail(test, result, __FILE__, __LINE__, __VA_ARGS__)

Fail the test.

Calling this function will cause the test to terminate with a failure. It will return directly to the test suite core, not to the caller.

Parameters
testTest case which failed
resultThe result of the test (may not be 0)
...printf()-style format string and arguments

Status codes returned by test cases and fixtures.

Note that test cases and especially fixtures may return any of the status codes defined by status_code as well.

Enumerator
TEST_ERROR 

Test error.

TEST_PASS 

Test success.

TEST_FAIL 

Test failure.

static int test_call ( void(*)(const struct test_case *)  func,
const struct test_case test 
)
static

Call a test or fixture function.

This function will initialize test_failure_jmpbuf and call func with test as the parameter.

Parameters
funcPointer to function to call.
test_casePointer to the function's originating test case.
Returns
TEST_PASS if func was executed successfully, or the result value passed to test_fail() on failure.

References test_failure_jmpbuf, and TEST_PASS.

Referenced by test_case_run().

void test_case_fail ( const struct test_case test,
int  result,
const char *  file,
unsigned int  line,
const char *  fmt,
  ... 
)

Report a failure and jump out of current test case function.

Parameters
testCurrent test case.
resultResult value of failure.
Note
Should not be TEST_PASS.
Parameters
fileName of file in which function resides.
lineLine number at which failure occurred.
fmtFailure message, as printf-formatted string.
...Values to insert into failure message.

References dbg_error, dbg_putchar, dbg_vprintf_pgm, test_case::name, and test_failure_jmpbuf.

static int test_case_run ( const struct test_case test)
static

Run a test case.

This function will call the setup, run and cleanup functions of the specified test case, outputting debug information as it goes, then returning the result value of the test case.

If the setup stage does not pass, the rest of the test case if skipped. If the test stage itself does not pass, the cleanup is still executed.

The result from the first non-passing function in the test case is returned, meaning a failing cleanup will override a successful test.

Parameters
test_caseTest case to run.
Returns
TEST_PASS if all functions execute successfully. Otherwise, the first non-zero value which is returned from the test case.

References test_case::cleanup, dbg, dbg_info, test_case::name, NULL, test_case::run, test_case::setup, test_call(), test_report_failure(), and test_set_case().

Referenced by test_suite_run().

static struct test_case* test_get_case ( void  )
static

Get pointer to current test.

References test_case_ptr.

static void* test_get_data ( void  )
inlinestatic

Get the private data pointer for the current test.

Returns
Pointer to arbitrary run-time data for the test currently being run, previously registered using test_set_data().

References test_priv_data.

Referenced by run_data_integrity_test().

static void test_report_failure ( const struct test_case test,
const char *  stage,
int  result 
)
static

Report a failing test stage.

Parameters
testCurrent test case.
stageName of test stage that failed.
resultResult value of test stage.

References dbg_error, and test_case::name.

Referenced by test_case_run().

static void test_set_case ( const struct test_case test)
inlinestatic

Set pointer to current test.

Referenced by test_case_run().

static void test_set_data ( void *  data)
inlinestatic

Set private data pointer for the current test.

Parameters
dataPointer to arbitrary run-time data for the test currently being run.

References data, and test_priv_data.

Referenced by main().

int test_suite_run ( const struct test_suite suite)

Run a test suite.

Run all tests in suite, in the order in which they are found in the array.

Returns
The number of tests that didn't pass.

References dbg_info, i, test_suite::name, test_suite::nr_tests, test_case_run(), TEST_PASS, and test_suite::tests.

Referenced by main().

struct test_case* test_case_ptr = NULL

Pointer to current test case.

See Also
test_set_case(), test_get_case()

Referenced by test_get_case().

struct test_case* test_case_ptr

Pointer to current test case.

See Also
test_set_case(), test_get_case()

Referenced by test_get_case().

jmp_buf test_failure_jmpbuf
static

Context saved before executing a test or fixture function.

This is used for doing non-local jumps from the test cases on failure.

Referenced by test_call(), and test_case_fail().

void* test_priv_data

Data pointer for test cases.

See Also
test_set_data(), test_get_data()

Referenced by test_get_data(), and test_set_data().

void* test_priv_data

Data pointer for test cases.

See Also
test_set_data(), test_get_data()

Referenced by test_get_data(), and test_set_data().