By using pytest, it becomes simple to run one, several or all test-functions. It has many advanced features, which are not needed for this exercise; but feel free to visit the website.
Pytest uses autodiscovery to find all tests. This makes all test-scripts a lot shorter (and easier to maintain), as the “main-trick” isn’t needed in all those files.
Without pytest all test-files should have a section like:
if __name__ == "__main__": test_P() test_clip() ... # list ALL your test here!
Effectively, pytest will automatically discover all test-functions; and execute them as-if that section is added, with all test-function listed (in file-order).
Installation of pytest is trivial; use:
[[email protected]:] % pip install pytest
Running all tests (in 1 directory) is trivial too:
[[email protected]:../dPID] % pytest ======================================================== test session starts ========================================================= platform darwin -- Python 3.4.1, pytest-3.0.4, py-1.4.31, pluggy-0.4.0 rootdir: /Users/albert/work/MESS,hg/pyMESS/training/dPID/dPID, inifile: collected 2 items test_examples.py .F ============================================================== FAILURES ============================================================== _____________________________________________________________ test_clip ______________________________________________________________ def test_clip(): c = dPID(1,2,3) c.set_min_max(min_result=10) c.set_min_max(max_result=10) for sp in range(-100,100,10): c.setpoint(sp) c.measured(0) got = c.result() > assert got == 10, "Both min and max are clipped to 10; so result should be 10!. But it is: %s" % c.result() E AssertionError: Both min and max are clipped to 10; so result should be 10!. But it is: 0.0 E assert 0.0 == 10 test_examples.py:33: AssertionError ================================================= 1 failed, 1 passed in 0.09 seconds =================================================
expect AssertionErrors (ONLY)
- As the class isn’t implemented, one should expect Asserts during those (initial) runs.
- Make sure you find AssertionErrors only; no syntax-errors etc! They denote mistakes in your code!
The used test-file (test_examples.py) can be found here
To make this (autodiscovery) possible, one has to fullfil a few conventions:
- All (python) files containing test-functions, should start with
- Alternative: end with
- Typically, I use the prefix for black-box and glass-box tests. And the suffix for white-box tests.
- Alternative: end with
- All test-functions should have a name starting with
- No other function should not use that prefix!
- Test-functions are called without arguments
- We don’t use/need fixtures here; which look like function-parameters. So, define all test-functions without parameters!
OK or NOK: Assert on failure¶
Every test should result in a single-bit of information: OK nor Not-OK. Sometimes it may be useful to log (print) intermediate results; that can’t replace the OK/NOK bit however.
With pytest this is easy: use the
Typically a test ends with an assert. However, it’s perfectly normal to have many asserts in one test-function; each one acts as a kind of sub-test. When a test succeeds hardly any output is generated; preventing cluttering of the test-reports.
When the first assert-expression results in
False the test Fails. Then that AssertionError is
show with some context. Giving the programmer feedback on which test fails and why.
Assert is NOT a function
assert is a keyword with one or two expressions.
Don’t use it as a function; which is a common (starters) mistake. Then, it is read as a single
expression: a tuple with two elements. Which is always
True. So the
assert never fails!
Typically, the second expression is a string explaining what is expected. And so, documents that part of the test.