Pytest introduction

status:Beta

By using pytest, it becomes simple to run one, several or all test-functions. It has many advanced features, which are not needed for this exercise; but feel free to visit the website.

Pytest uses autodiscovery to find all tests. This makes all test-scripts a lot shorter (and easier to maintain), as the “main-trick” isn’t needed in all those files.

Without pytest all test-files should have a section like:

if __name__ == "__main__":
   test_P()
   test_clip()
   ...
   # list ALL your test here!

Effectively, pytest will automatically discover all test-functions; and execute them as-if that section is added, with all test-function listed (in file-order).

Example

  • Installation of pytest is trivial; use:

    [[email protected]:] % pip install pytest
    
  • Running all tests (in 1 directory) is trivial too:

    [[email protected]:../dPID] % pytest
    ======================================================== test session starts =========================================================
    platform darwin -- Python 3.4.1, pytest-3.0.4, py-1.4.31, pluggy-0.4.0
    rootdir: /Users/albert/work/MESS,hg/pyMESS/training/dPID/dPID, inifile:
    collected 2 items
    
    test_examples.py .F
    
    ============================================================== FAILURES ==============================================================
    _____________________________________________________________ test_clip ______________________________________________________________
    
        def test_clip():
            c = dPID(1,2,3)
    
            c.set_min_max(min_result=10)
            c.set_min_max(max_result=10)
    
            for sp in range(-100,100,10):
                c.setpoint(sp)
                c.measured(0)
    
                got = c.result()
    >           assert got == 10, "Both min and max are clipped to 10; so result should be 10!. But it is: %s" % c.result()
    E           AssertionError: Both min and max are clipped to 10; so result should be 10!. But it is: 0.0
    E           assert 0.0 == 10
    
    test_examples.py:33: AssertionError
    ================================================= 1 failed, 1 passed in 0.09 seconds =================================================
    

    Note

    expect AssertionErrors (ONLY)

    • As the class isn’t implemented, one should expect Asserts during those (initial) runs.
    • Make sure you find AssertionErrors only; no syntax-errors etc! They denote mistakes in your code!
  • The used test-file (test_examples.py) can be found here

Conventions

To make this (autodiscovery) possible, one has to fullfil a few conventions:

  1. All (python) files containing test-functions, should start with test_
    • Alternative: end with _test.py
    • Typically, I use the prefix for black-box and glass-box tests. And the suffix for white-box tests.
  2. All test-functions should have a name starting with test_
    • No other function should not use that prefix!
  3. Test-functions are called without arguments
    • We don’t use/need fixtures here; which look like function-parameters. So, define all test-functions without parameters!

OK or NOK: Assert on failure

Every test should result in a single-bit of information: OK nor Not-OK. Sometimes it may be useful to log (print) intermediate results; that can’t replace the OK/NOK bit however.

With pytest this is easy: use the assert statement!

Typically a test ends with an assert. However, it’s perfectly normal to have many asserts in one test-function; each one acts as a kind of sub-test. When a test succeeds hardly any output is generated; preventing cluttering of the test-reports.

When the first assert-expression results in False the test Fails. Then that AssertionError is show with some context. Giving the programmer feedback on which test fails and why.

Warning

Assert is NOT a function

In python assert is a keyword with one or two expressions.

Don’t use it as a function; which is a common (starters) mistake. Then, it is read as a single expression: a tuple with two elements. Which is always True. So the assert never fails!

Typically, the second expression is a string explaining what is expected. And so, documents that part of the test.

Comments

comments powered by Disqus