Pytest introduction

status:

Beta

By using pytest, it becomes simple to run one, several or all test-functions. It has many advanced features, which are not needed for this exercise; but feel free to visit the website.

Pytest uses autodiscovery to find all tests. This makes all test-scripts a lot shorter (and easier to maintain), as the “main-trick” isn’t needed in all those files.

Without pytest all test-files should have a section like:

if __name__ == "__main__":
   test_P()
   test_clip()
   ...
   # list ALL your test here!

Effectively, pytest will automatically discover all test-functions; and execute them as-if that section is added, with all test-function listed (in file-order).

Example

  • Installation of pytest is trivial; use:

    [Albert@pyMESS:] % pip install pytest

  • Running all tests (in 1 directory) is trivial too:

    [Albert@pyMESS:../dPID] % pytest
======================================================== test session starts =========================================================
platform darwin -- Python 3.4.1, pytest-3.0.4, py-1.4.31, pluggy-0.4.0
rootdir: /Users/albert/work/MESS,hg/pyMESS/training/dPID/dPID, inifile:
collected 2 items

test_examples.py .F

============================================================== FAILURES ==============================================================
_____________________________________________________________ test_clip ______________________________________________________________

    def test_clip():
        c = dPID(1,2,3)

        c.set_min_max(min_result=10)
        c.set_min_max(max_result=10)

        for sp in range(-100,100,10):
            c.setpoint(sp)
            c.measured(0)

            got = c.result()
>           assert got == 10, "Both min and max are clipped to 10; so result should be 10!. But it is: %s" % c.result()
E           AssertionError: Both min and max are clipped to 10; so result should be 10!. But it is: 0.0
E           assert 0.0 == 10

test_examples.py:33: AssertionError
================================================= 1 failed, 1 passed in 0.09 seconds =================================================

    Note

    expect AssertionErrors (ONLY)

    • As the class isn’t implemented, one should expect Asserts during those (initial) runs.

    • Make sure you find AssertionErrors only; no syntax-errors etc! They denote mistakes in your code!

  • The used test-file (test_examples.py) can be found here

Conventions

To make this (autodiscovery) possible, one has to fullfil a few conventions:

  1. All (python) files containing test-functions, should start with test_

    • Alternative: end with _test.py

    • Typically, I use the prefix for black-box and glass-box tests. And the suffix for white-box tests.

  2. All test-functions should have a name starting with test_

    • No other function should not use that prefix!

  3. Test-functions are called without arguments

    • We don’t use/need fixtures here; which look like function-parameters. So, define all test-functions without parameters!

OK or NOK: Assert on failure

Every test should result in a single-bit of information: OK nor Not-OK. Sometimes it may be useful to log (print) intermediate results; that can’t replace the OK/NOK bit however.

With pytest this is easy: use the assert statement!

Typically a test ends with an assert. However, it’s perfectly normal to have many asserts in one test-function; each one acts as a kind of sub-test. When a test succeeds hardly any output is generated; preventing cluttering of the test-reports.

When the first assert-expression results in False the test Fails. Then that AssertionError is show with some context. Giving the programmer feedback on which test fails and why.

Warning

Assert is NOT a function

In python assert is a keyword with one or two expressions.

Don’t use it as a function; which is a common (starters) mistake. Then, it is read as a single expression: a tuple with two elements. Which is always True. So the assert never fails!

Typically, the second expression is a string explaining what is expected. And so, documents that part of the test.

Comments

comments powered by Disqus