This talk is designed to discuss the concepts of software testing. In it, I will outline the concept of software testing and summarize the current real-world best practices and procedures. We will then discuss how to, and the difficulties of, applying this to scientific software, but we won't actually discuss how to do it this week: next time will be a tutorial for that, based on this week's feedback.

CategoryTutorial

The presentation

Before you start

Outline

Basic concepts of unit testing

Testing is considered one of the cornerstones to good software.

Test driven development

https://en.wikipedia.org/wiki/Test-driven_development

Examples of unit tests of different libraries

Good projects have a policy of never accepting contributions without tests.

Good projects have a policy: if someone reports a bug, add a test to reproduce the bug, then fix it. Bug will not appear again.

Tests can be a lot of work! You can easily write more lines of code to test than to solve the problem sometimes. But if code is designed well, it can be very easy.

Different types of testing

How to test

As scientists, should we, and how should we, use testing in our work?

Discussion time.

Unit testing tools and workflows

unittest / nose

The simplest way to do unit tests is to use nose.

Example:

from nose.tools import assert_true, assert_equal, assert_greater_equal, assert_less

from pcd.support.growsf_gb import *

def test_sole():
    # For small graphs we can exactly specify what the outcome should be:                       
    # alpha=0, delta=0                                                                          
    assert_isomorphic(sole(T=3, alpha=0, delta=0),
                      G({0:(1,2), 1:(0,2)}))

    assert_isomorphic(sole(T=4, alpha=0, delta=0),
                      G({0:(1,2), 1:(0,2), 3:(1,2)}))

http://docs.python.org/library/unittest.html https://nose.readthedocs.org/

* Example bits:

doctests

Example:

def factorial(n):
    """Return the factorial of n, an exact integer >= 0.

    If the result is small enough to fit in an int, return an int.
    Else return a long.

    >>> [factorial(n) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
    >>> [factorial(long(n)) for n in range(6)]
    [1, 1, 2, 6, 24, 120]
    >>> factorial(30)
    265252859812191058636308480000000L
    >>> factorial(30L)
    265252859812191058636308480000000L

https://docs.python.org/2/library/doctest.html

Assertions

Example usage:

Python syntax:

assert test_expression, message
    # test__expression - evaluated, if True then nothing happens, if false raise AssertionError
    # message - only evaluated if expression is False, used as the assertion message.

C syntax:

#include <assert.h>

assert(expression);

Code coverage

Examples:

Thought process behind making test scripts

Benefits from this:

Scientific software testing

"Is this worth it?"

"Is this worth it?" Part 2

Code structure issues

Combinatiorial issues

Stochastic issues

Management issues

What should our standards be for our group library and our code?

Discussion time

Recommendations for now

Summary

My goal is to study feedback from this presentation, and prepare a follow-up that gives specific instructions for how to use this in your work.

What do you want for the next talk?

Please give me feedback and requests.

Resources

Simply doing an internet search for most of these topics will yield plenty of reading and tutorials of all sorts of levels.

Reading list

SoftwareTesting (last edited 2014-04-29 07:57:15 by RichardDarst)