Testing Tool Tour : QuickNet Preview

---

Update 24/12/2009

The information (especially syntax) in this post is quite out of date, please visit the projects home page on google code for links to up to date examples or use the QuickNet category link on this blog.
The principles in the article are still valid, although these days I would probably rewrite the multply spec like so :

new Spec(() => Ensure.Equal( input.First, output / input.Second )

in order to avoid the ‘you’re duplicating business logic’-argument.
I’ve gotten better at coming up with specs lately .

---

Recently I had the enjoyable experience of working on a dot net 3.5 project, instead of 2.0, as was the case for the previous year and a half. As a result I’ve been coding quite a bit lately, not downloading IDE’s and testers in order to take them for a spin.
I was planning on taking the more historically correct road and tour JUnit next. But in view of these recent developments, I opted for this QuickNet tour instead.

QuickNet is my poor man’s c# implementation of a property based tester for dot net 3.5.
I tend to call it property based as I’ve seen John Hughes’s QuviQ been described as such, and that, and QuickCheck (from the Haskell language) are what inspired QuickNet. Although inspired might be an overstatement. I just really quickly needed a better way to organize my unit/integration tests as they were getting too hard to maintain. The ideas from above mentioned testing tools seemed just what I needed.

Taking it for a spin : (using the cannonical calculator example)
You need dot net 3.5 and a MS Visual Studio, which pretty much means : get a job in the computer industry, and then you need wait for an alpha release of QuickNet .

Our first Calculator C# implementation :

public class Calculator
{
    public int Multiply(int a, int b)
    {
        return 6;
    }
}

And the first tests :

[TestFixture]
public class TheThirdFixture
{
    [Test]
    public void UnitTestCalculatorMultiply()
    {
        Assert.AreEqual(6, new Calculator().Multiply(3,2));
        Assert.AreEqual(15, new Calculator().Multiply(5,3));
    }

    [Test]
    public void CalculatorTest()
    {
        IntGenerator gen = new IntGenerator(0, 200);
        new TestRun()
            .AddTransition(new Transition<Input<int, int>, int>()
            {
                GenerateInput = 
                    () => new Input 
                                { 
                                    ParamOne = gen.Value, 
                                    ParamTwo = gen.Value 
                                },
                Execute = 
                    input => new Calculator().Multiply(input.ParamOne, input.ParamTwo))
            }
            .RegisterProperty(( input, output ) => new Property(() =>
                QnAssert.AreEqual( input.ParamOne * input.ParamTwo, output ))))
            .Verify()
            .ThrowTestFailedExceptionIfAnyOnePropertyFailed()
            .ReportPropertiesTested(new ConsoleReporter());
        }
    }
}

I included an NUnit test with two asserts. The first one will pass. The second assert fails as our implementation is buggy.

The QuickNet test also fails. Let’s see how it’s defined.
The local IntGenerator variable is an instance of a QuickNet helper class that gets a random int.
Next we create a TestRun, to which we add the following Transition.

new Transition<Input<int, int>, int>()

The types were passing in here are a type definition of Input and Output members of the transition.
The GenerateInput (lambda) member of the transition uses the IntGenerator to obtain two random ints.
The Execute (lambda) member of the transition uses the values from GenerateInput, calls multiply on the calculator and returns an int result.
We then register a property on this transition. A property receives a transition’s input and output and asserts something. This particular property states the the output of the transition should equal the first parameter of it’s input multiplied by the second parameter of it’s input.
We then call Verify on the TestRun to assert all our registered properties, call ThrowTestFailedExceptionIfAnyOnePropertyFailed to make the NUnit test fail if QuickCheck fails, and in case that it passes we report some run details to the console.

We correct our buggy method to correctly return a * b and both tests pass.

A Trickier Bug
Even though QuickNet originated out of a need for less code, some of you might have noticed that the QuickNet test is quite verbose compared to the NUnit test. So what’s the point ?
Suppose some work get’s done on our calculator and the following is implemented :

public class Calculator
{
    private int count = 0;
    public int Multiply(int a, int b)
    {
        count++;
        if(count == 10)
            throw new Exception()
        return a * b;
    }
}

Both tests will still pass.
At some point an unfortunate user of this code tries multiply ten times, reports the bug and we modify our tests to demonstrate it.
For the NUnit part : add the following test :

[Test]
public void UnitTestCalculatorMultiplyManyTimes()
{
    Calculator calculator = new Calculator();
    for(int i = 0; i < 100; i++)
    {
          Assert.AreEqual(6, calculator.Multiply(3,2));
    }
}

For the QuickNet test :
Declare a local Calculator below the IntGenerator and instantiate it.
Modify the lambda to use this local instance instead of calling new.
Just run it a hundred times or more by changing the call to the testrun constructor to

new TestRun(1, 100) 

Both test will now fail. We had to add an NUnit test to find the bug. QuickNet had to be tweaked.

Even Trickier
Now we need to implement Divide.
For NUnit we add the following test :

[Test]
public void UnitTestCalculatorDivide()
{
    Calculator calculator = new Calculator();
    Assert.AreEqual(3, calculator.Divide(6,2));
    Assert.AreEqual(4, calculator.Divide(16,4));
}

The QuickNet test needs a new Transition for this. The full test looks like this :

[TestFixture]
public class TheThirdFixture
{
    [Test]
    public void CalculatorTest()
    {
        Calculator calculator = new Calculator();
        CalculatorInputGenerator gen = new CalculatorInputGenerator();
        new TestRun(1,100)
            .AddTransition(new Transition<Input<int, int>, int>()
            {
                GenerateInput = () => gen.Value,
                Execute = input => calculator.Multiply(input.ParamOne, input.ParamTwo))
            }
            .RegisterProperty(( input, output ) => new Property(() =>
                QnAssert.AreEqual( input.ParamOne * input.ParamTwo, output ))))
            .AddTransition(new Transition<Input<int, int>, int>()
            {
                GenerateInput = () => gen.Value,
                Execute = input => calculator.Divide(input.ParamOne, input.ParamTwo))
            }
            .RegisterProperty(( input, output ) => new Property(() =>
                QnAssert.AreEqual( input.ParamOne / input.ParamTwo, output ))))
            .Verify()
            .ThrowTestFailedExceptionIfAnyOnePropertyFailed()
            .ReportPropertiesTested(new ConsoleReporter());
        }
    }
}

Now someone implemented this as such :

public class Calculator
{
    private string bug = "";
    public int Multiply(int a, int b)
    {
        bug += "1";
        if(bug == "112121")
           throw new Exception();
        return a * b;
    }
    public int Divide(int a, int b)
    {
        bug += "2";
        return a / b;
    }
}

The NUnit tests will pass. QuickNet fails.
For NUnit :
Add a test.

[Test]
public void UnitTestCalculatorDivide()
{
    Calculator calculator = new Calculator();
    Assert.AreEqual(6, calculator.Multiply(3,2));
    Assert.AreEqual(6, calculator.Multiply(3,2));
    Assert.AreEqual(3, calculator.Divide(6,2));
    Assert.AreEqual(6, calculator.Multiply(3,2));
    Assert.AreEqual(4, calculator.Divide(16,4));
    Assert.AreEqual(6, calculator.Multiply(3,2));
}

Fix the bug.

For QuickNet : Just fix the bug.

Bonus Bugs
As I was working out the simple example, QuickNet pointed out to me that division by zero throws an exception. Just setting the TestRun constructor to (1, 5000) fails it every time. In NUnit we would need another test. Quicknet needs a new property, and a precondition on the existing one.

About these ads