A short while ago Davy Brion posted the first Agatha QuickNet tests.
The first version was the initial attempt, the second a refactoring. There’s a small difference in philosophy between both versions.
I was quite curious about the comments on these pieces of code for obvious reasons, but not a lot of them were forthcoming.
One of the main reasons for this is IMO this one :
Too much magic for my taste. I don’t understand any of those tests / specifications.
So let’s remedy that and walk you through another Agatha test.
The RequestProcessor test.
It asserts the way Agatha’s default serverside IRequestProcessor implementation deals with exceptions that could be thrown by the IRequestHandler implementations it calls.
Setup and context for the Agatha tests are quite complicated and QuickNet could surely do with a bit of work on this issue, but here goes :
First I coded some concrete implementations of a Request and Response and declared a handler interface.
public class FirstRequest : Request { }
public class FirstResponse : Response { }
private static IRequestHandler<FirstRequest> handlerForFirstRequest;
There’s actually three of these in the code. A way to make this more dynamic needs to be devised. This probably means adding an assembly to the solution specifically for containing test data.
private static IRequestProcessor requestProcessor;
private static ServiceLayerConfiguration serviceLayerConfiguration;
The IRequestProcessor is our system under test and it takes a ServiceLayerConfiguration in it’s constructor. The RequestProcessor uses the serviceLayerConfiguration to determine which type of exception is the Business exception and which type of exception is the Security exception.
public RequestProcessorSpecs() : base(10, 20) { }
The test’s constructor. This means it will run two hundred tests. In QuickNet terms : ten tests, containing twenty transitions each. As you’ll see later there’s only one transition ‘RequestProcessor.Process()’ in this fixture. So the
alchemist calls the Setup function (see below) and then executes the transition twenty times. It then calls setup again, reinitializing state, etc… It does this ten times as defined by the first int parameter in the constructor.
public override void SetUp()
{
IoC.Container = new Agatha.Castle.Container();
serviceLayerConfiguration = MockRepository.GenerateStub<ServiceLayerConfiguration>(null, null, IoC.Container);
serviceLayerConfiguration.BusinessExceptionType = typeof (BusinessException);
serviceLayerConfiguration.SecurityExceptionType = typeof(SecurityException);
requestProcessor = new RequestProcessor(serviceLayerConfiguration);
handlerForFirstRequest = MockRepository.GenerateMock<IRequestHandler<FirstRequest>>();
handlerForSecondRequest = MockRepository.GenerateMock<IRequestHandler<SecondRequest>>();
handlerForThirdRequest = MockRepository.GenerateMock<IRequestHandler<ThirdRequest>>();
IoC.Container.RegisterInstance(handlerForFirstRequest);
IoC.Container.RegisterInstance(handlerForSecondRequest);
IoC.Container.RegisterInstance(handlerForThirdRequest);
}
The test’s setup function initializes an IoC container as the RequestDispatcher uses this to resolve the RequestHandlers, the mocked RequestHandlers are then registered. ServiceLayerConfiguration is stubbed and we set BusinessExceptionType and SecurityExceptionType.
And ofcourse we instantiate the RequestProcessor.
public class BusinessException : Exception { }
public class SecurityException : Exception { }
public class UnknownException : Exception { }
public class AnotherUnknownException : Exception { }
private static List<Exception> exceptionsThrown;
Some context. We define some exceptions, two of which have been used in test setup, and we declare a list which we’ll use to store state.
Next there’s a bit of complicated code needed to define input, which needs to be refactored and will probably look a lot simpler if we go for above mentioned strategy of defining test data in another assembly. I’ll spare you that for now 
. I was having fun writing it though.
The important bit is the following :
IGenerator<Action<IList<Exception>>> actionGenerator =
new ChoiceGenerator<Action<IList<Exception>>>(
new Action<IList<Exception>>[]
{
list => list.Add(null),
list =>
{
Exception exception = new BusinessException();
list.Add(exception);
throw exception;
},
list =>
{
Exception exception = new SecurityException();
list.Add(exception);
throw exception;
},
list =>
{
Exception exception = new UnknownException();
list.Add(exception);
throw exception;
},
list =>
{
Exception exception = new AnotherUnknownException();
list.Add(exception);
throw exception;
}
});
This generator randomly chooses one of the functions defined in the list so that the transition can plug it into the RequestHandler when it’s Handle function is called. As you can see there’s one that does nothing, one that throws an exception that is defined in the ServiceLayerConfiguration as a business exception, one that throws an exception that is defined in the ServiceLayerConfiguration as a security exception, and two that throw an unknown exception. These functions store the exception they throw (or null) in the test’s ‘context’, the exceptionsThrown list.
class Process : MetaTransition<ProcessInput, Response[]>
{
public Process()
{
Generator = new ProcessInputGenerator();
Execute =
input =>
{
int ix = 0;
exceptionsThrown = new List<Exception>();
foreach (Type key in input.RequestsAndHandlers.Keys)
{
input.RequestsAndHandlers[key].StubIt(input.Actions[ix], exceptionsThrown);
ix++;
}
return requestProcessor.Process(input.Requests);
};
}
}
The transition, our function under test. The ProcessInputGenerator is responsible for jumbling the handler functions randomly into a list of RequestsAndHandlers which are part of the ProcessInput class. Before running the transition we clear the exceptionsThrown list and set the Rhino.Mocks stubs for each of the handler mocks. Here’s the function that does the latter and is part of afore-mentioned complicated bit :
public void StubIt(Action<IList<Exception>> action, IList<Exception> exceptionsThrown)
{
RequestHandler
.Stub<IRequestHandler>(r => r.Handle(Request))
.Return(Response)
.WhenCalled(arg => action(exceptionsThrown))
.Repeat.Once();
RequestHandler
.Stub<IRequestHandler>(r => r.CreateDefaultResponse())
.Return(Response)
.Repeat.Once();
}
Now with all that setup we can finally start defining specs.
Here’s the first one, in case no exception is thrown by any of the handlers :
[SpecFor(typeof(Process))]
public Spec ProcessRequestsWithoutException(ProcessInput input, Response[] output)
{
return new Spec(
() => output.ForEach(
r => Ensure.Equal(ExceptionType.None, r.ExceptionType)))
.IfAfter(() => !exceptionsThrown.Exists(e => e != null));
}
The IfAfter part is a ‘PostCondition’. This means the specs ‘Invariant’, defined in the constructor, is only evaluated if the postcondition holds. In this case it says that the exceptionsThrown list should only contain null values.
The invariant then
Ensures that all Response instances in the output have their ExceptionType correctly set to ExceptionType.None.
[SpecFor(typeof(Process))]
public Spec ProcessRequestsWithBusinessException(ProcessInput input, Response[] output)
{
Predicate<Exception> predicate =
exception => exception != null && exception.GetType() == typeof(BusinessException);
return new Spec(
() =>
{
int ix = exceptionsThrown.FindIndex(predicate);
Ensure.Equal(ExceptionType.Business, output[ix].ExceptionType);
Ensure.Equal(exceptionsThrown[ix].Message, output[ix].Exception.Message);
Ensure.Equal(exceptionsThrown[ix].StackTrace, output[ix].Exception.StackTrace);
Ensure.Equal("Tests.RequestProcessorSpecs+BusinessException", output[ix].Exception.Type);
})
.IfAfter(() => exceptionsThrown.Exists(predicate));
}
This spec only fires if any Handler threw a BusinessException, looks up which one was the first one and Ensures some stuff on the corresponding Response.
There’s a similar spec for a Security Exception and another two for the ‘unknown’ exceptions.
Finally here’s the spec that ensures that all responses after the failing one have ExceptionType EarlierRequestAlreadyFailed.
[SpecFor(typeof(Process))]
public Spec ProcessRequestsWithExceptionFailsAllFollowingRequests(ProcessInput input, Response[] output)
{
Predicate<Exception> predicate =
exception => exception != null;
return new Spec(
() =>
{
int ix = exceptionsThrown.FindIndex(predicate) + 1;
for (int i = ix; i < output.Length; i++)
{
Ensure.Equal(ExceptionType.EarlierRequestAlreadyFailed, output[ix].ExceptionType);
Ensure.Equal("EarlierRequestAlreadyFailed", output[ix].Exception.Message);
Ensure.Null(output[ix].Exception.StackTrace);
Ensure.Equal("System.Exception", output[ix].Exception.Type);
}
})
.IfAfter(
() => exceptionsThrown.FindIndex(predicate) >= 0
&& exceptionsThrown.FindIndex(predicate) >= exceptionsThrown.Count - 1);
// excluding cases where only last request fails
// as this wouldn't ensure anything but the spec would have been considered to hold
}
Conclusion :
I agree it’s not the easiest code to read (or write ), and that it might look like a lot of work, but there’s a bonus : Every possible combination of exceptions thrown is tested. I don’t have to think about all the possibillities, QuickNet does that. Now Agatha is pretty stateless, but even then, not completely, during a request (or batch thereof) there is state kept in the objects. Which is why in the second version of the tests in Davy’s post I modelled the Response.Error and Response.Result properties as different transitions. If there are bugs arising from keeping state in an incorrect fashion they will surface.
Also QuickNet specs are conclusive, unit tests aren’t. If I define a spec and there is a case in which it doesn’t hold, QuickNet will tell me and I would have to add a pre- or postcondition to make the spec pass. Makes things far more explicit, even though context and setup is harder to read.
And for you Code Coverage Fanatics out there, this type of testing easily gets that stat up to a hundred percent whereas using the unit testing approach I usually have to duplicate a lot of test code in order to get there.
