Parameterizing Test Methods - Revisited

In my last post on refactoring and unit tests, Steve Freeman noted in one of his blog comments ..

I can't see the point in the data provider. Why don't you just call the method three times with different arguments?

In the process of refactoring, I had used TestNG's parameterized test method feature to decouple test data from test methods. While invoking a test method multiple times with different parameter gives me the same functional result, parameterized test methods provide great scalability and a neat separation between the test logic and test data suite. Complex business methods may involve lots of complicated calculations and conditionals - a unit test of the method needs to ensure all conditionals are exercised to ensure full coverage. This may result in lots of data combinations, which if handled directly through separate invocations of the same test methods may result in real explosion of test cases.

The feature of parameterizing test methods in TestNG through DataProviders provide a great way to DRY up the test methods. I can have the test data combination either in testNG.xml or in separate factory methods with specific signatures that provide huge extensibility. The test method can then have parameters and an annotation that specify the data source. Take the following example from the earlier post :

public class AccruedInterestCalculatorTest extends MockControl {
  //..
  @DataProvider(name = "test1")
  public Object[][] createAccruedInterestCalculationData() {
    return new Object[][] {
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(1000), BigDecimal.valueOf(30)},
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(60)},
      {30, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(90)},
    };
  }

  //..

  // the test method now accepts parameters
  //..
  @Test(dataProvider = "test1")
  public void normalAccruedInterest(int days,
    BigDecimal rate, BigDecimal principal, BigDecimal interestGold) {
    new CalculatorScaffold()
      .on(principal)
      .forDays(days)
      .at(rate)
      .calculate()
      .andCheck(interestGold);
  }
  //..
}

The calculation of the accrued interest is an extremely complicated business method which needs to consider lots of alternate routes of computation and contextual data. A typical data set for testing the calculation effectively will result in repeating the test method invocation lots of times. Instead of that, the parameterization technique allows me to declare all the varying components as parameters of the test method. And the data source comes from the method createAccruedInterestCalculationData(), suitably annotated with a data source name.

Making the test method parameterized has the additional side-effect of decoupling the data set from the logic - the entire data set can be populated by non-programmers as well, typically domain guys who can enrich the test coverage by simply putting in additional data points in the factory method that generates the data. We have actually used this technique in a real life project where domain experts took part in enriching parameterized unit test cases through data fulfillment in data provider methods.

There's actually more to it. The data provider method can be used to fetch data from other complicated data sources as well e.g. database, XML etc. and with logic to generate data also. All these can be encapsulated in the DataProvider methods and transparently fed to the test methods. The logic of test data generation is completely encapsulated outside the test method. I found this feature of TestNG a real cracker ..

Refactoring Unit Test Methods to speak the Domain

I do TDD, but I do not start writing unit tests before writing production code. It's not that I didn't try - I gave it an honest attempt for quite some time, before I gave up. Somehow the approach of writing tests first does not seem intuitive to me. At least I need to figure out the collaborators and the initial subset of public contracts before my TestNG plugin kicks in. Since then, it's all a game of collaborative pingpong between my production code and test code. Having said that, I honestly believe that an exhaustive unit test suite is no less important than the code itself. I take every pain to ensure that my unit tests are well organized, properly refactored and follow all principles of good programming that I espouse while writing the actual production code.

Refactoring is one practice that I preach, teach and encourage vigorously to my teammates. Same for unit tests - if I strive to make my production code speak the language of the domain, so should be my unit tests. This post is about a similar refactoring exercise to make unit tests speak the DSL. At the end of this effort of vigorous iterations and cycles of refactoring, we could achieve a well engineered unit test suite, which can probably be enhanced by the domain guys with minimum of programming knowledge.

The Class Under Test

The following is the fragment of a simplified version of an AccruedInterestCalculator, a domain class, which calculates the interest accrued for a client over a period of time. For brevity, I have a very simplified version of the class with only very simple domain logic. The idea is to develop a fluent unit test suite for this class that speaks the domain language through an iterative process of refactoring. I will be using the artillery consisting of the state of the art unit testing framework (TestNG), a dynamic mocking framework (EasyMock) and the most powerful weapon in programming - merciless refactoring.

public class AccruedInterestCalculator {
  //..
  //..
  private IAccruedDaysCalculator accruedDaysCalculator;
  private IInterestRateCalculator interestRateCalculator;

  public final BigDecimal calculateAccruedInterest(final BigDecimal principal)
    throws NoInterestAccruedException {
    int days = accruedDaysCalculator.calculateAccruedDays();
    if (days == 0) {
      throw new NoInterestAccruedException("Zero accrual days for principal " + principal);
    }
    BigDecimal rate = interestRateCalculator.calculateRate();
    BigDecimal years = BigDecimal.valueOf(days).divide(BigDecimal.valueOf(365), 2, RoundingMode.UP);
    return principal.multiply(years).multiply(rate);
  }

  AccruedInterestCalculator setAccruedDaysCalculator(IAccruedDaysCalculator accruedDaysCalculator) {
    this.accruedDaysCalculator = accruedDaysCalculator;
    return this;
  }

  AccruedInterestCalculator setInterestRateCalculator(IInterestRateCalculator interestRateCalculator) {
    this.interestRateCalculator = interestRateCalculator;
    return this;
  }
}

and the two collaborators ..

public interface IAccruedDaysCalculator {
  int calculateAccruedDays();
}

public interface IInterestRateCalculator {
  BigDecimal calculateRate();
}

Mocks are useful, but Noisy ..

I am a big fan of using mocks for unit testing - EasyMock is my choice of dynamic mocking framework. Mocks provide the most seamless way of handling collaborators while writing unit tests for a class. However, often, I find mocks introducing lots of boilerplate codes which need to be repeated for every test method that I write ..

public class AccruedInterestCalculatorTest {
  protected IMocksControl mockControl;

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
  }

  @Test
  public void normalAccruedInterestCalculation() {
    IAccruedDaysCalculator acalc = mockControl.createMock(IAccruedDaysCalculator.class);
    IInterestRateCalculator icalc = mockControl.createMock(IInterestRateCalculator.class);

    expect(acalc.calculateAccruedDays()).andReturn(2000);
    expect(icalc.calculateRate()).andReturn(BigDecimal.valueOf(0.5));

    mockControl.replay();

    BigDecimal interest =
      new AccruedInterestCalculator()
        .setAccruedDaysCalculator(acalc)
        .setInterestRateCalculator(icalc)
        .calculateAccruedInterest(BigDecimal.valueOf(2000.00));

    mockControl.verify();
  }
}

Refactoring! Refactoring!

Have a look at the above test method - the mock framework setup and controls pollute the actual business logic that I would like to test. Surely, not a very domain friendly approach. As Howard has pointed to, in one of his NFJS writings, we can abstract away the mock stuff into separate methods within the test class, or still better in a separate MockControl class altogether. This makes the actual test class lighter and free of some of the noise. Here is the snapshot after one round of refactoring ..

// mock controls delegated to class MockControl
public class AccruedInterestCalculatorTest extends MockControl {

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
  }

  @Test
  public void normalAccruedInterestCalculation() {
    IAccruedDaysCalculator acalc = newMock(IAccruedDaysCalculator.class);
    IInterestRateCalculator icalc = newMock(IInterestRateCalculator.class);

    expect(acalc.calculateAccruedDays()).andReturn(2000);
    expect(icalc.calculateRate()).andReturn(BigDecimal.valueOf(0.5));

    replay();

    BigDecimal interest =
      new AccruedInterestCalculator()
        .setAccruedDaysCalculator(acalc)
        .setInterestRateCalculator(icalc)
        .calculateAccruedInterest(BigDecimal.valueOf(2000.00));

    verify();
  }
}

and the new MockControl class ..

public abstract class MockControl {
  protected IMocksControl mockControl;

  protected final <T> T newMock(Class<T> mockClass) {
    return mockControl.createMock(mockClass);
  }

  protected final void replay() {
    mockControl.replay();
  }

  protected final void verify() {
    mockControl.verify();
  }
}

The test class AccruedInterestCalculatorTest is now cleaner and the test method is lighter in baggage from the guts of the mock calls. But still it is not sufficiently close to speaking the domain language. One of the litmus tests which we often do to check the domain friendliness of unit tests is to ask a domain guy to explain the unit test methods (annotated with @Test) and, if possible, to enhance them. This will definitely be a smell to them, with the remaining litterings of mock creation and training still around. The domain guy in this case, nods a big NO, Eclipse kicks in, and we start the next level of refactoring.

One thing strikes me - each test method is a composition of the following steps :

1. create mocks


2. setup mocks


3. replay


4. do the actual stuff


5. verify

And refactoring provides me the ideal way to scaffold all of these behind fluent interfaces for the contract that we plan to test. How about a scaffolding class that encapsulates these steps ? I keep the scaffold as a private inner class and try to localize all the mockeries in one place. And the scaffold can always expose fluent interfaces to be used by the test methods. Here is what we have after a couple of more rounds of iterative refactoring ..

public class AccruedInterestCalculatorTest extends MockControl {
  private IAccruedDaysCalculator acalc;
  private IInterestRateCalculator icalc;

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
    acalc = newMock(IAccruedDaysCalculator.class);
    icalc = newMock(IInterestRateCalculator.class);
  }

  // the scaffold class encapsulating all mock methods
  private class CalculatorScaffold {
    private BigDecimal principal;
    private int days;
    private BigDecimal rate;
    private BigDecimal interest;

    CalculatorScaffold on(BigDecimal principal) {
      this.principal = principal;
      return this;
    }

    CalculatorScaffold forDays(int days) {
      this.days = days;
      return this;
    }

    CalculatorScaffold at(BigDecimal rate) {
      this.rate = rate;
      return this;
    }

    CalculatorScaffold calculate() {
      expect(acalc.calculateAccruedDays()).andReturn(days);
      expect(icalc.calculateRate()).andReturn(rate);

      replay();

      interest =
        new AccruedInterestCalculator()
        .setAccruedDaysCalculator(acalc)
        .setInterestRateCalculator(icalc)
        .calculateAccruedInterest(principal);

      verify();
      return this;
    }

    void andCheck(BigDecimal interestGold) {
      assert interest.compareTo(interestGold) == 0;
    }
  }

  //..
  //.. the actual test methods

  @Test
  public void normalAccruedInterest() {
    new CalculatorScaffold()
      .on(BigDecimal.valueOf(1000.00))
      .forDays(days)
      .at(BigDecimal.valueOf(0.5))
      .calculate()
      .andCheck(BigDecimal.valueOf(30.0));
  }

  @Test
  public void normalAccruedInterest() {
      new CalculatorScaffold()
      .on(BigDecimal.valueOf(2000.00))
      .forDays(days)
      .at(BigDecimal.valueOf(0.5))
      .calculate()
      .andCheck(BigDecimal.valueOf(60.0));
  }

  //..
  //.. other methods
}

This class has test methods which now look more closer to the domain, with all mock controls refactored away to the scaffold class. Do you think the domain guys will be able to add more methods to add to the richness of coverage ? The accrued interest calculation is actually quite complicated with more collaborators and more domain logic than what I have painted here. Hence it is quite natural that we need to enrich the test cases with more possibilities and test coverage. Instead of making separate methods which work on various combinations of days, rate and principal (and other factors in reality), why not DRY them up with parameterized tests ?

Parameterized Tests in TestNG

TestNG provides a great feature for providing parameters in test methods using the DataProvider annotation. DRY up your test methods with parameters .. here is what it looks like in our case ..

public class AccruedInterestCalculatorTest extends MockControl {
  private IAccruedDaysCalculator acalc;
  private IInterestRateCalculator icalc;

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
    acalc = newMock(IAccruedDaysCalculator.class);
    icalc = newMock(IInterestRateCalculator.class);
  }

  @DataProvider(name = "test1")
  public Object[][] createAccruedInterestCalculationData() {
    return new Object[][] {
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(1000), BigDecimal.valueOf(30)},
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(60)},
      {30, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(90)},
    };
  }

  private class CalculatorScaffold {
    //..
    //.. same as above
  }

  // the test method now accepts parameters
  //..
  @Test(dataProvider = "test1")
  public void normalAccruedInterest(int days,
    BigDecimal rate, BigDecimal principal, BigDecimal interestGold) {
    new CalculatorScaffold()
      .on(principal)
      .forDays(days)
      .at(rate)
      .calculate()
      .andCheck(interestGold);
  }

  @Test(expectedExceptions = {NoInterestAccruedException.class})
  public void zeroAccruedDays() {
    new CalculatorScaffold()
      .on(BigDecimal.valueOf(2000))
      .forDays(0)
      .at(BigDecimal.valueOf(0.5))
      .calculate()
      .andCheck(BigDecimal.valueOf(90));
  }
}

The test methods look DRY, speak the domain language and do no longer have to administer the mock controls. All test conditions and results are now completely decoupled from the test methods and fed to them through the annotated provider methods. Now the domain guy is happy!

XML - Not for Human Consumption

I have never been a big fan of using XML as a language for human interaction. I always felt that XML is too noisy for human comprehension and never a pleasing sight for your eyes. In an earlier post I had pitched in with the idea of using Lisp s-expressions as executable XML instead of the plethora of angle brackets polluting your comprehension power. XML is meant for machine interpretation, the hierarchical processing of XML documents gives you the raw power when you are programming for the machine. But, after all, Programs must be written for people to read, and only incidentally for machines to execute (from SICP).

With the emergence of the modern stream of scripting languages, XML is definitely taking a backseat in handling issues like software builds and configuration management. People are no longer amused with the XML hell of Maven and have been desperately looking for alternatives. Buildr, a drop-in replacement for Maven uses Ruby (inspired by Rake), SCons uses Python scripts, Groovy has also been used in the configuration space for Jini-based ComputeCycles project (via Artima). XML has also started looking like just yet another option for configuring your beans in the Spring community. We are just spoilt with other options of configuration - Spring JavaConfig uses Java configuration, Springy does it with JRuby, Groovy SpringBuilder does it with Groovy.

Configuration and builds of systems are nontrivial activities and need a mix of both declarative and procedural programming. Using XML as the brute force approach towards this has resulted in complex hierarchical structures, too complex for human comprehension. A perfect example in hand is Maven which attempts to do too many things with XML. The result is extremely complex XML hell much to the anguish of developers and build managers. A big build system configured using Maven becomes a maintenance nightmare in no time.

People are getting increasingly used to the comforts of DSLs and configurations of large systems have never been too trivial an artifact. Hence it is logical that we would like to have something which provides a cool humane interface. And XML is definitely not one of them ..

Unit Tests - Another Great Tool for Refactoring

In my last post on unit testing, I had talked about some techniques on how to make your classes unit test friendly. Unit testing is the bedrock of writing good software - never compromise on writing unit tests for the next class that you design. Unit testing is about testing classes in isolation - get them out of the natural environment and take them for a ride. That way unit tests give you a feedback about the correctness of the behavior of the *unit* as a whole. The other aspect of unit tests which I find extremely useful while designing classes is the support for refactoring. Unit tests are great vehicles for refactoring object collaborations - thus there is a 2-way feedback loop between class design and its unit test design. And once you have cycled through rounds of iteration in this feedback loop, you can expect to have stabilized your abstractions and collaborations.

Here's an example ..

class Payroll {
  // ..
  // ..

  void calculateSalary(Employee emp) {
    BigDecimal charge = emp.getSalary();
    // calculate total salary including taxes
  }
}

When I look at the possible unit test for the class Payroll, the first thing that strikes me is the argument Employee in calculateSalary(). The first attempt tries to instantiate an Employee and invoke the method calculateSalary() from the test case :

class PayrollTest extends TestCase {
  // ..
  // ..

  public void testSalaryCalculation() {
    Employee emp = new Employee(..);
    // ..
  }
}

A Mockery of sorts ..

Instantiating an Employee object brings into action lots of other machineries like the database layer, the ORM layer etc., which we already know should be best kept away from the unit test harness. Think Mocks ..

When you need to mock a concrete class, take a second look, if necessary take a third look as well. Mock roles, not objects and roles are best specified through interfaces. In the above example, we are really exploiting the fact that an employee is salaried and has a basic salary component for being employed in a company. In this method calculateSalary(), we are not concerned with the other details of the Employee class, which may be used in other context within the software.

Extract Interface

Now I have a new interface Salaried as :

interface Salaried {
  BigDecimal getBasicSalary();
}

and ..

public class Employee implements Salaried {
  // ..
}

The method calculateSalary() now takes an interface Salaried instead of the concrete class Employee - and the mock becomes idiomatic in the test case (I am using EasyMock as the mocking framework) :

class Payroll {
  // ..
  // ..

  public void calculateSalary(Salaried emp) {
    // ..
  }
}

class PayrollTest extends TestCase {
  // ..
  // ..

  public void testSalaryCalculation() {
    IBillable mock = createMock(Salaried .class);
    expect(mock.getBasicSalary()).andReturn(BigDecimal.valueOf(7000));
    replay(mock);
    // ..
  }
}

Thus unit testing helped me use the Extract Interface principle and improve the collaboration between my classes. Notice how the class Payroll now collaborates with a more generic role Salaried instead of a concrete class Employee. Tomorrow if the company decides to employ contractors with different salary computation, we will have a separate abstraction :

public class Contractor implements Salaried {
  // ..
  // ..
  BigDecimal getBasicSalary() {
    // .. different implementation than Employee
  }
  // ..
}

but the class Payroll remains unaffected, since it now collaborates with Salaried and NOT any concrete implementation.