Parameterizing Test Methods - Revisited

In my last post on refactoring and unit tests, Steve Freeman noted in one of his blog comments ..

I can't see the point in the data provider. Why don't you just call the method three times with different arguments?

In the process of refactoring, I had used TestNG's parameterized test method feature to decouple test data from test methods. While invoking a test method multiple times with different parameter gives me the same functional result, parameterized test methods provide great scalability and a neat separation between the test logic and test data suite. Complex business methods may involve lots of complicated calculations and conditionals - a unit test of the method needs to ensure all conditionals are exercised to ensure full coverage. This may result in lots of data combinations, which if handled directly through separate invocations of the same test methods may result in real explosion of test cases.

The feature of parameterizing test methods in TestNG through DataProviders provide a great way to DRY up the test methods. I can have the test data combination either in testNG.xml or in separate factory methods with specific signatures that provide huge extensibility. The test method can then have parameters and an annotation that specify the data source. Take the following example from the earlier post :

public class AccruedInterestCalculatorTest extends MockControl {
  //..
  @DataProvider(name = "test1")
  public Object[][] createAccruedInterestCalculationData() {
    return new Object[][] {
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(1000), BigDecimal.valueOf(30)},
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(60)},
      {30, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(90)},
    };
  }

  //..

  // the test method now accepts parameters
  //..
  @Test(dataProvider = "test1")
  public void normalAccruedInterest(int days,
    BigDecimal rate, BigDecimal principal, BigDecimal interestGold) {
    new CalculatorScaffold()
      .on(principal)
      .forDays(days)
      .at(rate)
      .calculate()
      .andCheck(interestGold);
  }
  //..
}

The calculation of the accrued interest is an extremely complicated business method which needs to consider lots of alternate routes of computation and contextual data. A typical data set for testing the calculation effectively will result in repeating the test method invocation lots of times. Instead of that, the parameterization technique allows me to declare all the varying components as parameters of the test method. And the data source comes from the method createAccruedInterestCalculationData(), suitably annotated with a data source name.

Making the test method parameterized has the additional side-effect of decoupling the data set from the logic - the entire data set can be populated by non-programmers as well, typically domain guys who can enrich the test coverage by simply putting in additional data points in the factory method that generates the data. We have actually used this technique in a real life project where domain experts took part in enriching parameterized unit test cases through data fulfillment in data provider methods.

There's actually more to it. The data provider method can be used to fetch data from other complicated data sources as well e.g. database, XML etc. and with logic to generate data also. All these can be encapsulated in the DataProvider methods and transparently fed to the test methods. The logic of test data generation is completely encapsulated outside the test method. I found this feature of TestNG a real cracker ..