Monday, July 30, 2007

Configurable Decorators using AOP

Sometime back I had posted on the increasing misuse (or overuse) of aspects by today's developers. In many of those cases, plain old decorators would have just been fine. However, very recently, I used a combination of decorators and aspects to add another degree of configurability to a domain model.

Decorators, by design, should be lightweight objects which can be attached to the skin of other domain objects to add to the roles and responsibilities of the latter. They are added and/or removed dynamically from original objects - hence we say that decorators offer an added level of flexibility over static inheritance. Consider the following aggregate, which is modeled as a Composite :

Here is the base abstraction for the Composite, which models the Salary components of an employee :


public abstract class Salary {
  public Salary add(final Salary component) {
    throw new UnsupportedOperationException("not supported at this level");
  }

  public Salary remove(final Salary component) {
    throw new UnsupportedOperationException("not supported at this level");
  }

  public Salary getChild(final int i) {
    throw new UnsupportedOperationException("not supported at this level");
  }

  public abstract double calculate();
}



and here is the composite ..


public class SalaryComposite extends Salary {
  private List<Salary> components = new ArrayList<Salary>();

  @Override
  public Salary add(Salary component) {
    components.add(component);
    return this;
  }

  @Override
  public double calculate() {
    double total = 0;
    for(Salary salary : components) {
      total += salary.calculate();
    }
    return total;
  }

  @Override
  public Salary getChild(int i) {
    return components.get(i);
  }

  @Override
  public Salary remove(Salary component) {
    components.remove(component);
    return this;
  }
}



and a couple of leaf level components ..


public class BasicPay extends Salary {
  private double basicPay;

  public BasicPay(double basicPay) {
    this.basicPay = basicPay;
  }

  @Override
  public double calculate() {
    return basicPay;
  }
}

public class HouseRentAllowance extends Salary {
  private double amount;

  public HouseRentAllowance(double amount) {
    this.amount = amount;
  }

  @Override
  public double calculate() {
    return amount;
  }
}



Decorators and Composites - a Marriage

Decorators work nicely with Composites, and I have some decorators to allow users add behaviors to the Composite elements dynamically. In order to have decorators for the Composite, I need to have the Decorator share the base class with the Composite.


public abstract class SalaryDecorator extends Salary {
  private Salary component;

  public SalaryDecorator(final Salary component) {
    this.component = component;
  }

  @Override
  public double calculate() {
    return component.calculate();
  }
}



and a couple of concrete decorators ..

a senior citizen adjustment which can be applied to some of the components of the salary ..


public class SeniorCitizenAdjustment extends SalaryDecorator {
  protected double adjustmentFactor;
  public SeniorCitizenAdjustment(final Salary component, double adjustmentFactor) {
    super(component);
    this.adjustmentFactor = adjustmentFactor;
  }

  @Override
  public double calculate() {
    //.. calculation of absolute amount based on adjustmentFactor
    //.. complex logic
    adjustment = ...
    return super.calculate() + adjustment;
  }
}



and a city compensatory allowance which varies based on the city where the employee leaves ..


public class CityCompensatoryAdjustment extends SalaryDecorator {
  private double adjustmentFactor;

  public CityCompensatoryAdjustment(final Salary component, double adjustmentFactor) {
    super(component);
    this.adjustmentFactor = adjustmentFactor;
  }

  @Override
  public double calculate() {
    //.. calculation of absolute amount based on adjustmentFactor
    //.. complex logic
    adjustment = ...
    return super.calculate() + adjustment;
  }
}



Now my clients can make use of the Composite and decorate them using the decorators designed to compose the individual salary components ..


//..
Salary s = new SalaryComposite();
s.add(new SeniorCitizenAdjustment(new BasicPay(1000), 100))
 .add(new CityCompensatoryAdjustment(new HouseRentAllowance(300), 150));
//..



In the above example, the use of decorators provide the flexibility that the particular design pattern promises and allows instance level customization of individual components of the composite.

Decorating the Decorators

How do you handle fine grained variations within decorators ? In our case many of the decorators had fine grained variations within themselves, which calls for either inheritance hierarchies between them or some other way to decorate the decorators (super-decorators ?). The problem with static inheritance hierarchies is well-known. They have to be defined during compile time and the codebase for each deployment need to have the explosion of all possible variations modeled as subclasses. Looks like the problem that decorators are supposed to solve and that which we have already solved for the Composite Salary aggregate. Now we have the same problem for the decorators themselves.

We rejected the idea of static inheritance hierarchies and decided to model the variations as yet another level of decorators. We did not want to make the original decorators too complex and focused on making small composable, additive abstractions that can be tagged on transparently onto the domain objects and decorators alike.

e.g. in an implementation we found that the CityCompensatoryAdjustment has another fine grained variation. When CityCompensatoryAdjustment is applied, we have the additional variation that states that if the employee's residence is in one of a group of premium cities, then he is eligible for an additional premium allowance on top of the normal CityCompensatoryAdjustment. We rejected the idea of changing CityCompensatoryAdjustment to incorporate the new variations for 2 reasons :

  1. Trying to incorporate too much logic into decorators will make them complicated, which defeats the basic design motivation for the decorators

  2. These variations change across deployments - hence it will be a bloat trying to incorporate everything into a single class


Modeling the new requirement as another decorator (PremiumCityAdjustment), we have the following usage of the above snippet ..


//..
Salary s = new SalaryComposite();
s.add(new SeniorCitizenAdjustment(new BasicPay(1000), 100))
 .add(new PremiumCityAdjustment(
          new CityCompensatoryAdjustment(
              new HouseRentAllowance(300), 150),
          adjustmentFactor));
//..



But the Variations are Deployment specific!

We cannot change the main codebase with the above fine-grained variations, since they are deployment specific. We need to find out ways to externalize the invocation of these decorators. We found out that these finer variations are policies that need to change the behavior of the original decorators whenever they are applied. In the above example, every invocation of CityCompensatoryAdjustment needs to be decorated with PremiumCityAdjustment.


public class PremiumCityAdjustment extends SalaryDecorator {
  public PremiumCityAdjustment(final Salary component) {
    super(component);
  }

  @Override
  public double calculate() {
    //.. calculation of adjustment amount
    adjustment = ..
    return super.calculate() + adjustment;
  }
}



Aspects again !

We decided to implement the finer grained variations as decorators which would be applied through aspects to the original decorators. This is the base aspect for all such SuperDecorators ..


public abstract aspect SalarySuperDecoratorBase<extends SalaryDecorator> {

  pointcut decoratorCalculate(T s) : target(s)
        && execution(double T.calculate());

  abstract pointcut myAdvice();
}



and here is the implementation for adding PremiumCityAdjustment as a decorator for CityCompensatoryAdjustment ..


public aspect SuperDecoratorCityCompensatoryAdjustment
    extends SalarySuperDecoratorBase<CityCompensatoryAdjustment> {

  pointcut myAdvice(): adviceexecution()
    && within(SuperDecoratorCityCompensatoryAdjustment);

  Object around(CityCompensatoryAdjustment s) : decoratorCalculate(s) && !cflow(myAdvice()) {
    return new PremiumCityAdjustment(s).calculate();
  }
}



This way we had a set of core decorators for the main domain model and another set of deployment specific decorators which decorated the core ones through aspect weaving. By adopting this strategy we were able to keep the decorators lightweight, avoided deep inheritance hierarchies and managed to keep customizable code base completely separate from the core one.

Tuesday, July 24, 2007

Why API Design Matters

So, why are there so many bad APIs around? The prime reason is that, for every way to design an API correctly, there are usually dozens of ways to design it incorrectly.

Good API design is hard, while bad APIs are easy. Michi Henning's API Design Matters in May/June issue of ACM Queue is a comprehensive guide to designing good APIs.

He dissects the .NET socket Select() function in C# and points out the deficiencies which many of our day-to-day usable frameworks have. The article also discusses the interplay of abstraction hierarchies with good api design.
Much of software development is about creating abstractions, and APIs are the visible interfaces to these abstractions. .. The lower in the abstraction hierarchy an API defect occurs, the more serious are the consequences.

And finally, the most important aspect of API design is to manage the expectations of the closure of users who will be exercising them. And once you have published your API, it becomes immutable. Can't agree more ..

A very good read ..

Monday, July 23, 2007

Customizable Business Rules using scripting on the Java Platform

Executing Javascript within the JVM has opened up a new avenue to add to your architectural paradigms. In today's enterprise applications lots of code (validations etc.) needs to be run both for the client and the server. Often we find repetitions in the form of Javascript for the client side code and Java for the server side. With server side Javascript being thrown up as an option in the JVM, the same client side code can be executed within the server using the ScriptEngine of Mustang. Rails people will suggest writing DSLs, which can be converted to Javascript and executed within the JVM.

This post is about using the new architectural paradigm in a project which we executed recently for a publishing house. It was a medium sized project where lots of stuff used to be run in VBScript within ASP.NET. The hapless users wanted to migrate to the JVM and sync up the application with the rest of their MIS running on big Java platforms. No additional functionality was needed to be developed. The VBScript code generates lots of SQL using complex business logic, which ran in the .NET server platforms.

Problem #1 : We do not know what the VBScript code does!

This was the first shocker that we received when visiting our client. The developers who had written the code have left - does this ring any bell? The usual problem of migration - and none of the existing MIS managers where prepared to explain us the flow of the business logic as they exist today.

Option #1 : Go dig it out ..

Struggle through the reams of VBScript code (no code comments - we are dynamic!) and enlighten yourself with the existing business process flow. The only thing that we realized was that all those VBScripts were being used to prepare tons of SQL queries that were later executed in the server. And that the logic code had the usual share of complicated if-then-else that sets up the SQL statements.

Problem #2 : We want externalization of business rules that exists today

The users today have the flexibility of manually tweaking SQL queries and minor business logic in the customary VBScript style of operations. And since some of the business rules change quite frequently, they need to have the same flexibility in the new system as well.

Wow! Rhino!

After shuddering in the thought of cost overrun that Option #1 would incur, we decided to look for an alternative. And there it was - the new scripting engine available for executing Javascript within the JVM. Without going into the details of the business logic, we took the route of converting the VBScript code to Javascript and then spitting out the result to the Rhino interpreter. This will also solve Problem #2 as well, since the same business logic will be available to the users today, only on a different scripting platform. The conversion process was made easier by googling some tools crafted by one generous soul. We wrote a script handler ourselves that we could invoke and execute all scripts from within the main Java application ..


boolean loadScriptFile(String fileName) {
  String jsFile = fileName + "." + scriptBean.getScriptExtension();
  InputStream is;
  Reader reader;

  try {
    is = this.getClass().getResourceAsStream(
      scriptBean.getScriptingSourcePath() + jsFile);
    reader = new InputStreamReader(is);
    engine.eval(reader);
    //..
    //..
  } catch (ScriptException e) {
    //.. handle
  }
  //..
  //..
}



The script handler had an initialization code that set the environment up for all executions to take place in the later stages .. we had developed a utility library which also needed to be bootstrapped ..


private void init() {
  ScriptEngineManager engineMgr =
      new ScriptEngineManager();
  ScriptEngine engine =
      engineMgr.getEngineByName(scriptBean.getScriptingEngine());

  try {
    // execute utility library
    InputStream is = this.getClass().getResourceAsStream(
      scriptBean.getScriptingSourcePath() + "defaultScripts.js");
    engine.eval(new InputStreamReader(is));

    // set up values in the state of the ScriptEngine
    engine.put(SCRIPT_RESULT_IDENTIFIER, engineResultList);
    engine.put(SCRIPT_MAP_RESULT_IDENTIFIER, engineResultMap);

  } catch (ScriptException e) {
    //.. handle
  }
}



In the above example snippet, the put() method allows assigning any variable used in the scripting environment from within Java code. Another alternative will be to use Bindings. Suppose we have a list of names in a java variable listOfNames, which we need to provide to a script to form an SQL statement ..


engine.put("namesKey", listOfNames);



and the script to be loaded contains the following ..


function performancePubReg() {
  //.. form the sql
  sqlStr = "select ...
  sqlStr = sqlStr + "where master.name in " + makeString(namesKey);
  //..
}



In course of the application, wherever we needed to execute queries, these sql queries were fetched by load-eval of Javascripts using the script handler implementation.

A Useful Extension to the Architectural Toolbox

In general, this approach proved to be a useful paradigm for small to medium sized migration projects. In an ideal scenario, these customizable business rules which generate SQL queries could be coded up as a DSL for the business users and then appropriately converted to any scripting platform. And the best part is that you need not always dig into the details of legacy code in situations like ours.

Monday, July 16, 2007

Domain Driven Design : Handling Alternate Business Scenarios

In one of my earlier posts, I had discussed about domain driven exceptions and how alternate business flows can be modeled using checked exceptions of Java. Recently there has been a thread going on in domain-driven-design discussion forum on a similar subject on the rationale of throwing in case of violations of business rules.

Business Rule Violation - violates what ?

Typically when a business rule is violated, the domain has a set of alternative rules, which kicks in to handle the situation. It's not that we have a fatal exception that needs to fall flat on the user's face. When I have an account.debit(amount) failing the balance check, there is an alternate path that the domain prescribes and which needs to be modeled as part of the implementation. So, instead of violation, I prefer to use the term alternate business scenario, since it's always choosing between one of the many pathways through which the model flows in course of the domain lifecycle. One of them is the most commonly executed path, while the others may be less frequently traversed, but very much a part of the domain logic.

In the last post, I had talked about Java's checked exceptions as a possible means to handle what I call alternate business scenarios. Many people has, since then, suggested the option of using the Specification pattern of domain driven design to handle alternate execution paths in the domain model. In this post, let us see how much we can improve using the Specification pattern both with respect to encapsulating the business constraints and also handling alternate sequence of execution on violation of those constraints.

Using the Specification pattern to handle alternate business scenarios

I take the example from my previous post, modified slightly to reflect the current understanding.

The class Position is the abstraction for an account's balance or position for particular instruments. The example is from the financial services domain where accounts are managed by the firm and updated with information from the trading and settlement systems.


class Position {
  private Account account;
  private Instrument instrument;
  private double balance;
  //..
  private PositionUpdationStrategy strategy;
  //..

  void doUpdate(..) {
    strategy.doUpdate(..);
  }
}



The method Position#doUpdate(..) updates the Position with the new information, which gets reflected in the state of the object. And we have a Specification class which validates if a Position is short - this class encapsulates the domain knowledge for determining a short position for an account.


class ShortPositionSpecification
    extends PositionSpecification {
  boolean isSatisfiedBy(Position pos) {
    //..
    //.. short position logic
    //return true if short position
  }
}



Now, here is a service class which uses the specification to determine short position and act accordingly.


class PositionManager {
  //..
  //..
  Position updatePosition(..) {
    Position current = getPosition(..);
    current.doUpdate(..);

    //.. invoke the specification
    if (new ShortPositionSpecification().isSatisfiedBy(current)) {
      // .. alternate business path
      //.. do what ?
      return null;
    }
    return current;
  }
  //..
  //..

  Position getPosition(..) {
    //..
  }
}



Now, we have a couple of issues here :-

  • What do we return from PositionManager#updatePosition(..) if the position is short and we need to execute the alternate business logic ? Null ? Sounds yucky - explicit null checks need to be done by clients of PositionManager, which looks smelly.

  • One option may be to have a flag in Position which indicates short-position, and have it set when the Specification indicates short position. Once again the clients of PositionManager need to do an if-then-else check to determine the short position of the account and handle the alternate business logic. Yuck! Still smelly ..


Let us put our Position model on some more steroids. The domain language says that in case of existing delinquent positions, the method PositionManager#updatePosition(..) should bail out early without doing any updates. This adds yet another PositionSpecification to our module ..


class DelinquentPositionSpecification
    extends PositionSpecification {
  boolean isSatisfiedBy(Position pos) {
    //..
    //.. delinquent position logic
    //return true if delinquent position
  }
}



and add yet another check in PositionManager#updatePosition(..) :


class PositionManager {
  //..
  //..
  Position updatePosition(..) {
    current = getPosition(..);

    if (new DelinquentPositionSpecification().isSatisfiedBy(current)) {
      //.. alternate business path
      //.. do what ?
      return null;
    }

    current.doUpdate(..);
    if (new ShortPositionSpecification().isSatisfiedBy(current)) {
      // .. alternate business path
      //.. do what ?
      return null;
    }
    return current;
  }
  //..
  //..

  Position getPosition(..) {
    //..
  }
}



Umm .. Things start getting a bit messy and procedural. In both the cases, we return null, since we do not have any typed abstraction indicating the alternate path of execution. Instead of nulls, we can think of implementing the Null Object pattern, NullPosition. While this will make things slightly better, still the client code has to rely on checking flags to determine the exact alternate pathway to take. The Specification pattern encapsulates the constraints (as it should be), but it does not provide any abstraction for identifying the alternate flow of control. The caller still has to process null returns and differentiate them based on some flags (which may be multiple). Here is what the client of PositionManager can do ..


Position pos = PositionManager.getInstance().updatePosition(..);
if (pos == null) {
  if (pos.isDelinquent()) {
    //.. handle delinquent position
  }
  if (pos.isShort()) {
    //.. handle short position
  }
  //.. any other ?
}
//..
//..



This is definitely not the OO way of handling things! Also, all the procedural if-then-else code violate the principle of Making Implicit Concepts Explicit, as specified by Eric Evans in his DDD book. The alternate business flow for short position or delinquent position is buried within the stream of if-then-else statements and flag manipulations. None of the abstractions publish any explicit contract that indicates the alternate pathway which our execution can take. Ideally we would like to have our apis publish all alternative paths explicitly, so that the api user can figure out all of them before programming-to-the-api set.

Can Polymorphism help ?

I can sense several fans of polymorphism raising their heads with the idea of returning polymorphic variants of Position from the updatePosition(..) method. This will never scale and we are bound to have an explosion of subclasses with this approach. Let us look elsewhere ..

Flamebait : The love and hate of Checked Exceptions

In my last post, I had indicated how checked exceptions in Java help us provide typed abstractions publishing the alternate execution paths as part of the contract. And, as we saw above, Specifications allow us to encapsulate the business rule itself, whose violation leads to the execution of those alternate paths. Combine the two, and we can have a better typed model for both the cause and the execution of the alternate flows.

Move the invocation of the specifications to the source instead of checking them as post-conditions in PositionManager#updatePosition(..) ..


class PositionManager {
  //..
  //..

  //.. explicit publication
  Position getPosition(..)
      throws DelinquentPositionException {
    //.. find the current position
    current = ..
    if (new DelinquentPositionSpecification().isSatisfiedBy(current)) {
      throw new DelinquentPositionException(..);
    }

  }
}



and short-position handling moves to the strategy ..


class DefaultPositionUpdationStrategy
    implements PositionUpdationStrategy {

  void doUpdate(Position position)
    throws ShortPositionException {
    //.. check for short position
    if (new ShortPositionSpecification().isSatisfiedBy(position)) {
      throw new ShortPositionException(..);
    }
    //..
  }
}



Now PositionManager#updatePosition(..) becomes cleaner with lots of specification checking moving inside the guts of respective classes ..


class PositionManager {
  //..
  //..
  Position updatePosition(..) {
    try {

      //.. normal flow
      current = getPosition(..);
      current.doUpdate(..);
      return current;

    } catch (ShortPositionException spex) {
      //.. short position handling
    } catch (DelinquentPositionException dpex) {
      //.. delinquent position handling
    }
  }
  //..
  //..
}



Does this look better than the procedural if-then-else code above ? Checked exceptions are typed contracts that the caller needs to honor and handle. When I am designing typed apis, I consider them to be my friend. There is a community out there divided on the benefits of checked exceptions - recently we have seen lots of flamebaits once again on this very topic. And there are enough examples to showcase on either side of the argument. I find checked exceptions very useful while handling alternate business logic paths in my application. There are many situations where checked exceptions look like getting into your way, resulting in too much try-catch-finally in your codebase. But in handling alternate business scenarios, checked exceptions serve the purpose of advertising explicit business rule violations to the caller. They are part of the api and appear in the Javadoc - the user of your api gets to be aware of all alternate business scenarios that the domain model handles. And this makes the implicit concept explicit.

Sunday, July 15, 2007

Can you differentiate between an Abstraction and a Lie ?

In the June 2007 issue of Communications of the ACM, David Parnas differentiates between an abstraction and a lie. He mentions that abstractions that embody unrealistic assumptions represent nothing at all. He learnt the correct definition of abstraction from the late E. W. Dijkstra :
An abstraction is one thing that represents several real things equally well.

And he concludes ..
Finding the simplest model that is not a lie is the key to better software design.

So true, and how lucidly expressed! Go read the full article here. [Membership required for full text].

Wednesday, July 11, 2007

In Defense of Static Typing

Just thought it may be relevant to bring this thread up in context of the debate going on in the Artima forum between the camps of static and dynamic programming languages. The languages under discussion have changed, the thought leaders have changed, however, the topics debated have remained the same .. the verbosity of the static type system, benefits (if any) of type inferencing, power of dynamic typing and the defense of the straw man argument.

Here are some snippets :-

From Matthias Felleisen (of The Little Schemer and the rest of the series fame)

Static types are necessary
(1) to establish basic properties of software and
(2) to establish absolute yet cheap abstraction boundaries between modules.


and more of some of the basic values of static typing ..

With types, you can say something like this at a module boundary:

there is a type t, here are some instances of type t, i even give you functions to create t's, and work with t's, but i won't tell you what they are implemented in.

Now you or your partners have no way of corrupting these values. And you know what, they are actually integers, and if you had known, you could have used all kinds of numeric hacks to compute values. But fortunately you didn't, because I, the module writer, can now change t to a hash table and do all kinds of nice things with it.

What's the cost for that? Zero. Yeap, in a statically typed world, there is no run-time cost.


and of course there is Paul Graham on the opposite camp .. regarding static typing ..

It's like the way the military does things. Making everyone fill out forms to do anything certainly prevents mistakes. But it also inhibits innovation terribly.

I'm not saying the military is mistaken here. There are places where restrictions are good-- particularly when you don't trust individual people not to screw up. So requiring type declarations or other assertions (e.g. all the boilerplate that seems to be required in Java) could be a good idea in some big company where the programmers are guys who walked in off the street after Learning Java in 21 Days. But at the other end of the continuum, there is also a role for languages that put as few constraints on the programmer as possible.


Enjoy the complete thread here !

I think I would like to have some militarism in contracts when I am executing a big-bang enterprise project. As Cedric has rightly mentioned,
nothing beats a dynamic language to write a prototype, but if you know that the code will have to be maintained and that it will be in use for several years, the initial cost of using a statically typed language is an investment that is very quickly amortized...


Types are elements promoting Design By Contract at the lowest level and I would not want to lose the enforcements that they bring to my higher level abstractions. And finally, the refactoring support that I get from statically typed languages is way too precious to lose when I am working on a large codebase.