Designing Internal DSLs in Scala

In an earlier post I had talked about building external DSLs using parser combinators in Scala. External DSLs involve parsing of syntax foreign to the native language - hence the ease of developing external DSLs depends a lot on parsing and parse tree manipulation capabilities available in existing libraries and frameworks. Internal DSLs are a completely different beast altogether, and singularly depends on the syntax and meta programming abilities that the language offers. If you have a language that is syntactically rigid and does not offer any meta (or higher level) programming features, then there is no way you can design usable internal DSLs. You can approximate to the extent of designing fluent interfaces.

Scala is a statically typed language that offers lots of features for designing concise, user friendly internal DSLs. I have been working on designing a DSL in Scala for interacting with a financial trading and settlement system, implemented primarily in Java. Since Scala interoperates with Java quite well, a DSL designed in Scala can be a potent tool to provide your users with nicely concise, malleable APIs even over existing Java domain model.

Here is a snippet of the DSL in action, for placing client orders for buy/sell of securities to the exchange ..

val orders = List[Order](

  // use premium pricing strategy for order
  new Order to buy(100 sharesOf "IBM")
            maxUnitPrice 300
            using premiumPricing,

  // use the default pricing strategy
  new Order to buy(200 sharesOf "GOOGLE")
            maxUnitPrice 300
            using defaultPricing,

  // use a custom pricing strategy
  new Order to sell(200 bondsOf "Sun")
            maxUnitPrice 300
            using {
              (qty, unit) => qty * unit - 500
            }
)


The DSL looks meaningful enough for the business analysts as well, since it uses the domain language and does not contain much of the accidental complexities that we get in languages like Java. The language provides easy options to plug in default strategies (e.g. for pricing orders, as shown above). Also it offers power users the ability to define custom pricing policies inline when instantiating the Order.

Here are some of the niceties in the syntax of Scala that makes it a DSL friendly language ..

• Implicits


• Higher order functions


• Optional dots, semi-colons and parentheses


• Operators like methods


• Currying

Implicits are perhaps the biggest powerhouse towards designing user friendly DSLs. They do away with the static cling of Java and yet offer a safe way to extend existing abstractions. Implicits in Scala is perhaps one of the best examples of a meaningful compromise in language design between uncontrolled open classes of Ruby and the sealed abstractions of Java. In the above example DSL, I have opened up the Int class and added methods that convert a raw number to a quantity of shares. Here is the snippet that allows me to write code like sell(200 bondsOf "Sun") as valid Scala code.

class PimpedInt(qty: Int) {
  def sharesOf(name: String) = {
    (qty, Stock(name))
  }

  def bondsOf(name: String) = {
    (qty, Bond(name))
  }
}

implicit def pimpInt(i: Int) = new PimpedInt(i)


And the best part is that the entire extension of the class Int is lexically scoped and will only be available within the scope of the implicit definition function pimpInt.

Scala, having strong functional capabilities, offer higher order functions, where, first class methods can be passed around like objects in the OO world. This helps define custom control abstractions that look like the natural syntax of the programming language. This is another great feature that helps design DSLs in Scala. Add to that, optional parentheses and dots, and you can have syntax like ..

to buy(200 sharesOf "GOOGLE")
maxUnitPrice 300
using defaultPricing


where defaultPricing and premiumPricing are functions that have been passed on as arguments to methods. The method using in class Order takes a function as input. And you can define the function inline as well, instead of passing a predefined one. This is illustrated in the last Order created in the above example.

Another small subtlety that Scala offers is the convenience to sugarize methods of a class that takes one parameter. In the above example, 100 sharesOf "IBM" is actually desugared as 100.sharesOf("IBM"). Though the former looks more English like, without the unnecessary dot and parenthesis. Nice!

Here is the complete listing of an abbreviated version of the DSL and a sample usage ..

object TradeDSL {

  abstract class Instrument(name: String) { def stype: String }
  case class Stock(name: String) extends Instrument(name) {
    override val stype = "equity"
  }
  case class Bond(name: String) extends Instrument(name) {
    override val stype = "bond"
  }

  abstract class TransactionType { def value: String }
  case class buyT extends TransactionType {
    override val value = "bought"
  }
  case class sellT extends TransactionType {
    override val value = "sold"
  }

  class PimpedInt(qty: Int) {
    def sharesOf(name: String) = {
      (qty, Stock(name))
    }

    def bondsOf(name: String) = {
      (qty, Bond(name))
    }
  }

  implicit def pimpInt(i: Int) = new PimpedInt(i)

  class Order {
    var price = 0
    var ins: Instrument = null
    var qty = 0;
    var totalValue = 0
    var trn: TransactionType = null
    var account: String = null

    def to(i: Tuple3[Instrument, Int, TransactionType]) = {
      ins = i._1
      qty = i._2
      trn = i._3
      this
    }
    def maxUnitPrice(p: Int) = { price = p; this }

    def using(pricing: (Int, Int) => Int) = {
      totalValue = pricing(qty, price)
      this
    }

    def forAccount(a: String)(implicit pricing: (Int, Int) => Int) = {
      account = a
      totalValue = pricing(qty, price)
      this
    }
  }

  def buy(qi: Tuple2[Int, Instrument]) = (qi._2, qi._1, buyT())
  def sell(qi: Tuple2[Int, Instrument]) = (qi._2, qi._1, sellT())

  def main(args: Array[String]) = {

    def premiumPricing(qty: Int, price: Int) = qty match {
      case q if q > 100 => q * price - 100
      case _ => qty * price
    }

    def defaultPricing(qty: Int, price: Int): Int = qty * price

    val orders = List[Order](

      new Order to buy(100 sharesOf "IBM")
                maxUnitPrice 300
                using premiumPricing,

      new Order to buy(200 sharesOf "CISCO")
                maxUnitPrice 300
                using premiumPricing,

      new Order to buy(200 sharesOf "GOOGLE")
                maxUnitPrice 300
                using defaultPricing,

      new Order to sell(200 bondsOf "Sun")
                maxUnitPrice 300
                using {
                  (qty, unit) => qty * unit - 500
                }
    )
    println((0 /: orders)(_ + _.totalValue))
  }
}