Monday, September 27, 2010

Domain Models - Thinking differently in Scala & Clojure

A language that doesn't affect the way you think about programming, is not worth knowing.
- Alan J Perlis

When you model a domain you map the artifacts from the problem domain to the solution domain. The problem domain artifacts are the same irrespective of your solution domain. But the mapping process depends on your medium of modeling, the target platform, the programming language and the paradigms that it offers. Accordingly you need to orient your thought process so as to adapt to the language idioms that you have chosen for implementation.

Recently I did a fun exercise in modeling the same problem domain on to 2 target languages - Scala, that offers a mix of OO and functional features and Clojure, that's much more functional and makes you think more in terms of functions and combinators. The idea is to share the thought process of this domain modeling exercise and demonstrate how even similar architectural patterns of the solution can map to entirely different paradigms in the  implementation model.

This is also one of the underlying themes of my upcoming book DSLs In Action. When you think of a DSL you need to think not only of the surface syntax that the user gets to know, but also of the underlying domain model that forms the core of the DSL implementation. And the underlying host language paradigms shape the way you think of your DSL implementation. In the book there are plenty of examples where I take similar examples from one problem domain and design DSLs in multiple languages. That way you get to know how your thought process needs to be re-oriented when you change implementation languages even for the same problem at hand.

Modeling in Scala, a hybrid object functional language

Consider an abstraction for a security trade. For simplicity we will consider only a small set of attributes meaningful only for the current discussion. Let's say we are modeling a Trade abstraction using a statically typed language like Scala that offers OO as one of the paradigms of modeling. Here's a sample implementation that models a Trade as a case class in Scala ..

Objects for coarse-grained abstractions ..

type Instrument = String
type Account = String
type Quantity = BigDecimal
type Money = BigDecimal

import java.util.{Calendar, Date}
val today = Calendar.getInstance.getTime

case class Trade(ref: String, ins: Instrument, account: Account, unitPrice: Money,
  quantity: Quantity, tradeDate: Date = today) {
  def principal = unitPrice * quantity

Ok .. that was simple. When we have classes as the primary modeling primitive in the language we try to map artifacts to objects. So the trade artifact of the problem domain maps nicely to the above class Trade.

But a trade has a definite lifecycle and a trade abstraction needs to be enriched with additional attributes and behaviors in course of the various stages of the trading process. How do we add behaviors to a trade dynamically ?

Consider enriching a trade with tax and fee attributes when we make an instance of a Trade. Similar to Trade we also model the tax and fee types as separate artifacts that can be mixed in with the Trade abstraction.

trait TradeTax { this: Trade =>
  def tradeTax(logic: Money => Money): Money = logic(principal)

trait Commission { this: Trade =>
  def commission(logic: (Money, Quantity) => Money): Money = logic(principal, quantity)

Now we can instantiate a trade decorated with the additional set of taxes and fees required as per the market regulations ..

lazy val t = new Trade("1", "IBM", "a-123", 12.45, 200) with TradeTax with Commission

Note how the final abstraction composes the Trade class along with the mixins defined for tax and fee types. The thought process is OO along with mixin inheritance - a successful implementation of the decorator pattern. In a language that offers classes and objects as the modeling primitives, we tend to think of them as abstracting the coarse grained artifacts of the domain.

Functions for fine grained domain logic ..

Also note that we use higher order functions to model the variable part of the tax fee calculation. The user supplies this logic during instantiation which gets passed as a function to the calculation of TradeTax and Commission.

lazy val tradeTax = t.tradeTax { p => p * 0.05 }
lazy val commission = t.commission { (p, q) => if (> 100) p * 0.05 else p * 0.07 }

When modeling with Scala as the language that offers both OO and functional paradigms, we tend to use the combo pack - not a pure way of thinking, but the hybrid model that takes advantage of both the paradigms that the language offers.

And now Clojure - a more functional language than Scala

As an alternate paradigm let's consider the same modeling activity in Clojure, a language that's more functional than Scala despite being hosted on top of the Java language infrastructure. Clojure forces you to think more functionally than Scala or Java (though not as purely as Haskell). Accordingly our solution domain modeling thoughts also need to take a functional bend. The following example is also from my upcoming book DSLs In Action and illustrates the same point while discussing DSL design in Clojure.

Trade needs to be a concrete abstraction and one way of ensuring that in Clojure is as a Map. We can also use defrecord to create a Clojure record, but that's not important for the point of today's discussion. We model a trade as a Map, but abstract its construction behind a function. Remember we are dealing with a functional language and all manipulations of a trade need to be thought out as pure functions that operate on immutable data structures.

This is how we construct a trade from a request, which can be an arbitrary structure. In the following listing, trade is the constructor function that returns a Map populated from a request structure.

Abstractions to map naturally to functions ..

; create a trade from a request
(defn trade
    "Make a trade from the request"
  {:ref-no (:ref-no request)
   :account (:account request)
   :instrument (:instrument request)
   :principal (* (:unit-price request) (:quantity request))
   :tax-fees {}})
0x3b a sample request 
(def request
  {:ref-no "trd-123"
   :account "nomura-123"
   :instrument "IBM"
   :unit-price 120
   :quantity 300})

In our case the Map just acts as the holder of data, the center of attraction is the function trade, which, as we will see shortly, will be the main subject of composition.

Note that the Map that trade returns contains an empty tax-fees structure, which will be filled up when we decorate the trade with the tax fee values. But how do we do that idiomatically, keeping in mind that our modeling language is functional and offers all goodness of immutable and persistent data structures. No, we can't mutate the Map!

Combinators for wiring up abstractions ..

But we can generate another function that takes the current trade function and gives us another Map with the tax-fee values filled up. Clojure has higher order functions and we can make a nice little combinator out of them for the job at hand ..

; augment a trade with a tax fee value
(defn with-values [trade tax-fee value]
  (fn [request]
    (let [trdval (trade request)
          principal (:principal trdval)]
       (assoc-in trdval [:tax-fees tax-fee]
         (* principal (/ value 100))))))

Here trade is a function which with-values takes as input and it generates another function that decorates the original trade Map with the passed in tax-fee name and the value. Here the value that we pass is a percentage that's calculated based on the principal of the trade. I have kept it simpler than the Scala version that models some more logic for calculating each individual tax fee value. This is the crux of the decorator pattern of our model. We will soon dress it up a little bit and give it a user friendly syntax.

and Macros for DSL ..

Now we can define another combinator that plays around nicely with with-values to model the little language that our users can relate to their trading desk vocabulary.

Weeee .. it's a macro :)

; macro to decorate a function       
(defmacro with-tax-fee
    "Wrap a function in one or more decorators"
  [func & decorators]
  `(redef ~func (-> ~func ~@decorators)))

and this is how we mix it with the other combinator with-values ..

(with-tax-fee trade
  (with-values :tax 12)
  (with-values :commission 23))

Note how we made with-tax-fee a decorator that's replete with functional idioms that Clojure offers. -> is a Thrush combinator in Clojure that redefines the original function trade by threading it through the decorators. redef is just a helper macro that redefines the root binding of the function preserving the metadata. This is adapted from the decorator implementation that Compojure offers.

We had the same problem domain to model. In the solution domain we adopted the same overall architecture of augmenting behaviors through decorators. But the solution modeling medium were different. Scala offers the hybrid of OO and functional paradigms and we used both of them to implement decorator based behavior in the first case. In the second effort, we exploited the functional nature of Clojure. Our decorators and the subject were all functions and using the power of higher order functions and combinator based approach we were able to thread the subject through a series of decorators to augment the original trade abstraction.


Stephan.Schmidt said...

My problem with this:

lazy val tradeTax = t.tradeTax { p => p * 0.05 }
lazy val commission = t.commission { (p, q) => if (q > 100) p * 0.05 else p * 0.07 }

is reusability and maintanence. I have the feeling this will lead to many undocumented, nameless functions scattered around the code base - or you need a lot of discipline. I'd prefer your approach with type classes for this. Type classes have names, can easily be reused etc.


Debasish said...

Hi Stephan -

Thanks for visiting.

Type classes are definitely a way to handle variabilities in your abstraction. In the current example, it makes sense to use type classes if we are going to have a number of variations in computing tradeTax or commission. However, the moot point of the current post is to demonstrate that in languages like Scala that offer the hybrid approach of OO and functional programming, we use OO for modeling coarse grained abstractions and use functional approach for modeling fine grained behaviors and control flow. Hence I thought simple higher order functions can also make this point clear. Implementing type classes would have made the post longer and possibly could have shifted the focus of discussion as well. And I have discussed enough of type classes in my blog :)


Javier said...

Have you considered use clojure features like structs, records (with protocols or not) to implement in clojure somewhat more similar to scala approx?
i guess you have chosen the diff approx in clojure for the purpose of demonstrating the point but imo a impl closer to scala is possible in idiomatic clojure

Debasish said...

Hi Javier -

As I mentioned in the post, the Scala approach is to focus towards composition of classes/objects. While the Clojure approach is to compose functions. Irrespective of how you model the underlying trade (Map, Record, struct-map etc.), I think the idiomatic way to decorate an abstraction is to use the Thrush (the -> macro) to thread through the subject function and the decorators. In fact I mentioned it in the post as well that we could have modeled the underlying trade structure using defrecord. And yet implement the dynamic addition of behaviors through function composition.


Thoth said...

i'm interested in how you learned those languages if you don't mind. you provided a comparison in two languages and i'm curious how did you go about learning these:

1) how long did it take you ?
2) what is your level of expertise ?
3) did you walk this path alone ?

4) did some one help with the code for this article ?


Sergio Bossa said...

Why didn't you use clojure protocols rather than macros?
I think the former is better suited for domain modeling, while the latter is better suited for higher-level composition, such as expressing a whole use case logic (and using clojure protocols/records under the hood). Think at it like "domain layer" vs "service layer" :)


Sergio B.

Debasish said...

Hi Sergio -

Thanks for visiting my blog.

The main focus of this post is to demonstrate how implementing even the same architectural pattern like *behavior extension* makes you think differently in two languages espousing two different paradigms.

I have modeled the Trade abstraction as a Map. I mentioned this in the article that I could have modeled it as a Clojure protocol also. In that case I would have a defrecord implementing the protocol. But ultimately a defrecord provides an implementation of a persistent map only and offers the same keyword accessors for fields. The final composition of the functions would have been the same and there would not have been any change in the thrush based decorator implementation. Hence I thought to keep the implementation simpler with a bare bone Map as the Trade abstraction.

And the macro was just for spicing up a nice DSL that abstracts the thrush application.

Javier said...

definitely i think i have catched the main point (describe two ways of thinking encouraged for the two langs)
The compojure redefine in "read time" and the macro sintactic way to abstract decoration is nice but i agree with Sergio Bossa that maybe for domain modelling the semantic way fits better.
For curiosity i have tried to mimic your scala impl in clojure, although my knowledge of scala is very basic (but clojure one is not too advanced :-P )

Javier said...

definitely i think i have catched the main point (describe two ways of thinking encouraged for the two langs)
The compojure redefine in "read time" and the macro sintactic way to abstract decoration is nice but i agree with Sergio Bossa that maybe for domain modelling the semantic way fits better.
For curiosity i have tried to mimic your scala impl in clojure, although my knowledge of scala is very basic (clojure one is no too advanced :-P )

Debasish said...

Hi Javier -

You Clojure implementation almost mimics the Scala one. However when u define a defrecord for Trade along with a composition of TradeTax and Commission protocols, you have hardwired them as part of Trade. Note that u cannot have any other type of tax-fees as part of Trade abstraction. This was not the case with the Scala implementation where we used a mixin based approach that gives me the flexibility to compose abstractions while creating the object.

In my Clojure implementation also it was dynamic - not the with-values function and how the thrush threads the subject with all the decorators during runtime.

In fact this is one the underlying thoughts of the post. We should not try to mimic an implementation in one language into another. We should use what's idiomatic in every language.

Javier said...

(sorry but i use twitter almost automatically, i rewrite here my twitter comments to continue the thread):
You're right with defrecord,i've changed my code to use extend,a little closer to scala -but not the same-. However i think it would be idiomatic clojure too,
i agree with the main point: clojure shines more in one facet (abstracting syntactically) and scala in the other (abstracting semantically).

mario said...

Great post, really.

I admit I am far more comfortable with Scala than with Clojure so probably my judgment could be a bit biased. Anyway I find the Scala solution far more readable and easy to understand. I guess also syntax issues also play a role in that. I am afraid completely functional languages are not a good playgroud for creating DSLs.

In my experience, if I design well my DSL in Scala, I am often able to allow business people to read, understand and validate my business logic even if they never wrote a line of code. A few months ago one of them (one of the smartest I met in my life to be honest) was so fascinated that he asked me a short dictionary of the keywords I used in my DSL and by looking at my examples he was almost able to write the entire business logic he needed in pure Scala making only a few trivial and basically syntactical mistakes.

Provided that the biggest part of problems in our job come from the frequent misunderstandings between men-with-tie and geeks, shouldn't be fantastic if business men were able to write (and modify) their business logic by their own? Do you think you could achieve a similar result in Clojure? I am very skeptical of that.

zarama said...

thanks for a very good blog on a subject of these two languages, Scala and Clojure.

I started reading your entries to figure out which of these two language to select and the more I read I find that inevitably both languages offer different ways to think and implement solutions.

Reading about Clojure and experiment a bit with it I very much like its elegance but soon I find material on Scala and same effect often for different reasons and it appears that you and others master both quite well.

Thanks again for your work re subject languages and look forward to reading your book

Sergio Bossa said...

Posted my thoughts about the subject on my blog: ... your feedback would be greatly appreciated :)

Steve said...

It's not clear to me why those vals are lazy. Could you elaborate?