Wednesday, February 11, 2015

Functional Patterns in Domain Modeling - Composing a domain workflow with statically checked invariants

I have been doing quite a bit of domain modeling using functional programming mostly in Scala. And as it happens when you work on something for a long period of time you tend to identify more and more patterns that come up repeatedly within your implementations. You may ignore these as patterns the first time, get a feeling of mere coincidence the next time, but third time really gives you that aha! moment and you feel like documenting it as a design pattern. In course of my learnings I have started blogging on some of these patterns - you can find the earlier ones in the series in:

  • Functional Patterns in Domain Modeling - The Specification Pattern

  • Functional Patterns in Domain Modeling - Immutable Aggregates and Functional Updates

  • Functional Patterns in Domain Modeling - Anemic Models and Compositional Domain Behaviors

  • In this continuing series of functional patterns in domain modeling, I will go through yet another idiom which has been a quite common occurrence in my explorations across various domain models. You will find many of these patterns explained in details in my upcoming book on Functional and Reactive Domain Modeling, the early access edition of which is already published by Manning.

    One of the things that I strive to achieve in implementing domain models is to use the type system to encode as much domain logic as possible. If you can use the type system effectively then you get the benefits of parametricity, which not only makes your code generic, concise and polymorphic, but also makes it self-testing. But that's another story which we can discuss in another post. In this post I will talk about a pattern that helps you design domain workflows compositionally, and also enables implementing domain invariants within the workflow, all done statically with little help from the type system.

    As an example let's consider a loan processing system (simplified for illustration purposes) typically followed by banks issuing loans to customers. A typical simplified workflow looks like the following :-

    The Domain Model

    The details of each process is not important - we will focus on how we compose the sequence and ensure that the API verifies statically that the correct sequence is followed. Let's start with a domain model for the loan application - we will keep on enriching it as we traverse the workflow.

    case class LoanApplication private[Loans](
      // date of application
      date: Date,
      // name of applicant
      name: String,
      // purpose of loan
      purpose: String,
      // intended period of repayment in years
      repayIn: Int,
      // actually sanctioned repayment period in years
      actualRepaymentYears: Option[Int] = None,
      // actual start date of loan repayment
      startDate: Option[Date] = None,
      // loan application number
      loanNo: Option[String] = None,
      // emi
      emi: Option[BigDecimal] = None

    Note we have a bunch of attributes that are defined as optional and will be filled out later as the loan application traverses through the sequence of workflow. Also we have declared the class private and we will have a smart constructor to create an instance of the class.

    Wiring the workflow with Kleisli

    Here are the various domain behaviors modeling the stages of the workflow .. I will be using the scalaz library for the Kleisli implementation.

    def applyLoan(name: String, purpose: String, repayIn: Int, 
      date: Date = today) =
      LoanApplication(date, name, purpose, repayIn)
    def approve = Kleisli[Option, LoanApplication, LoanApplication] { l => 
      // .. some logic to approve
        loanNo = scala.util.Random.nextString(10).some,
        actualRepaymentYears = 15.some,
        startDate = today.some
    def enrich = Kleisli[Option, LoanApplication, LoanApplication] { l => 
      //.. may be some logic here
      val x = for {
        y <- l.actualRepaymentYears
        s <- l.startDate
      } yield (y, s)
      l.copy(emi = { case (y, s) => calculateEMI(y, s) }).some

    applyLoan is the smart constructor that creates the initial instance of LoanApplication. The other 2 functions approve and enrich perform the approval and enrichment steps of the workflow. Note both of them return an enriched version of the LoanApplication within a Kleisli, so that we can use the power of Kleisli composition and wire them together to model the workflow ..

    val l = applyLoan("john", "house building", 10)
    val op = approve andThen enrich
    op run l

    When you have a sequence to model that takes an initial object and then applies a chain of functions, you can use plain function composition like h(g(f(x))) or using the point free notation, (h compose g compose f) or using the more readable order (f andThen g andThen h). But in the above case we need to have effects along with the composition - we are returning Option from each stage of the workflow. So here instead of plain composition we need effectful composition of functions and that's exactly what Kleisli offers. The andThen combinator in the above code snippet is actually a Kleisli composition aka function composition with effects.

    So we have everything the workflow needs and clients use our API to construct workflows for processing loan applications. But one of the qualities of good API design is to design it in such a way that it becomes difficult for the client to use it in the wrong way. Consider what happens with the above design of the workflow if we invoke the sequence as enrich andThen approve. This violates the domain invariant that states that enrichment is a process that happens after the approval. Approval of the application generates some information which the enrichment process needs to use. But because our types align, the compiler will be perfectly happy to accept this semantically invalid composition to pass through. And we will have the error reported during run time in this case.

    Remembering that we have a static type system at our disposal, can we do better ?

    Phantom Types in the Mix

    Let's throw in some more types and see if we can tag in some more information for the compiler to help us. Let's tag each state of the workflow with a separate type ..

    trait Applied
    trait Approved
    trait Enriched

    Finally make the main model LoanApplication parameterized on a type that indicates which state it is in. And we have some helpful type aliases ..

    case class LoanApplication[Status] private[Loans]( //..
    type LoanApplied  = LoanApplication[Applied]
    type LoanApproved = LoanApplication[Approved]
    type LoanEnriched = LoanApplication[Enriched]

    These types will have no role in modeling domain behaviors - they will just be used to dispatch to the correct state of the sequence that the domain invariants mandate. The workflow functions need to be modified slightly to take care of this ..

    def applyLoan(name: String, purpose: String, repayIn: Int, 
      date: Date = today) =
      LoanApplication[Applied](date, name, purpose, repayIn)
    def approve = Kleisli[Option, LoanApplied, LoanApproved] { l => 
        loanNo = scala.util.Random.nextString(10).some,
        actualRepaymentYears = 15.some,
        startDate = today.some
    def enrich = Kleisli[Option, LoanApproved, LoanEnriched] { l => 
      val x = for {
        y <- l.actualRepaymentYears
        s <- l.startDate
      } yield (y, s)
      l.copy(emi = { case (y, s) => calculateEMI(y, s) })[LoanEnriched])

    Note how we use the phantom types within the Kleisli and ensure statically that the sequence can flow only in one direction - that which is mandated by the domain invariant. So now an invocation of enrich andThen approve will result in a compilation error because the types don't match. So once again yay! for having the correct encoding of domain logic with proper types.


    EECOLOR said...

    I like your use of shadow types.

    I must be missing something (which might be hidden in the terminology you are using). The following seems to work just fine:

    val approve: LoanApplied => LoanApproved =
    loanNo = Some(scala.util.Random.nextString(10)),
    actualRepaymentYears = Some(15),
    startDate = Some(today))

    val enrich: LoanApproved => LoanEnriched = { l =>
    val x = for {
    y <- l.actualRepaymentYears
    s <- l.startDate
    } yield (y, s)

    l.copy(emi = { case (y, s) => calculateEMI(y, s) })

    val l = applyLoan("john", "house building", 10)
    val op = approve andThen enrich

    Debasish Ghosh said...

    When you have functions with return type LoanApplied => LoanApproved, you cannot handle effects. e.g. the approval process may fail and this needs to be indicated in the return type. Also the sequence needs to be broken if one of the steps fail. Hence the return type can be LoanApplied => Option[LoanApproved] where Option indicates the possibility that the approval may fail.

    And Kleisli is precisely a generalization of this - a Kleisli models an effectful function application. Kleisli[M, A, B] models A => M[B] where M is the effect (Option in this example).

    Hence Kleisli is a better option since we can have effects along with the sequencing.

    Stephan.Schmidt said...

    Very nice article, learned something, and the neat little usage of identity to change types.

    Debasish Ghosh said...

    Thanks Stephan .. Glad that you liked it.

    Satish said...

    Nice article Debashish

    Andrae Muys said...

    Excellent article.

    This only appears to work for simple linear workflows.

    How do you model such things as loops, optional stages, or joins/meets? I can't help thinking extending this might end up needing dependent types?

    pagoda_5b said...

    Good article as ever.
    I don't really like the way you have to map with identity to force the correct phantom type.
    Is there a neater solution to that?
    And could the code be made better still, using lenses?

    Thank you

    Debasish Ghosh said...

    Hi Ivano -

    I don't think there's any option other than using the map(identity[..]). This is because the phantom types are there only for this purpose and do not take any part in logic. So we need to force a coercion on to them.

    Regarding lenses, yes, we can of course replace the copy with lens. In fact with complex domain models that's the recommended approach. Here it's a simple example and the point I wanted to demonstrate is something different. Hence I used the copy for illustration purposes.


    EECOLOR said...

    l.copy[Approved](...) would probably work

    Unknown said...

    Great post! I'm still trying to grasp the advantage of using Kleisli arrows over simple monad binding to compose functions. I rewrote your example using flatMaps: HERE.

    Could you give me a hint what do we lose by switching to the second option in this case?

    Debasish Ghosh said...

    @Unknown -

    You can always use monads natively to model this use case. Kleisli offers a higher level of abstraction and hence makes your code much more readable. Compare the version of process() function with the one written using kleisli - it reads better from the domain modeling point of view. The sequencing is more explicit as a DSL. But the more important point is that the andThen combinator of Kleisli uses the same algebra of the monad. See it's implementation at which uses the bind. It's all about programming at the right level of abstraction. Once you have the functions returning a Kleisli you have at your disposal all the combinators that a Kleisli arrow offers. You suddenly have more higher order power to glue your components.

    Mona Borham said...
    This comment has been removed by the author.