Monday, February 11, 2008

Why I like Scala's Lexically Scoped Open Classes

Ruby allows you to open up any class definition and plug in your methods into the guts of the existing class. John Carter opens up Integer and adds factorial ..

class Integer
  def factorial
    return 1 if self <= 1
    self * (self-1).factorial

This allows him to write

puts 10.factorial

While this can lead to designing nice looking expressions of "pleasing symmetry", that read consistently from left-to-right (5.succ.factorial.succ.odd), it looks scary to me. Not the openness of the class, but the global context in which it is open.

Matz, replying to a thread, on the difference between ruby and python, mentions ..
"open class" is so strong (often too strong), we can break things easily. In other word, Ruby trust you to give you sharp knives, where Python don't. From the Python point of view, it's wrong, I guess.

In a way, I think the implementation of globally open class in Ruby brings up similar problems as the obliviousness property of aspects. You really never know what has been added to a class and whether the new implementation that you are plugging in, breaks apart any of the methods added by another developer. And it is only because all extensions in Ruby classes take place on the global namespace without any context whatsoever. Check out the discussion between Reginald Braithwaite and Avi Bryant in the comments section of the above Raganwald post.

Scalability problems in large projects ?

Scala has put in better thoughts in designing lexically scoped open classes. They call it "implicits" and allow extension of existing classes through a lexically scoped implicit conversion. Taking the example from Martin Odersky's blog, in order to add the append() method to Scala's built-in Array, define a class as ..

class RichArray[T](value: Array[T]) {
    def append(other: Array[T]): Array[T] = {
        val result = new Array[T](value.length + other.length)
        Array.copy(value, 0, result, 0, value.length)
        Array.copy(other, 0, result, value.length, other.length)

and add an implicit conversion from plain arrays to rich arrays:

implicit def enrichArray[T](xs: Array[T]) = new RichArray[T]

Now we can apply the append() method to plain arrays as well. It's not as organic as Ruby's open classes, but gives you a much better control towards evolution of APIs as your application codebase scales up. Let's see how ..

I love to program in languages that offer extensibility of abstractions - no, not the inheritance way .. public inheritance is possibly the second most tightest coupling between abstractions (guess what the first one is .. correct! .. the friend class in C++ ..). Scala offers extensibility even to classes written in Java, and with a rich repertoire of features. It almost obviates the necessity of plugging in your favorite Dependency Injection framework. And implicits offer features to add extensions to existing class structures.

Multidimensionally Open Classes ..

The core APIs that any class exposes abstract its prime responsibility, which are invariant across the entire lifecycle of the domain. This is the minimalist view of designing a class (when in doubt, leave it out !). Whether the large Array class of Ruby is the ideal approach of designing an abstraction, has been beaten to death. But definitely it is the result of a mindset that offers open classes - if you do not put assoc in Array, someone else will.

When we consider extensibility of abstractions, typically we want to extend an abstraction in multiple dimensions. A core domain object often needs to be extended with behaviors that help developers design smart APIs. But, more often than not, these behaviors make sense in a specific context of the application and may seem irrelevant or redundant in other contexts. In John Carter's earlier example, extending Integer with factorial may make sense when you are designing smart APIs for mathematical calculations. But why should an Integer class generally be bothered about computing it's own factorial ? This way, adding all possible math calculations in the global class object, can only lead to a big big bloat violating all good principles of OO design. The extension has to be in a specific context of the class and should be invisible to any other context, for which it seems like a noise.

Have a look at how Scala handles this wreck effect through a simple example ..

I have a class Person, which is a domain object in the core package ..


class Person(val lastName: String,val firstName: String,val age: Int) {
    //.. details

    override def toString(): String = {
        lastName + " " + firstName + " " + age

While working with my UI classes, I would like to have an API person.toLabel() which will display a JLabel Swing component for rendering a person's details on a frame. And while working on the messaging part of the application, I would like to have a person.toXML(), which will help me generate an XML message out of the Person object. But obviously I would not like to have any javax.swing imports in my messaging component.

I would like to extend the same core abstraction (Person), but along two mutually orthogonal dimensions in a share nothing mode. The messaging component should not be able to invoke person.toLabel(), even though it is the same core abstraction that it is extending. The extensions have to honor the context in which they are being used. Then only can we ensure proper separation of concerns and the right modularity in designing application component boundaries.

In Ruby, class objects are global variables, and indiscriminate extension methods plugged into existing classes can lead to reduced maintenability and long term reliability of class structures. Not with Scala *implicits* though ..

Within the ui package, I define an enhancer class RichPerson, which provides all extensions to my Person class *only* for the UI context ..


import javax.swing._

class RichPerson(person: Person) {
    def toLabel(): JLabel = {
        new JLabel(person.toString)
    //.. other UI context extensions

and define a mixin that provides me the implicit conversion ..



trait UIFramework {
    implicit def enrichPerson(person: Person): RichPerson = {
        new RichPerson(person)

When I write my UI components, the extension designed for the UI context kicks in and serves me the UI-only view of the extended core abstraction. I can use the nice syntax on the core abstraction itself, the compiler does the magic underneath through invocation of the conversion function ..


import java.awt.event.WindowAdapter
import javax.swing._

object Main extends UIFramework {

    def main(args: Array[String]) = {
        val p = new Person("ghosh", "debasish", 35)
        val frame = new JFrame()
        frame.addWindowListener(new WindowAdapter(){})
        frame.getContentPane().add(p.toLabel) // nice person.toLabel() usage

Trying to use person.toXML() will result in a syntax error ! In fact the IDE can only display the available extensions as part of auto-completion features.

Similarly for the messaging component, I define the following extensions in the messaging package ..



class RichPerson(person: Person) {
    def toXML(): scala.xml.Elem = {

and the mixin ..



trait MessageFramework {

    implicit def enrichPerson(person: Person): RichPerson = {
        new RichPerson(person)

and my messaging application code ..



object Main extends MessageFramework {

    def save(person: scala.xml.Elem) = {
        scala.xml.XML.saveFull("person.xml", person, "UTF8",
                true, null)

    def main(args: Array[String]) = {
        val p = new Person("ghosh", "debasish", 41)

and with Controlled Visibility ..

We extend our Person class in different dimensions for two orthogonal concerns and both of the extensions are mutually exclusive of each other. We cannot access the UI extension from within the messaging part of the application and vice versa. The visibility of the *implicit* conversion functions, along with Scala mixin techniques, ensure that the UI component *only* gets access to the RichPerson class meant for UI extension. Hence the global namespace does not get polluted and the class structure does not get bloated. Yet we have the power of open classes.


Germán said...

I like it. What happens if you need to mix in both traits? The conversion methods have the same signature (only different return types).
I'll see if I can convert myself into a RichPerson!

abas said...

Let's say we have a class Student that is a subclass of Person, extension class RichStudent which is a subclass of RichPerson. Is it possible to determine the proper extension class if the declared type is Person, but the actual type is Student?

Anonymous said...

"You really never know what has been added to a class and whether the new implementation that you are plugging in, breaks apart any of the methods added by another developer"

It doesnt matter for several reasons:
- my own code and time is worth more than the code of other people
- IF there is a conflict like this, I would blame the other developer (if not I did a mistake). After all, he should ensure that certain things _should_ be possible or not.

Can anyone think of a reallife example to this? I never encountered a problem with that in my 4 and a half years with ruby

Debasish said...

In a sense, implicits are treated as overloaded functions. If there are multiple implicits applicable for a conversion, the Scala compiler will apply overloading resolution to pick a most specific one. If there is no unique best conversion, an ambiguity error results.

Debasish said...

Sure .. if you have your classes properly declared, then you can do the following :

val s: Person = new Student("ghosh", "debasish", 41, 56)

where foo() is the extension method added to RichStudent.

Debasish said...

"my own code and time is worth more than the code of other people"

Unfortunately you need to care about other's code as well when you are dealing with the global namespace. As I mentioned in the blog, the problem is NOT the openness of classes, but the globality of the openness, which has all the evil sideeffects of global variables.

Germán said...

This is what I meant:

scala> class A { def doA = println("aaa") }
defined class A

scala> class B { def doB = println("bbb") }
defined class B

scala> implicit def c2x(c:Char): A = new A
c2x: (Char)A

scala> implicit def c2x(c:Char): B = new B
c2x: (Char)B

scala> 'e'.doB

scala> 'e'.doA
console>:12: error: value doA is not a member of Char

Both implicit conversions have the same signature and same name, the result is that the first one was overwritten.
So it's probably a good idea to make the enrichPerson() names more specific so that they don't clash when mixing both traits in.

Chris said...

I found the article interesting, but I fail to see what is the advantage of mixins above subclasses in this particular case.

In Java you can achieve the same result with a UiPerson which would subclass Person and add toLabel, and MsgPerson again a subclass but with method toXml.

Also, traits exist to add similar behaviour to different places in hierarchies but in this example aren't the traits tailored for Person only? Your traits import Person! A sign that subclasses would be more appropriate in this case, don't you think?

Debasish said...

I think you r confusing 2 aspects of the solution :-

Firstly traits and subclasses are different. In Java u can have only one "extends" - u can implement multiple inheritance through multiple "implements" of an interface. But again, you cannot provide any implementation within an interface. Traits give u the best of both worlds. You can mixin multiple traits with one class and as well can provide default implementations within the traits definition.

Regarding the other observation (UiPerson and MsgPerson), the point of the post is to be able to use the extension methods on the core class itself. If u subclass, you need to write, where foo() is the new method which u add to the subclass. The beauty of extension classes is that u need to define an implicit conversion only once and then u can use the extension methods on the core abstraction itself. Have a look at my earlier post how using extension methods we can decorate Java objects as well.

Chris said...

I know the difference between traits and subclasses. What I didn't see was any advantage of using traits over subclasses in this particular example.

So the advantage is that you can use the same Person class in both UI and MSG packages. Okay, fine, why not.

You didn't reply to my last question. I think the traits are tailored for the Person class only. And when a trait can only be "included" (excuse me the Ruby way of saying it) in one class, why is it still a trait? It should be a subclass instead IMHO.

Another way to put it: is there any class apart from Person that could benefit from the UIFramework's toLabel method?

Debasish said...

"is there any class apart from Person that could benefit from the UIFramework's toLabel method?"

The trait UIFramework is not specific to Person or RichPerson class. What it provides is a namespace which allows us to segregate extensions. This is an area where Scala shines wrt Ruby, where all extensions to a class are on the global namespace. One module of the application may define it's extensions in trait UIFramework, while another module defines its extensions in trait MsgFramework. Note in Scala in order to define a class X as an extension to class Y, you need to define an implicit conversion function - the traits only provide you the enclosing namespace. It is not mandatory to have the conversion functions inside a trait - I have done that intentionally to provide a namespace based separation. Finally when you define your application class, you just mixin the traits relevant to your modules. This way you avoid name clashes with other modules' extensions, which I think is a definite advantage. e.g. we can have the following definitions ..

trait UIFramework {
implicit def enrichPerson(person: Person): RichPerson = {

implicit def enrichInvoice(invoice: Invoice): RichInvoice = {

implicit def enrichList(..): RichList = {

Similarly we can have a different set of extensions in other traits. Had all extensions been defined globally on the same class (as in Ruby), there is a chance that names will clash and functionalities may conflict. In Scala, we can include only the required extensions by mixing in the appropriate trait with the main class. And the advantage of using traits as opposed to subclassing is that you can mixin many of them depending on your requirements.

Germán said...

The suggestion to make it a subclass of Person doesn't look good to me for several reasons.
One is that you may need several extra behaviors together, and you cannot extend more than one class, whereas you can mix the traits in.
Another one is that good OO design mandates that a subclass is a specialization of the superclass, for example a natural hierarchy would be Person <- Employee. UI support is orthogonal to this hierarchy.
And finally, consider that you might be dealing with a Person object that was returned by a service layer (which of course must not bother about UI issues). If UIPerson was a subclass you would have to create a new UIPerson from that Person's data. Having the UI behavior mixed in, with the help of an implicit conversion (also called "view"), you just deal with the Person object!

tkr said...
This comment has been removed by the author.
tkr said...

Is it necessary to make the UIFramework a trait? If UIFramework is a singleton object, we have namespacing too and we don't have to extend the trait.

Debasish said...

The singleton object is like a static reference. With traits, you can easily have implementation inheritance, and have your classes / objects mixed in.

Stephan.Schmidt said...

I like implicit views, but from the doA/doB example we see that views have the same problems as multiple inheritance. One needs to be very careful what one does.


roger said...

wow ruby reopening style with static. It's genius!