Monday, February 20, 2012

It's the familiarity model!

James Iry recently blogged on code density and tries to find out the meaning of "dense code". As an example he took upon the topic of regular expressions, which despite being dense are not frowned upon in coding and hardly get replaced for making the code fragment more readable.

So the question is what makes code dense so that it's not acceptable to programmers and they complain about it's incomprehensibility ?

Later in the blog post James himself identifies unfamiliarity as one of the culprits. People don't complain about regexes since they are familiar with them, but will surely complain of something else which they are not familiar with.

Almost during the same time Ola Bini blogged about expressiveness in programming language syntax. He mentions that a well designed syntax should help programmers *read* the code easily. But he also questions about the target programmers ..

who is this person reading ? It makes a huge difference if we’re trying to design something that should be easy to read for a novice or we’re trying to design a syntax that makes it easier for an expert to understand what’s going on.

Once again we get into this territory of familiarity and mental model. An expressive piece of code becomes readable only to a person who is familiar with the underlying model. In my programming career I have come across this dichotomy a number of times where programmers complain of something being too dense the moment it crosses the threshold of his familiarity level. I have seen developers taking every pain to understand the nuances of a Spring XML configuration. Or who have spent zillions of hours mastering the whole bunch of performance tuning Hibernate with stuffs like query cache configuration. Believe me it's not simple with tonnes of corner cases to take care of and even today I am not sure if it can be achieved in a deterministic way for all kinds of data models. But these same developers complain when they are faced with maintaining code that needs a basic understanding of functional programming, set theory or algebraic data types. I think it's purely because these form outside the limits of their familiarity model.

For a programmer who is not familiar with higher order functions, combinators like map, fold or filter will look too dense. So when you say map (+1) [1..5], the code fragment looks much less comprehensible to him than his familar variant of using an imperative mutated-indexed for-loop. To the unfamiliar the functional variant appears dense, to the expert it becomes succinct.

One of the challenges that I face today is to make programmers believe that learning new stuff will only help them think better. It's not mandatory that they will need all of these tools as part of their day job. But broadening your mental model can only help your thought process to leverage a wider playground. Maybe in our part of the world big companies give no incentive to transform yourself from a billable offshore resource to a thinking programmer. But you really need to transcend the limits of your familiarity model in order to appreciate code which experts certify as succinct.


Jim O'Flaherty said...

Very nicely put.

From what I can tell, the fundamental issue has to do with misplaced optimization (i.e. efficiency) combined with a psychologically undermining fear based self-suggestion of "what if this next abstraction I've yet to learn is the one that is just beyond my limit of capability". At least that is what I am currently facing as I attempt to learn Scala and Functional Programming after having _invested_ two decades into OOP (Eiffel, Delphi and Java).

Do I spend the time attempting to make the leap to understand Functional Programming with all it's weird foreign and non-OO way of doing things? Example: the term monad has kept me up with nightmares of non-grokking now for almost a year, and that's after reading article after article attempting to explain it. Or, do I leverage what I already know and for what the market is already paying top dollar, or take the risk and time to invest in something that is foreign, it will be years before I am at an expert level again and during that time, I will suffer internal confusion and a confidence crisis as I continue to do OOP in my day job and attempt to do OOP + FP in my spare time?

At this point, I am making the investment to learn Scala and Functional Programming. And I am framing it as an intellectual challenge knowing I might not ever get to leverage either in any real financial way like I have been able to with Java and OOP. And the cost, psychologically to me, has been pretty expensive. I figure I have another year, at least, of being uncomfortably ego-challenging confronted before I finally feel a sufficient level of Scala and FP confidence. So, that's two years I am pushing myself to invest.

So, given that, I can see how many will choose to wait until Scala and FP are much more mainstream before taking their own investment risks, both psychologically and intellectually.

Tavis Rudd (openid only - no blog) said...

As Rich Hickey says, elegance and familiarity are orthogonal. Same goes for density, readability, etc.

Dax said...

I tend to write fairly dense code myself, but before turning in the final version of my code, I always consider who is going to be maintaining it and get rid of some of the clever code tricks that they might have trouble following.

Andrae Muys said...

I can honestly say you rest easy about monads. They are extremely simple concepts, but grounded in a foreign conception of programming. Have no fear, as you accustom yourself to the statically typed FP view of the world, the "problem of monads" will resolve itself.

Terms to pay attention to as they pass your view: Algebras, and Partial-orders; and ponder: "How many concepts we use in programming are partial-orders, and how might you represent this aspect of them algebraically"