The Compounding Value of Programmers & Processes

In reaction to No Silver Programmers, several people have spoken to the compounding benefits of good programmers (or the compounding pain of bad programmers). This is an excellent point. I think it’s best put in this blog post, which takes the focus away from the developers and speaks of the quality of the codebase and the ease with which changes can be made.

[N]o one ever talks about code base productivity. If we do, where does it lead us?…[D]oing many common tasks in this code easily took 3x longer than necessary….[S]o bad that I was almost giddy whenever I could work on the new-style code. I’m sure you know what kind of code I’m talking about. Doing anything in this code easily took 3x even longer….A good code base makes any programmer more productive….

Maybe silver programmers don’t exist. But is there “silver code”?

[Valued Lessons]

The author thinks “yes” and I agree. I alluded to this in my original post, but consider this curve:

image

This curve — the cost of a change in software increases exponentially over time — was gospel in the world of software development until less than a decade ago. It was absolutely accepted that, essentially, as the artifacts of software development (requirements, documentation, and, of course, source code) grew in size and complexity, the cost of making a change became ever-higher. In other words, as value grew in more-or-less linear form (one more feature), complexity grew in a non-linear form (like compounded interest).

About a decade ago, Kent Beck said “What if that assumption is wrong? What if we could have a curve like:

image

How would our processes and artifacts have to change to make that come about?”

And his conclusion was “Yes, we can achieve that second curve,” by the processes that he called “Extreme Programming.”

(One of the reasons why I was so heated in “No Silver Programmers” is precisely because the canard of “5% 20x productivity” not only accepts but exaggerates received wisdom. Just as Beck changed the industry by questioning the received wisdom of exponential cost increases, we cannot afford to perpetuate the myth of super-programmers.)

The way to achieve that second curve (which we can call the “agile cost curve”) is, ultimately, to have a “silver codebase.” (Well, a silver “artifact” base, because you have to do documentation and training and all of that stuff.) If the complexity of your codebase is growing non-linearly, no programmer (no matter their linear multiple of base productivity) can give you that second curve.

image

The point is that no heroic performance can change the shape of the exponential rise in cost over time. Tool choices don’t change the shape either. What changes the shape of the rise in cost over time are practices.

So, the question becomes “How do you develop and maintain a silver codebase?” And it’s widely believed that we have some answers to that:

  • Refactoring (continual rewrites of the codebase rather than simple accretion)
  • Incremental delivery with shorter timeframes (feedback and validation rather than the risk of diverging concepts)
  • An emphasis on quality during development (increased cohesion, decreased coupling, improved project guidance)
  • Preference for working code over documentation

Of course, practices are dependent on people — all the people on the team. To me, a good programmer is one who contributes to the practices that maintain a “silver codebase” — no matter how quickly they write code! Bad programmers are those who don’t care about the quality of the codebase, either due to:

  • Arrogance (“Oh, that’s not a problem. I can fix that in a couple minutes.” “I just pulled an all-nighter!”)
  • Lack of discipline (“Works on my machine!” “I’m done when I see my feature run the first time”)
  • The wrong temperament (delete a loop to hoist a variable)

In addition, the quality of the programmers contribute additionally to the interpersonal dynamics of the team, likelihood of a disastrous miscommunication, etc.

So, the interesting question then becomes “How do we maintain that second curve in the face of bringing in new developers?” In recent posts relating to programming talent, I know I’ve come across as a hard-ass (even though I’m a sweet guy, love my dog, wave people in at intersections, etc.). That’s because I’ve reluctantly concluded that the “agile cost curve” is not stable in the face of poor hires.

In an agile team, there is a great emphasis on individual responsibility. We all learn new practices. We all make mistakes. We all break the build. In such a team, an irresponsible developer (not a slow developer, not an inexperienced developer) can subvert the entire process. And the result, in my experience, is that the “agile cost curve” can jump right back to the classic, exponential cost curve.

Bad programmers are not good programmers who are slow. They are actively counter-productive to the team.

Tina’s fine

She’s got to get a CT to confirm, but basically the Doctor told her “no problem.” So ends probably the least-productive week of the past several years…

My sincere thanks to everyone who got in touch and kept us in your thoughts. It really helped.

Sun Buys MySQL

My initial reaction to this is fear that Sun will try to force things a little too hard. When I think of MySQL, I think of ease-of-use and reliability. When I think of SQL Server and Oracle, I think of the ancillary tools and the fierceness with which DBAs cling to their specialized knowledge. DBAs don’t have the programmers predilection to throw aside their tools for The Next Big Thing.

Kicking Around the Monads

Actually, just a brief post. LINQ has injected into the mainstream a whole range of functional programming topics previously seen as esoteric. Putting aside the merits of each of them, the interesting dynamic to watch will be if these approaches generate a new sub-niche of programming. For instance, C++ templates led to template metaprogramming, just such a sub-niche. To what extent will awareness of lambda functions and monads be integrated into mainstream awareness and to what extent will they exist as a self-perpetuating “eddy” in the mainstream? (Or will functional programming actually spread and come to be a dominant way of thinking about software construction in the same way that objects did? I still view it as unlikely, but not nearly as low-chance as I did a few years ago.)

Bad Programmers Are Not Good Programmers Who Are Slow

In response to “No Silver Programmers,” a commenter points out:

Say I’m a pretty good developer and there’s this guy who is 5x worse than me, meaning it takes him a full work week to finish what i will finish in a day.

but then, what happens when it has 3x as many bugs, and it takes him 5x the time to fix each one?   suddenly we’re at 15x for the bug fixing phase.  and actually, it should take him more than 5x as long to fix each one — fixing bugs is much harder the more there are in the same code, and harder the longer it the time between writing them and finding them.  (and think about how much more time occurs between writing a bug and fixing one for this guy compared to me — he has 3x as many bugs, often blocking other bugs from being found and always blocking other bugs from being fixed, and he takes 5x as long to fix each one…)

and it doesn’t scale linearly when we look at code that will take me a month.  that’s going to take him more than 5 months.  his design mistakes will pile up and his buggy code will pile up.  something that takes me a year he may simply be incapable of doing.

Yes. Further, a team with him on it will require more coordination than one made of average-or-better developers. A 90% finished system will suddenly reveal a crucial weakness because of his work violating encapsulation or somesuch.

Bad programmers are not good programmers who are slow. They are actively counter-productive to the team.

This is another great point why, to improve your team’s productivity, it’s important to focus not on the hope that you can discover a magical super-programmer but on rooting out incompetence.

No Silver Programmers

Someone who should know better gave a commencement address based on the premise “5% of programmers are 20x more productive than the other 95%.” This is utter BS and it’s important to say so.

First, as boring as it may be to say “we don’t have the data,” go to scholar.google.com and find me a peer-reviewed study of individual programmer productivity among professionals not students. I’ll wait.

What did you find? 1992‘s Watts Humphrey study? DeMarco’s 1989 references to Coding Wars (the source for Peopleware and the likely source of the misquote in the commencement)? Maybe 1981‘s work by Barry Boehm? The literature sucks. Anyone, including me, who tells you anything about software development productivity is telling you beliefs, not science. Understanding that is vital: the entire agile revolution is based on revisiting cost assertions taken as gospel. “What if the cost of a change didn’t increase over time?” That question is why the practices followed by the best software developers today have little in common with the practices of ten years ago.

So, accepting that everything anyone says about this is anecdotal…

Do programmers vary in their productivity? Absolutely. Is there a small percentage of programmers who are very much more productive than average? Absolutely. Are they 20x more productive than those not in their ranks? Absolutely not. The numbers quoted in Peopleware strike me as most realistic: the best are near 3x median and 10x worst. In other words, the significant thing is not that some professional programmers are awesome, it’s that some professional programmers suck.

Doesn’t that explain your world better? Have you ever met someone 20x better than you? Seriously, a day to do what you could produce in a month of undistracted, phones-off, heads-down programming sessions? Of course not. But have you met someone who’s 5x worse than you? Who you say “It took you a week to do this?” I’m going to guess yes.

That incompetents can stay in the profession is not nearly as mysterious and fun as a secret society of magical programmers. It’s a lot more gratifying to think “I’ve never met anyone 20x better than me, so therefore I’m part of the elite,” than “What does it mean that Bob and I have similar titles?”

Programmer productivity is essentially code creation: develop a function that satisfies specification X. I love code creation; I used to run a newsletter called “Those Who Can, Code.” I write software to help me deal with stress. But code creation is not what spells the difference in software development productivity, for which I absolutely believe that teams can achieve a significant multiple (perhaps 10x) over median productivity.

Software development productivity is the rate at which you deliver value to the client. If good data on the productivity of code creation is lacking, good data on software development productivity is essentially non-existent. It’s extraordinarily difficult to measure. Measuring satisfaction is an insufficient proxy, because satisfaction will tend to be a delta from the last experience, not an absolute.

There are only a few things we can say with certainty about software development productivity:

  • We don’t generally do a great job at it
  • Processes can improve it, if the processes are a good fit to the team
  • Tools can improve it, if the tools are a good fit to the team
  • It is always a people issue

Now I’ve come full circle to agree with the conclusion of the commencement address.

What does the actual sparse data say? This is the graph from DeMarco’s Coding Wars paper, the graph that is generalized in Peopleware (where the probably-significant shading of COBOL entries is missing):

image

(Remember “good” is to the left, not the right!)

Pretty shaky foundation for the whole damn super-programmer myth, wouldn’t you say?

I’m Physically Sick With Fear

Tina was diagnosed with leukemia when she was 29 and we were newlyweds. Two years ago, she was diagnosed with breast cancer. I got home from a business trip to find out that her routine dental x-rays had led to a referral to a dental surgeon because two things might be cancerous. One a shadow, the other white like another tooth. “With your history, just to be sure,” the dentist said, but didn’t offer a “but there are lots of other things that are more likely.”

You Google “jaw cancer” and you hear about oral cancers in general — bad — or osteosarcoma — bad.

She sees the surgeon on Friday.