Archive for December 2011

Comment Spam: The Voice Of Experience

Author : Effortless Products For nail fungus treatment – Some Practical Guidance (IP: 118.96.251.234 , 118.96.251.234)
E-mail : bhthhyh@jflui.com
URL    :
Whois  : http://whois.arin.net/rest/ip/118.96.251.234
Comment:
When you are starting up a new organization you certainly want to come up with a catchy business title that will be special from other folks out there. An beautiful organization identify will do far more than just make your business stand out from other people.

Thanks for the advice “Effortless Products for nail fungus treatment”!

“Moneycode” not “Developernomics”: My Hunch as to the Distribution of Software Excellence

Part 1: Forbes is wrong about “Developernomics”

Part 2: Why 10x Ticks Me Off

If this subject’s of interest to you, you might enjoy (or despise) the column on the subject I’ve been writing for SD Times for the past decade.

In order to understand my position on why I don’t believe in 10x Superprogrammers, it may help to show my hunch as to how productivity is distributed in the software development world. I think that it is sadly the case that there are bad programmers ranging from utterly incompetent to simply inept and that they form a long tail of widely variant “below median” productivity. But I think that half of all professional programmers are quite good and that a median programmer is a good bit better than average. That is, that the bad professional programmers drag down the average.

In schematic form, my hunch is that the distribution is something like this:

Those who are highly productive face a headwind: increasing complexity, the frustrations of human communication, the limits of our tools, etc. All sorts of things, some of which are accidental, but some of which I believe are fundamental. It may be that at the tail of the distribution, there are some who combine technical genius with interpersonal genius with organizational genius and, I dunno’, if they decide not to become Evil Geniuses, maybe they stay in software. But I do contend that such folk are vanishingly rare and one ought not base one’s company’s strategy on finding them.

I believe that the realistically-findable excellent programmers are perhaps 2-3x as productive as median programmers. I don’t see that view as dismissive of their abilities in any way; a running back need not score 10x the touchdowns of a median NFL running back to be considered great, nor a batter to bat in 10x the median number of runners-on-base to be a star.

I feel that when an excellent programmer is felt to be much better than that multiple, it’s almost certainly context-sensitive. It’s highly likely that they are doing, in their excellent way, a task that they are familiar with: they are working with a codebase they’ve mastered, they are using an architectural pattern they’ve implemented before, etc.

Just as the Superprogrammer 10x distribution implies certain things for industry, so too does this distribution:

If developer productivity is Big O(context), then today’s average programmer might be tomorrow’s excellent programmer

Ditto for team productivity

If excellent productivity is a fairly small multiple of median productivity, then one’s hiring should focus on “above median” not “superstar”

If excellent productivity is context-sensitive and a fairly small multiple, investing in changing near-median developers into excellent developers is much more cost-effective than seeking Superprogrammers and paying them premiums

If there’s a long tail of below-median developers, the personnel strategy that one must embrace is ensuring that below-median developers are removed from the team

Although a circumstantial argument, this model fits better the industry I see. I think of it as the “Moneyball” model of software development excellence.

The Passing of Pioneers

The passing of pioneers – SD Times: Software Development News.

SD Times has published some brief memories of some of those the industry lost this year.

Why 10x Ticks Me Off

This is a follow-up to my Friday rant Forbes is wrong about “Developernomics”

First: If the subject’s of interest to you, you might enjoy (or despise) the column on the subject I’ve been writing for SD Times for the past decade. (I can’t help but wonder if I wrote in my normal, less-heated mode if I’d have had any chance of getting picked up by hackernews.)

Luckily, though, this is a subject that really sets me off, so I don’t have to manufacture passion. I get angry because the Myth of the Superprogrammer has lots of important consequences:

IF programming excellence is Big O(inherent talent) THEN that should guide developer careers.

IF excellent programmers are 10x median  productivity THEN that should guide software team structure.

Organizations will tend to structure their software teams according to the model of development excellence held by senior management.

So I don’t think this is a harmless matter of opinion, a “my language is better than yours” bar-room perennial. The Myth of the Superprogrammer affects your career, mine, and that of all of our colleagues.

It’s important that I reiterate that I fully understand that arbitrarily-high productivity multiples occur in any number of contexts: from the trivial “I’ve been working in this language / library / codebase and you haven’t” to the complex “this project was rescued.” (And since “the project was rescued” implies the uncommon but not rare situation of zero or negative productivity, the advantage is arbitrarily high.) The most interesting context to me, is the situation of  ”Tim in Colorado”, a commenter on my previous post, who was the original architect and implementer of a game engine. It may be that in a situation like that, transferring the mental model, decisions, tweaks, etc. of the codebase is so difficult that he will forever hold that 10x advantage.  (Although I’d point out that from an organizational perspective, such a situation is not necessarily wonderful: they have a bus factor of 1, they may trouble freeing up Tim to work on new projects, etc.).

But saying “In some contexts, Adam is 10x more productive than Charlie,” is not the Myth of the Superprogrammer. The Myth of the Superprogrammer is “Adam’s great advantage over Charlie is driven by an inherent advantage Adam has over Charlie. Adam is rare, Charlie is a commodity.”

Let’s be clear about what “10 times median productivity” means. It means this:

The claim is an order of magnitude: In one day they produce a sprint’s worth of functionality. In a month, they do a year’s work. In a year, their contributions oustrips the combined output of nine competent colleagues. They can master a new system library not in 6 months but in 3 weeks, they never walk more than, say, 15 minutes going down a wrong path, they never lose days waiting for an answer from a domain expert, they never lose the work of weeks to a changed or misconstrued requirement. They are The World’s Most Interesting Programmer. Except such people not only exist, they’re not vanishingly rare! There are lots of them: you should strive to hire such a paragon. Everyone else is hiring them! Why not you?

Whoops, there I go starting to get emotional…

And I’m not being unfair in my characterization of what “10x productivity” means. It has to mean such things, because that’s what programming productivity is: delivering value to the customer. It’s not writing code or debugging code or refactoring code: those are simply the means.

Sure, way out on the tail end of the distribution of the several million professional developers, there are some flat-out programming geniuses. And maybe if you squeeze down the tail far enough you can find some who combine flat-out programming genius with collaborate-with-users genius.  But the Superprogrammer Myth requires a high variance: there have to be enough developers who achieve 10x median productivity so that they regularly make an appearance in industry.

Let’s say that’s correct. What does this distribution imply?

It implies that hiring is overwhelmingly important, that it is better to be short-handed for long periods of time rather than fill the spot with a person who everyone agrees is clearly better than median. God forbid you hire a person who’s only twice as good as median.

It implies that experience with the tools and libraries that your company uses is entirely irrelevant: a 10x-er will pick them up 10x faster than median and going forward, of course, will be 10x more productive with those tools. (To be fair: I do actually believe that “you cannot teach tall” and that “x years with tool y” is a terrible thing in a job requirement. But I also believe that this is a significant trade-off.)

It implies that you should have an enormous variance in your compensation structure. If Charlie is a 10x-er and Adam is median, Adam should be compensated far less; his loss is unimportant to the productivity of the team, while Charlie’s retention is utterly critical.

It implies that median salary should be stable or decrease over time, since the abilities of a median or “only” quite good programmer do not drive overall team productivity. Quite good developers can be replaced with a cheaper quite bad programmer, as long as the team has a Superprogrammer whose 10x contribution drowns out such small signals.

It implies that you should structure your team as a Chief Programmer Team: everyone serves the needs of the 10x-er. Keeping the Superprogrammer loaded is the goal of every member. The Superprogrammer ought not to be required to explain or collaborate on important decisions.

It implies that the comings and goings of the Superprogrammer have an enormous impact on the company’s productivity: the Superprogrammers arrival coincides with a massive uptick in productivity, the Superprogrammers departure creates a massive loss of productivity until the Superprogrammer is replaced.

Those implications do not describe the industry that I know. They describe a TV show.

Part 3: Moneycode vs. Developernomics

Forbes is wrong about “Developernomics”

Forbes is wrong when it says:

The thing is, software talent is extraordinarily nonlinear…It’s still a kind of black magic…the 10x phenomenon, and the industry’s reliance on it, doesn’t seem to get engineered or managed away. Because the 10xers keep inventing new tools for themselves to stay 10xers…

This is folklore, not science, and it is not the view of people who actually study the industry.

Professional talent does vary, but there is not a shred of evidence that the best professional developers are an order of magnitude more productive than median developers at any timescale, much less on a meaningful timescale such as that of a product release cycle. There is abundant evidence that this is not the case: the most obvious being that there are no companies, at any scale, that demonstrate order-of-magnitude better-than-median productivity in delivering software products. There are companies that deliver updates at a higher cadence and of a higher quality than their competitors, but not 10x median. The competitive benefits of such productivity would be overwhelming in any industry where software was important (i.e., any industry); there is virtually no chance that such an astonishing achievement would go unremarked and unexamined.

This folklore arises, in part, because it is possible to be arbitrarily more productive than the worst. The software industry is no different than any other in terms of the “Peter Principle” of people rising to their level of incompetence: developers who succeed tend to move on to work on larger, more complex projects and, regrettably, not every developer or development team understands that different techniques are required at different scales. The tools and techniques that suffice for 5,000 lines of source-code don’t suffice for 50,000 and what suffices with 50,000 don’t suffice with half-a-million. The totally competent at one level of complexity may totally fail at the next. One of the depressing realities of being a software development consultant is regularly walking in to these situations, with their broken morale, finger-pointing, and distrust. 10x more productive? Nonsense! You’re dividing by zero!

Sure, there are tasks at which one programmer may well be an order of magnitude faster than his or her colleagues. Being well-practiced in a technique or the use of a component can easily lead to such gains. It’s difficult for non-developers to understand how fundamentally plastic software is; a programmer can deliver a feature in any number of ways and even when there is broad agreement about an overall approach, there is a vast advantage in knowing “these two components turn out to be incompatible” or “that answer is easily provided by this library.”

How do task-based advantages play out over the course of, say, a year? No one knows: I’ve never heard of a study that tried to quantify developer productivity over such a long timescale. The problem of even defining productivity at such a timescale is ample cause for argument. Professional software development is a team sport, and most development managers have experienced a trade-off of a developer with technical skills but less-than-desired soft skills. Conversely, time spent away from the keyboard actually working with users often leads to far more value than would be achieved by coding away and checking later.

None of this is to say that there are not individual differences in productivity. Based on the programmers I’ve met over the course of 20 years of working with software teams, I think that over long timescales, the top 5% of programmers are probably 2-3x as productive as median programmers (and I don’t think anyone in industry is 10x median: industry couldn’t provide sustenance for such a creature). I tend to think that this is a combination of inherent talent, the speed with which they can switch between tasks and techniques, and temperament.

I get stirred up about this folklore because it has major consequences. An industry where excellence is 10x median performance ought to be structured very differently than an industry where excellence is 2-3x median.

Variations in productivity compound: over the course of several release cycles, even in a world without super-programmers, the excellent team can crush the less-the-average (even putting aside the uncommon, but not vanishingly rare, competition that truly has zero productivity or is even in negative making-things-worse mode).

But even if there are real super-programmers, Forbes still is wrong about “developernomics.” The real issue in software development is which of these two charts correctly describes the cost of fixing a defect over time:

The first is the Boehm curve and the second is the Beck curve. The Boehm curve describes the view presented in “Software Economics” and holds that the cost of fixing a defect rises exponentially over time: it is cheap to correct a mistake when sitting around a table gathering requirements, more expensive to fix during initial coding, more expensive to fix late in testing before release, very expensive to fix after release. The Beck curve describes the view presented in “Extreme Programming Explained“: while it’s still cheapest to fix a mistake before a line of code is written, fixing a two-year-old defect in production code is not really *that* much more expensive than fixing the defect 10 minutes after it’s first introduced into code.

There is much to be said about these curves, but the crucial point is this: individual programmer excellence, whether 10x median or 2x, does not drive organizational software productivity. Any team, no matter how talented the individuals, whose costs follow the Boehm curve will, over time, lose to any team, no matter how mediocre the individuals, whose costs follow the Beck curve. It’s just a matter of where the cross-over occurs.

Does industry have teams whose costs follow the Boehm curve? Yes, absolutely. Does industry have teams whose costs follow the Beck curve? I’m not sure that the constant-over-time flatness is realistic, but absolutely teams can bend the curve towards the ideal. Call it “process-nomics” or “team-nomics” or, hell, call it “developernomics” because it is developer-centric. The problem with the Beck curve from the management perspective is that to achieve that curve, you have to give up traditional project management controls. Gantt charts don’t work with agile development. You can’t do fixed-price, fixed-feature, fixed-time budgeting. You can’t sign off on requirements and wait around for teh awesome to appear under the Christmas tree.

Management hates giving up that stuff so much that they would rather continue chasing unicorns and 10x super-programmer teams and magical complexity-free programming models. So now, because of this stupid Forbes article, you’re going to have to explain to your bosses why it’s stupid and off-putting to write a “we’re looking for rockstars!” job ad and why you shouldn’t either jump on the first “ninja” to walk in the door or spend a year filling a position and they’re going to be pissed when you ask the obvious “if you believe this, then which of our developers is going to be paid $700,000 per year?”

And the fact that they don’t structure salary like that? It shows that, not far from the surface, they know its bullshit.

Update: If the subject’s of interest to you, you might enjoy (or despise) the column on the subject I’ve been writing for SD Times for the past decade.

Update: A little less rant-y follow-up post Why 10x Ticks Me Off

Update: Moneycode vs. Developernomics

An Agile Thought Experiment

A team, unaccustomed to but enthusiastic about moving towards agile methodologies, begins an important project. The project has many facets and a number of strong developers, so it seems natural for the developers to concentrate on a single aspect: Adam is associated with the Widget feature, Barbara focuses on the Sprocket, Charlie with the Doohickey, etc. Charlie is the most familiar with the domain, the legacy codebase upon which the project is being built, etc.

The project is scheduled to last 36 weeks, divided into 12 3-week sprints. At the end of Sprint 8, Peter the Product Owner is satisfied with the feature set of the Sprocket, Doohickey, and other facets, but it has turned out that the Widget feature has been more complex and is clearly going to be the focus of the remaining sprints. Further, Peter has a number of additional features that he’d like to see in the finished project, if possible.

Charlie says “Well, I can add these new features as requested by Peter, but not using Adam’s code, which doesn’t capture several important domain concepts needed to rapidly develop them.” Adam says “These new features are outside the domain I’ve worked in. I think my code is fine, but I cannot guarantee where it will be in the few remaining sprints.” Adam’s code is shelved, the team realigns into a more traditional lead programmer structure with Charlie “doing the hard parts.” At the end of Sprint 12, the project moves across the finish line with an acceptable level of quality.

Would you say that this project was a success because “It delivered customer value on time; it discovered a problem and course-corrected, the switch into a non-agile mode for a short period is fine, like a 2-minute drill in football.”? Or would you say that the project was a failure because “It relied on ‘superhero’ efforts from Charlie at the last minute; it didn’t identify that the Widget feature was not coming together properly and 24 weeks of Adam’s efforts did not go into production.”?

What improvements to methodology and team structure could be made for future projects? Should the team structure themselves more along the lines of a Lead Programmer model (Charlie is clearly the most productive developer) or less (the argument being that the feature-focused structure distributes credit and blame unfairly)?

div>