Knowing .NET

A team, unaccustomed to but enthusiastic about moving towards agile methodologies, begins an important project. The project has many facets and a number of strong developers, so it seems natural for the developers to concentrate on a single aspect: Adam is associated with the Widget feature, Barbara focuses on the Sprocket, Charlie with the Doohickey, etc. Charlie is the most familiar with the domain, the legacy codebase upon which the project is being built, etc.

The project is scheduled to last 36 weeks, divided into 12 3-week sprints. At the end of Sprint 8, Peter the Product Owner is satisfied with the feature set of the Sprocket, Doohickey, and other facets, but it has turned out that the Widget feature has been more complex and is clearly going to be the focus of the remaining sprints. Further, Peter has a number of additional features that he’d like to see in the finished project, if possible.

Charlie says “Well, I can add these new features as requested by Peter, but not using Adam’s code, which doesn’t capture several important domain concepts needed to rapidly develop them.” Adam says “These new features are outside the domain I’ve worked in. I think my code is fine, but I cannot guarantee where it will be in the few remaining sprints.” Adam’s code is shelved, the team realigns into a more traditional lead programmer structure with Charlie “doing the hard parts.” At the end of Sprint 12, the project moves across the finish line with an acceptable level of quality.

Would you say that this project was a success because “It delivered customer value on time; it discovered a problem and course-corrected, the switch into a non-agile mode for a short period is fine, like a 2-minute drill in football.”? Or would you say that the project was a failure because “It relied on ‘superhero’ efforts from Charlie at the last minute; it didn’t identify that the Widget feature was not coming together properly and 24 weeks of Adam’s efforts did not go into production.”?

What improvements to methodology and team structure could be made for future projects? Should the team structure themselves more along the lines of a Lead Programmer model (Charlie is clearly the most productive developer) or less (the argument being that the feature-focused structure distributes credit and blame unfairly)?

Advice From An Old Programmer — Learn Python The Hard Way, 2nd Edition.

Programming as a profession….can be a good job, but you could make about the same money and be happier running a fast food joint. You’re much better off using code as your secret weapon in another profession.

People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines.

True that.

According to this release, Computer Science majors had the highest percentage of job offers of any major.

I have to admit to being somewhat surprised at this, as I thought the trend toward offshore development was still driving down the market for young developers.

If you are interested in high-performance programming on Windows, you know the name Joe Duffy, whose book Concurrent Programming On Windows is absolutely top-notch.

Today he posted an intriguing notice on his blog “We are hiring.” Check out some of the things he says:

My team’s responsibility spans multiple aspects of a new operating system’s programming model…. When I say languages, I mean type systems, mostly-functional programming, verified safe concurrency, and both front- and back-end compilation….All of these components are new and built from the ground up.

Huh. I’ve argued before that the manycore era requires a fundamental break in OS evolution. Every aspect of the machine has to be rethought; the fundamental metaphor of a computer as a von Neumann machine with, perhaps, a phone line to the outside world has been strained to the breaking point. Forget “the cloud,” we need to think about “the fog” — a computing system where every resource (including resources outside the box at which you happen to be typing) can be accessed concurrently, securely, and virtually.

I don’t think that the OS for the manycore era can evolve from any existing desktop OS. That’s why I think that the “Windows 7 vs. OS X vs. Linux” debates are short-sighted and even the “Windows vs. iOS vs. Android” debates are only skirmishes to determine who has the money, mindshare, and power to eventually win the real battle.

It needs to be said that Microsoft has lots of incubation and research projects whose results either are left to wither or are watered-down and incorporated into mainstream products. But the involvement of a top non-academic thought-leader makes me hopeful that Duffy’s project may have a bright future.

 import sys import clr sys.path.append("c:\\program files\\Microsoft Research\\Infer.NET 2.2\\bin\\debug") clr.AddReferenceToFile("Infer.Compiler.dll") clr.AddReferenceToFile("Infer.Runtime.dll") from MicrosoftResearch.Infer import * from MicrosoftResearch.Infer.Models import * from MicrosoftResearch.Infer.Distributions import *  firstCoin = Variable[bool].Bernoulli(0.5) secondCoin = Variable[bool].Bernoulli(0.5) bothHeads = firstCoin & secondCoin ie = InferenceEngine() print ie.Infer(bothHeads) --> c:\Users\Larry O'Brien\Documents\Infer.NET 2.2>ipy InferNetTest1.py Compiling model...done. Initialising...done. Iterating: .........|.........|.........|.........|.........| 50 Bernoulli(0.25) 

Sweet

The July 08 (Vol. 30, #7) IEEE Transactions on Pattern Analysis and Machine Intelligence has an incredible paper by Raykar, Duraiswami, and Krishnapuram. A Fast Algorithm for Learning a Ranking Function from Large-Scale Data Sets appears to be a game-changer for an incredibly important problem in machine learning. Basically, they use a “fast multipole method” developed for computational physics to rapidly estimate (to arbitrary precision) the conjugate gradient of an error function. (In other words, they tweak the parameters and “get a little better” the next time through the training data.)

The precise calculation of the conjugate gradient is O(m^2). This estimation algorithm is O(m)! (That’s an exclamation point, not a factorial!)

On a first reading, I don’t grok how the crucial transform necessarily moves towards an error minimum, but the algorithm looks (surprisingly) easy to implement and their benchmark results are jaw-dropping. Of course, others will have to implement it and analyze it for applicability across different types of data sets, but this is one of the most impressive algorithmic claims I’ve seen in years.

Once upon a time, I had the great fortune to write a column for a magazine on artificial intelligence and could justify spending huge amounts of time implementing AI algorithms (well, I think I was paid $450 per month for my column, so I’m not really sure that “justified” 80 hours of programming, but I was young). Man, would I love to see how this algorithm works for training a neural network…

I recently wrote an encomium to ResolverOne, the IronPython-based spreadsheet:

[T]heir use of pair programming and test-driven development has delivered high productivity; of the 140,000 lines of code, 110,000 are tests….ResolverOne has been in development for roughly two years, is written in a language without explicit type declarations, and is on an implementation that itself is in active development. It’s been brought to beta in a credible (if not downright impressive) amount of time despite being developed by pairs of programmers writing far more lines of test than application. Yet no one can credibly dismiss the complexity of 30,000 lines of application logic or spreadsheet functionality, much less the truly innovative spreadsheet-program features.

ResolverOne is easily the most compelling data point I’ve heard for the practices of Extreme Programming.

[Extreme Program, SD Times]

Allen Holub sees the glass as half-empty, writing:

I want to take exception to the notion that Python is adequate for a real programming project. The fact that 30K lines of code took 110K lines of tests is a real indictment of the language. My guess is that a significant portion of those tests are addressing potential errors that the compiler would have found in C# or Java. Moreover, all of those unnecessary tests take a lot of time to write, time that could have been spent working on the application.

I was taken aback by this, perhaps because it’s been a good while since I’ve heard someone characterize tests as evidence of trouble as opposed to evidence of quality.

There are (at least) two ways of looking at tests:

1. Tools for discovering errors, or
2. Quality gates (they’re one way — are they quality diodes?)

There’s no doubt that the software development tradition has favored the former view (once you’ve typed a line, everything you do next is “debugging”). However, the past decade has seen a … wait for it … paradigm shift.

The Agile Paradigm views change over time as a central issue; if it were still the 90s, I would undoubtedly refer to it as Change-Oriented Programming (COP). Tests are the measure of change — not lines of code, not cyclomatic complexity, not object hierarchies, not even deployments.

(Perhaps “User stories” or scenarios are the “yard-stick” of change, tests are the “inch-stick” of change, and deployments are the “milestone” of change.)

So from within the Agile Paradigm / COP, a new test is written that fails, some new code is written, the test passes — a one-way gate has been passed through, progress has been made, and credit accrues. From outside the paradigm, a test is seen as indicative of a problem that ought not to exist in the first place. The passing of the test is not seen as the salient point, the “need” for (i.e., existence of) the test is seen as evidence of low quality.

In true test-driven development, every pass fails at least once, because the tests are written before the code. What is perhaps not appreciated by those outside the Agile Paradigm, however, is that tests are written that one expects to run from the moment the relevant code is created. For instance, if one had fields for sub-total, taxes, and total, one would certainly write a test that confirmed that total = sub-total + taxes. One would also certainly expect that test to pass as soon as the code had been written.

As is often the case with paradigms, often just realizing that there are different mental models / worldviews in play is crucial to communication.

Update: This relates to Martin Fowler’s recent post on Schools of Software Development.

StartKey will be a technology that allows you to carry your Windows logon around on a USB keychain. Early reaction is mixed as to the value of this, but I loved something similar when I worked for a company developing software for Sun JavaStation network computers.

With JavaStation’s, you had a smartcard that you plugged in and, after 10 seconds or so, up would come your desktop. Since most of the time you work at your desk, most of the time this was not particularly valuable. But let me tell you — it was fantastic for meetings and presentations. No messing around with cables and display settings, no hand-waving when trying to describe an issue you were talking about when you happened to be on the other side of the office.

The difference is that the JavaStations were uniform hardware, too, and all your software lived on the server (which, it turned out at 7AM the morning of a major trade show, is a single point of failure). While you might have a good experience assuming that a random machine has Office on it (a smile creeps across Microsoft’s face), there would presumably have to be a solution for specialist software such as Visual Studio or Photoshop that could not be assumed to be local.

I would think the problem with that is that although memory sticks are probably getting capacious enough, the bus connection between the memory sticks and the main computer are going to be bottlenecks.

http://barrkel.blogspot.com/2008/02/langnet-symposium-videos-are-up-in-wmv.html

Recommendations :

http://langnetsymposium.com/talks/Videos/1-05 – Lively Kernel – Dan Ingalls – Sun.wmv

http://langnetsymposium.com/talks/Videos/2-01 – Newspeak – Gilad Braha – Cadence.wmv

http://langnetsymposium.com/talks/Videos/3-08 – Cobra – Chuck Esterbrook.wmv http://langnetsymposium.com/talks/Videos/3-09 – Intentional – Magnus Christerson.wmv

http://langnetsymposium.com/talks/Videos/3-00 – IronRuby – John Lam.wmv

http://langnetsymposium.com/talks/Videos/3-03 – Parsing Expression Grammars in FSharp – Harry Pierson.wmv

I was reading PC Magazine’s 25th anniversary issue in which they have the evergreen “what will the future bring?” essays. I was struck by how much talk of medical stuff (nanobots, non-invasive diagnosis, ubiquitous this-and-that) there was. And then it struck me:

Boomers.

Just as they do with every damn thing, boomers define the mainstream concern as “What does this mean to me?” In the past 25 years (to take PC Mag’s benchmark) it went from work (what is technology about? Business productivity!) to family (what is technology about? HDTVs, Internet predators, and bluetooth-enabled minivans!) and now, of course, it will shift again.

What will technology be about for the 25 years? Getting old.

Just as you wish you’d written a spreadsheet program 25 years ago or Facebook 10 years ago (well, you would have been flushed away in the dot-com bust, but aside from that…), the thing to think about now are the killer applications for aging, whether that’s medical support, post-retirement money management, or Am I Wrinkly Or Not?