Shiny Buckshot Rather Than Silver Bullets

Wes Moise’s musings on Supercompilation led me to this discussion of the the myth of the sufficiently smart compiler.

The “sufficiently smart compiler” is still trotted out regularly, even though the market has moved away from demanding even moderate attention to performance at the compiler level. Have you timed your rectangular arrays in C# lately? Or, to be inclusive, have you looked at what’s (not) hoisted out of loops in Java?

The existence of the Iron* languages from Microsoft stems from Jim Hugunin’s discovery that adding moderate smarts allowed dynamic languages to run fast on the CLR:

  1. Use native CLR constructs whenever possible. These constructs are all heavily optimized by the underlying runtime engine. Sometimes these constructs will have to be used creatively to properly match Python’s dynamic semantics.
  2. Use code generation to produce fast-paths for common cases. This can either be development-time code generation with Python scripts creating C# code or run-time code generation using System.Reflection.Emit to produce IL on the fly.
  3. Include fall-backs to dynamic implementations for less common cases or for cases where Python’s semantics requires a fully dynamic implementation.
  4. Finally, test the performance of every CLR construct used and find alternatives when performance is unacceptable. Type.InvokeMember is a great example a a function that must never be used in code that cares about performance.

That’s hardly the stuff of PhD theses (don’t misunderstand me: Hugunin’s paper, which actually said something important, is more valuable than 99% of CS theses).

The point, though, is that we are in a time of high tension between what is possible and what is practiced. This gives me hope that we might see true breakthroughs in programming languages. Fred Brooks spoke of a silver bullet defined as a “single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity.” [my emphasis]). I don’t believe in silver bullets, but I think there’s a possibility of shiny buckshot.

On the discouraging side, I think there are great difficulties to building such a system: the development of a shiny shotgun is, I think, the work of double-digit person-years. It’s work that’s too far over the horizon for VC funding, too pragmatic for grants, and too dependent on brilliant execution by a small, high-performance team for Open Source.

Bob the "programmer"

I realize that the preceding may, in fact, make me appear to be a hot-head. In fact, I may be a hot-head. But I think if there’s one thing that we should value, it’s our time. I spent an hour trying to “clear the cache” because he uploaded a file to the wrong directory. Never mind continuous integration, he created a new site on the test server (HTF was anyone supposed to know? WTF?). And then when I’d spent my time figuring all this out, I see that his fix wasn’t to move the link outside of the loop, but to delete the loop. Who thinks that way? I mean, seriously, how could anyone with more than 12 hours experience do that?

And, mind you, there are coding “standards” that include automated tests, defect-tracking, and CI. It’s not like this is the first time he’s heard me get angry at the ‘works fine on my machine’ excuse.

…for sufficiently small values of "programmer"

So we’ve got a page that allows clients to edit email addresses. Turns out that if there are no email addresses associated, the “Add an email” link doesn’t show up. I glance at the page and see that the button is inside the loop that’s iterating over the emails (this is ColdFusion, so what this means is that I see something like: <cfloop query=”emails”>…email output stuff … <a href=”./add_an_email.cfm”>Add an email</a></cfloop> (trying to get this system moved over to REST may be covered in another frustrated blog post at a later date)).

So I ask “Bob” to move the link outside the loop.

Twelve hours later:

Larry: What’s the story?

Bob: Fixed.

Larry: Great! *click* Hey, Bob, I still don’t see the link.

Bob: Yeah, ColdFusion must have the old version cached.

Larry: Really? I thought CF was really good about picking up changes.

Bob: Maybe it has something to do with timezones. I’ve seen it before.

Larry: That just seems incredible. You’re sure you made the change?

Bob: Yeah.

Larry: And you tested it?

Bob: Yeah. Works fine on my machine.

Larry: When you’ve seen it before, how long does it take to come online?

Bob: A day or two

Larry: WHAT? That’s incredible. I had no idea. *speed dial* Hey, Carol, reboot the test server, we need to clear the cache.

Carol: What time is it in Hawaii?

Larry: 8:00 PM. Yeah, I guess it’s kind of late there in New York, huh? *click*

— this morning —

*click*  … pause …

*ftp* … double checking …

*svn checkout* … incredulous review …

Larry: Bob, can I have a word?

Bob: How you doing this morning Larry?

Larry: I’m a little stressed out. Let’s try to establish some common ground. Can you FTP onto the site and confirm that the file there is the old one?

Bob: Why’s there only one file there?

Larry: What are you talking about?

Bob: There’s only one file on the site.

Larry: Um … Do you think it’s possible that you uploaded the file to the wrong directory?

Bob: Let me check.

* 5 minutes pass *

Larry: Bob, why’s it taken you 5 minutes to check out the name of the directory to which you FTPed?

Bob: Checking * 3 minutes * Oh, I see! You expected it in /TestSite. I uploaded it to /TestSite2!

Larry: WTF is /TestSite2? No, don’t answer that. WTF did you mean when you said you tested it?

Bob: Works fine on my machine.

Larry: You’re talking about the version 67 in Subversion? Do you have that open on your machine?

Bob: Hold on

* 5 minutes pass *

Larry: WTF is taking you so long? A Subversion checkout takes 5 seconds!

Bob: Hold on * 1 minute passes* Yeah, got it.

Larry: Did you delete the loop?

Bob: Huh?

Larry: You deleted the loop. It looped over email addresses. Instead of moving the link outside the loop, you just deleted the loop!

Bob: OK

Larry: No, it’s not OK! WTF do you mean OK? Every f***ing [original did not have asterisks] client has more than one email address!

Bob: OK

Larry: STOP SAYING OK! WTF were you thinking? No, don’t answer!!! WTF did you mean by ‘works fine’?

Bob: It worked fine on my machine

Larry: NO IT F***ING DIDN’T! YOU DELETED THE F***ING LOOP!

Bob: I didn’t understand what the page did.

Larry: IT WAS A F*** LOOP! YOU”RE SUPPOSED TO BE A PROFESSIONAL PROGRAMMER! YOU DIDN”T “GET” THAT A LOOP WAS NECESSARY? I GOT CAROL OUT OF BED AT 1 IN THE MORNING AND SPENT AN HOUR BETWEEN LAST NIGHT AND THIS MORNING BECAUSE YOU FTPED YOUR WRONG F*** PAGE TO THE WRONG DIRECTORY!

Bob: Control your temper, man.

*Larry’s brain explodes and he dies*

Comments Off As I Deal with dasBlog

My commenting problem has to do with dasBlog validating the viewstate. There is a helpful debugging suggestion in the logs, but my ISP tells me that it is not appropriate to my situation. Until I can spend some time trying to figure that out, I won’t subject people to the frustration of the buggy system. In the meantime, I’m afraid that email (lobrien -at- knowing.net) will have to suffice.

Hypothetically…

You’ve got friends visiting. They’re in town for 9 days. One day involves driving 4 hours to go to a botanical garden which you’ve been to before and flowers aren’t really your “thing” anyway. Do you do it, because, you know, they’re your friends, you’ll enjoy being with them, etc.? Or do you not do it because, you know, it’s a day of travel and affirmatively nodding every time someone says “OMYGOD LOOK AT THOSE LEAVES!”

What do you do?

Just hypothetically, I mean.

OLPC Advocates Python-Based Development

As the OLPC launch approaches, I thought I’d take a look at the development environment. Somewhat to my surprise, the OLPC Development Wiki says that:

  • Technically any language is usable
  • Python is strongly encouraged, to have a single language “under the hood” when the curious child looks inside
  • C/C++ should be used where Python performance is unacceptable, but try to keep it to a minimum, preferably as standard well-encapsulated and documented components
  • Smalltalk-speaking developers may wish to work within the eToys environment
  • Javascript can be used in web-based applications (Gecko or Opera engine Javascript implementation)

I like Python fine, but have to say that I think there’s a real trade-off with it in terms of the world-changing vision of the OLPC. I don’t think that Python is a language that facilitates software engineering and, although it’s easier to learn than a C-derived statically-typed language, as a very-first programming language, I think Smalltalk would be superior in every way.

The trade-off, I suppose, is that OLPC developers would have had to learn Smalltalk and a high-performance Smalltalk VM would have had to be implemented (perhaps — I don’t know — eToys has such a VM). To me it hardly seems like a burden to expect developers to learn and develop in Smalltalk.

I hate to be cynical, but every time I think about the OLPC it’s difficult not to wonder how $200 laptops will possibly stay in the hands of poor children. I definitely “get” that education and technology are crucial to fighting poverty, but I just wonder what percentage of the OLPCs sent to developing nations will end up being stolen and resold. Putting on my cold economic analysis hat, that could be seen as contributing to the society, but benefiting thieves and the able-to-afford-stolen-laptops doesn’t seem efficient.

Well, the thief part is inefficient. Contributing a laptop with the understanding that it is 90% likely to be instantly resold is somewhat efficient (albeit not as efficient as sticking $200 in an envelope and sending it to a random school).

Blech. It’s too early in the week to be cynical. Sign me up.

Universal Geometric Time

Alan Zeichick struck a nerve by calling BS on the premise that Daylight Savings Time saves money. Personally, I dislike DST because during the Summer (or, at this point, non-Winter) Hawai’i is 3 hours behind the West Coast, while during the Winter we’re an hour closer and I can begin work at 7 AM, not 6. (Fun fact: It’s plenty damn dark at 5:30 in the morning in the Winter, even in Hawai’i. Not cold, but dark.)

The solution, it is clear, is Universal Geometric Time: Universal time adjusted for your exact longitude (as measured by GPS). For instance, if I were to follow what The Man says, at 8:00 AM Hawai’ian, it’s 6:00PM Universal. However, I insist that at 6:00PM Universal, it’s 7:37:06 AM (when I’m at home. If I happen to be breakfasting at Java on the Rock, watching the dolphin pod come in from their night’s hunting, that same moment would be 7:37:05).

It’s true that adjusting to Universal Geometric Time requires some getting used to. Anytime I spend more than an hour at a place, I adjust my watch and manually set my cell phone. Plus, of course, I have to be prepared to deal with people using standard time, so I have additional calculators running on my PDA. It’s all quite burdensome, but ultimately it’s a small price to pay for rationality.

There may be trouble, though. While eating in a food court in Waimea  court not long ago, I spread my gear out (since I was sitting down for a meal). I tend to eat alone more since my “friends” have proved resistant to rendezvousing at properly specified times.

I admit that the sight of me blazing the way towards the future makes for a singular sight: GPS, PDA, cell phone, watch, cables, backup batteries, etc. Anyway, I was approached by a woman who turned out to be an astronomer for Keck. Exactly the sort of person who would understand, I thought and gave a quick overview of UGT (I had to break out my laptop, on which I carry an explanatory PowerPoint).

Well.

I don’t know who this “sidereal” fellow is, but I can assure you that he has no place in Universal Geometric Time.

Wrox Blox: $4 Microcontent

Wrox has begun producing low-cost DRM-free electronic content. Their initial product offering is nicely eclectic:

Leverage LINQ in ASP.NET 3.5 Projects
by Roger Jennings
This Wrox Blox introduces you to Language Integrated Query (LINQ), a .NET 3.5 application programming interface (API) and set of extensions to the Visual Basic and C# languages.

Working with Animation in Silverlight 1.0
by Mike Meyers
This Wrox Blox teaches you how to create animations using Microsoft’s new platform for building rich Internet applications – Silverlight. This Wrox Blox introduces animation concepts and answers questions, such as when and why you want to use animation in your Web development.

iPhone and iPod touch Programming: Handling Touch Interactions and Events for Mobile Safari
by Richard Wagner
In this Wrox Blox, Richard Wagner explains touch input events and illustrates how to detect an orientation change, capture two-finger scrolling inputs, and simulate a drag-and-drop action.

Introduction to Google Gears: Creating Off-Line Applications with Pre-built Components.
by Todd Meister
This Wrox Blox provides you with the information you need to use the classes provided from Google. The core classes within Google Gears include Factory, Database, HttpRequest, LocalServer, Timer, and WorkerPool. By reading this Wrox Blox, you’ll be able to determine when to use the different classes provided by Google Gears.

Building a Photo Gallery with Adobe AIR
by Todd A. Anderson
With your favorite text editor and the Flex 3 command-line tools in tow, this Wrox Blox walks you through building a desktop application, leveraging the Flex Framework to browse and manipulate images found on your local system.