Is It “All About The Framework”?

This thought-provoking post on /dev/why!?! argues that backward compatibility is the root cause of the new iPhone restrictions. The argument is summed up in this quote:

I can think of incidents where Apple reverted OS changes, dumped new APIs, or was forced to committing massive engineering resources to something it did not want to do because a Must Not Break™ app vendor told them to.

There’s no doubt that backward compatibility is often a tremendous anchor on technological evolution, especially at the platform level. The more you have “important” apps on your platform, the more pressure there is to consider them Must Not Break™. Let’s say that one of these credit card payment options for the iPhone really takes off and gets adopted by thousands of businesses. And Apple goes on record saying “Gee, we love this payment option, look how easy it is for Joe Mainstreet to use iPad to run his small business out of his car.” Could there be heated discussions when considering whether to release an OS update that breaks the payment-processing software for thousands of businesses or whether they could wait “just one more week until these guys iron out the last of the kinks”? Of course there could be.

Even worse, let’s say that you’ve got a Must Not Break™ vendor whose product was written in, let’s say, Prolog and compiled to the iPhone using PrologTouch. And they’re having trouble and pointing their fingers at PrologTouch and the PrologTouch people say “Yeah, that’s a known bug and we plan on addresssing it in 3 iterations. But they can workaround it by following this simple procedure…” Which is exactly how it goes (except for the part about people writing applications in Prolog). Everyone in the systems-level programming world knows that there’s a lot of finger-pointing about dependencies and drama and sausage (as in, “you don’t want to know what’s inside”) when it comes to synchronizing releases.

Apple certainly knows this as well as anyone and the idea that they want to try to change that dynamic in some way certainly makes sense. Limiting the number of dependencies between platform vendors and developers would probably diminish some of that complexity.

Nonetheless, it’s foolish to do it with contract terms and it’s absurd to do it in a way that outlaws good software practices such as code generation, Domain Specific Languages, and abstracted memory management (or at least type-safe memory management!). Forcing developers to use Objective C, C++, or C to develop applications means that you’ll have more, not fewer, serious defects relating to resource consumption and security. Especially if you’re a victim of your own success as a platform vendor and you have a lot of newcomers to your platform.

The success of the iPhone means that a lot of companies want to develop applications for it, including advertising and marketing applications and inward-facing corporate applications. A lot of those companies do not have experience with explicit memory management. As long as these new restrictive provisions stand, these companies will be forced to either stay off the iPhone platform or to develop software in a way that is unnecessarily error-prone. It’s a rotten choice.

Twitter Weekly Updates for 2010-04-11

  • Remember the Milk is hiring a Scala programmer: http://jobs.stackoverflow.com/default.asp?6700 #
  • Worrisome new rule for iPhone / iPad causing discussion on #monotouch — consensus seems to be that MonoTouch is fine. #
  • More scary stuff from new iPhone SDK license: “”Applications must be originally written in Objective-C, C, C++, or JavaScript…” #monotouch #
  • Restrictive clauses in new iPhone SDK agreement seem to only apply to AppStore Deployment: not enterprise deployment. #
  • Just got my first pair of rigid gas-permeable contacts. Blinking back tears like I’m watching “Brian’s Song” on TV. #
  • RIP Malcolm Mclaren http://www.nytimes.com/2010/04/09/arts/music/09mclaren.html #
  • I’m quoted (more politely) in this SD Times article on the iPhone restrictions: http://www.sdtimes.com/link/34262 #
  • Power just went out in Kalaoa #
  • Power back in Kalaoa #
  • Heading over to Hilo and BI Pizza for tweetup. Is it rainy today? #

Powered by Twitter Tools

“Originally Written in…” vs. Code Generation

My assertion that the iPhone license phrase “Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine” can only mean zero code generation has been questioned. Here’s why I stand by that assessment (and consider the clause beyond the pale).

Let me point out that the common-sense reading of that clause is straightforward. The word “originally” clearly implies that writing this:


UIView.BeginAnimations("");
UIView.SetAnimationTransition(UIViewAnimationTransition.FlipFromLeft, this.Superview, true);
UIView.SetAnimationCurve(UIViewAnimationCurve.EaseInOut);
UIView.SetAnimationDuration(1.0);

and transforming it into:


[UIView beginAnimations: nil context:context];
[UIView setAnimationTransition: UIViewAnimationTransitionFlipFromLeft forView: [self superView] cache: YES];
[UIView setAnimationCurve: UIViewAnimationCurveEaseInOut];
[UIView setAnimationDuration: 1.0]

is forbidden. To take it to the extreme, can you imagine betting your company on convincing a jury that writing the first (which is C# and MonoTouch) and generating the second means it was “originally written in Objective C”? If so, you have some hard lessons about the legal system to learn.

I should point out that even though this code is clearly very similar, there are some slight differences, such as C#’s enumerated values UIViewAnimationTransition.FlipFromLeft as opposed to Objective C’s constant value UIViewAnimationTransitionFlipFromLeft. The . makes a difference in the parse tree and in the programmer’s mind — a small one, but a helpful one.

I should also point out that today MonoTouch does not generate the Objective C 2nd listing. It probably generates something like:

{
.entrypoint
.maxstack 8
L_0000: nop
L_0001: ldstr ""
L_0006: call void Foo.UIView::BeginAnimations(string)
L_000b: nop
L_000c: ldc.i4.0
L_000d: ldsfld class Foo.UIView Foo.UIView::Superview
L_0012: ldc.i4.1
L_0013: call void Foo.UIView::SetAnimationTransition(valuetype Foo.UIViewAnimationTransition, class Foo.UIView, bool)
L_0018: nop
L_0019: ldc.i4.0
L_001a: call void Foo.UIView::SetAnimationCurve(valuetype Foo.UIViewAnimationCurve)
L_001f: nop
L_0020: ldc.r8 1
L_0029: call void Foo.UIView::SetAnimationDuration(float64)
L_002e: nop
L_002f: ret
}

which is “Common Intermediate Language.” The .NET Virtual Machine loads CIL files directly, while MonoTouch generates at compile time native code, which  which is an exercise left for the interested student (and, while hardly trivial, is a mechanical process that can be reviewed for efficiency, improved, etc.). (The “nop”s, by the way, are for breakpoints and wouldn’t be generated in a release build).

Generating two levels of low-level code (C# -> CIL -> Native Code) may seem to be the source of inefficiencies, but generating for an intermediate assembly language is very common with modern compilers. Code generation may be a source of inefficiencies or it may be a place of optimization — that’s dependent on the skill of the compiler writers.

The reason why it’s absurd to forbid native code compilers but allow Objective C code generation is that there is no difference between generating assembly language and generating C. If you can write an IL->Native Code compiler, of course you can write a IL->C compiler. So were code generation legal, a potential strategy for other alternative language vendors would be to do FooLanguage -> Foo IL -> C (“OK, lawyers, as you can see, we are simply generating C code”) -> Apple Approved C Compiler -> Native. Kafka-esque absurdity, but perfectly cromulent. Indeed, I strongly suspect that all of the alternative language vendors are preparing just such a tactic as “Plan B” or “Plan C” in case Apple relents on the word “originally.”

The Double-Edged Sword

Why should Apple allow alternative code generation? They fear inefficiencies, they cannot control the efficiency of code generation, therefore it is logical to forbid code generation. Why is that unreasonable?

The reason is that if you forbid code generation, you forbid huge swaths of good programming techniques. Rather than go on about lex and yacc and report generators and screen managers, though, we have the delightful “The Elements” iPad app to make the point. Don’t take it from me that code generation is good, read the article:

Even the amazing animation synchronized to Tom Lehrer’s iconic song “The Elements”, which plays the first time you open the e-book, was generated entirely by a Mathematica program.

QED.

Automatic Parallelization of Memory Management

MMT: Exploiting Fine-Grained Parallelism in Dynamic Memory Management, Tiware et al. is worth reading if you are interested in systems-level performance programming. If you work in a managed environment, the briefly described lockless protocol for request-reply queues is worth noting.

This paper describes the use of a “Memory Management Thread” (MMT) to automatically parallelize memory management in C/C++ programs. In allocation-intensive domains (often domains that involve traversing or transforming big graphs — constraint programming, parsing and translation, optimization problems, etc.), memory management overhead can be around 30% of execution time.

The fundamental challenge is that memory management is very fine-grained and that communication and coordination between cores and processors can easily introduce more overhead than is gained by “easy wins” like asynchronous deallocation.

The improvement appears to come from speculative bulk allocation and bulk de-allocation. What makes the “speculation” hard is synchronization and making sure that you avoid too much contention. Their win in this area appears to come from an elegant approach: dual request and reply queues access to which is ping-ponged between the client and server.

screen-shot-2010-04-07-at-112233-am

Their benchmark numbers are good.

The Absurdity of Apple’s New iPhone Restrictions

These are the new clauses:

  • Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine

I’m tempted to say that the clause is absurd and to talk about metaprogramming with C++ templates or JavaScript prototypes, but the fact is that it’s perfectly cromulent. The common-sense reading of this is straightforward: no code generation.

The constraint, however, is utterly illogical. Good programmers don’t write all their code: they write code that generates the boring stuff. This is probably the type of thing that was used in “The Elements” — automatic generation of the .xib files, asset directories and resources, and generation of  Objective C animation code (the video explicitly says that animations were generated). Forbidding Domain-Specific Languages (and the parser generators that help build them) flies in the face of the very people you want to attract to your platform.

C, C++, and Objective-C were all developed more than 20 years ago and all, on the iPhone, require explicit memory management — the most error-prone detail in software development. You know how you tell when an app for the iPhone was written in MonoTouch? It doesn’t leak memory.

You can play the semantic game of “Well, since a C program is not ‘originally written’ until it’s been compiled, if we use a higher-level language to generate C and compile that then we are within the letter of the clause.” But try getting that past your company’s lawyer when you’re trying to get the go-ahead for a business-critical application.

Apple may feel that they are in such a strong position in the market that they can dictate that outsiders use the same techniques to write software that are used internally at Apple. But that’s not how programming works: it is a continually evolving melange of abstractions, techniques, and tools. It has to be allowed to vary at the individual, team, and industry level if it is to progress. I’ve worked on C and C++ at the level of standards compliance suites, I’ve done my fair share of JavaScript, and I can stumble along in Objective C. But personally, when it comes to the iPhone, my language choice is C#: it’s significantly higher-level than C or C++, it’s a little more terse and cleaner than Objective C, it has certain language features that are undoubtedly ahead of Objective C (LINQ), and it provides a better type system. That is just one person’s opinion and Apple can certainly afford to ignore me, but if they think can ignore the last two decades of advancement in the programming marketplace, they’re over-confident.

Those with a sense of historic irony might observe that Objective C for many years existed only as a code-generating preprocessor and that a similarly restrictive clause could easily have killed its development and prevented NeXT from creating the software development toolchain that Apple is using today.

  • Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited

This clause makes more logical sense. Mobile development is still a resource-constrained environment. While memory, storage, and processing speed are beyond the dreams of yesterday’s hobbyists, processor use drains the battery and that is a very big deal indeed. Desktop developers today are generally free to live in whatever abstract model they choose: you want to live in a world where everything is an object, a mathematical function, or an S-Expression? Go for it! There’s enough power under the hood to make your program “fast enough.”

Not in mobile development. For instance, reading the color of a single pixel on the iPhone is kind of a big deal: you have to allocate a new data buffer, draw into it,and then read the raw bytes at the calculated offset. In Erica Sadun’s iPhone Developers Cookbook, she dedicates about 50 lines of Objective C code to demonstrate the technique. In MonoTouch, I did it in 45 lines of C# code. Of course, once you’ve written the function you can call it forever-more with a single line, but you probably will remember that your function has implications in terms of memory and computation resources. Unless you’re a very bad programmer, you probably would ensure that you don’t create and destroy that buffer every time you want to read or set a pixel.

On the other hand, a Flash developer has the functions pre-written for them: BitmapData.getPixel() and BitmapData.setPixel(). And they probably don’t think about those functions having significant overhead — part of the reason people use Flash is because it’s abstracted away just such things. So how can a cross-compiler for Flash deal with the line myBitmapData.getPixel()? There are three choices:

  1. Treat it as a single programmatic intention — allocate the buffer, read the data, and deallocate the buffer.
  2. Treat it as two intentions — (a) “allocate a buffer for further use” and (b) “read the pixel.” Keep the buffer around for further use.
  3. Divine the programmer’s intent — if they really only intend to do this once then treat it atomically, if they are going to be doing a lot of pixel manipulation, keep the buffer around

All three choices have problems:  if you choose 1, but the program does do lots of manipulation, you destroy performance. If you choose 2, you consume memory at a potentially prodigious rate (and even so, you have to write a garbage collector). If you choose 3, you’re going to have to wait until researchers develop the “do what I mean” compiler switch, which is still a research project.

That’s Apple’s legitimate concern with “intermediary translation or compatibility layers”: the efficiency and restrictiveness of such layers. MonoTouch is quite efficient, maybe some others won’t be. One thing that MonoTouch does (garbage collection), it does well — perhaps some other implementations won’t. MonoTouch makes it very easy to “drop down” to unsafe code or to link in new CocoaTouch APIs, that clearly won’t be the case with some others.

So what if Apple kept the clause forbidding translation and compatibility but dropped the first one? So that I  write my flow-of-control using whatever language I choose, but I don’t write my own “translation or compatibility layer”? That seems reasonable until you realize that there’s no difference between a translation layer and a 3rd-party library call. Functions are translation layers: the difference is solely one of intent and extent. And even as far as “extent” goes,  C++ template metaprogramming, Boost, and smart pointers have lots of consequences that absolutely reach throughout the application. What if I wanted to use a port of libgc? Legal or not?

In one day, Apple has damaged projects as diverse and good for the iPhone as MonoTouch, Unity3D, and Appcelerator. They’ve sowed Fear, Uncertainty, and Doubt about projects like RunRev and Flash Packager for iPhone. And they’ve done it with 2 lines of technical absurdity. One hopes that they can come to an understanding with 3rd parties that helps ensure the performance and consistency of applications written for CocoaTouch without draconian restrictions against fundamental programming techniques.

If not, one hopes they go fuck themselves.

Using Mathematica to Generate “The Elements” iPad App

The most intriguing part of this Popular Science article on the creation of the iPad app “The Elements” is the all-too-brief mention of the creation of a higher-level Mathematica program to generate the pages:

In about 7 of the precious 60 available days, I was able to create from scratch, using Mathematica, an interactive magic object page layout tool that designer Ann Grafelman was able to use to design all 118 individual element pages in record time.

….

A complete rendering pass for the e-book requires running eight parallel Mathematica processes for a couple of days on the fastest available 8-core Macintosh. But it is a completely automated process, turning a terabyte of image archives into a finished, fully operational 1.9 gigabyte iPad app. This complete automation meant that we were able to experiment with dozens of different layouts and styles, concentrating on creativity, not the grunt work of manual file processing, yet still be able to see the finished book in action after each tweak.

Behold the double-edged sword of iPhone’s new restrictions on code generation and translation layers! While the intention is to avoid a flood of banal and bloated marketing games, your effect is to outlaw the creation of a “book that Harry Potter would check out of Hogwarts.”

New iPhone OS License Threatens 3rd Party Languages

In certain circles, whatever interesting things there were in Apple’s iPhone OS 4.0 unveiling today were overshadowed by some new language in the iPhone Developer Program License Agreement:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited). [Emphasis added–LOB]

This is almost certainly a reaction to Adobe’s Flash-to-iPhone cross-compiler, but on the face of it, it catches a whole range of 3rd parties in the blast radius. In particular, I’ve been recommending MonoTouch as a great solution for C# developers and teams. Although Mono compiles C# code to native code, Mono applications are not “originally written” in one of Apple’s preferred languages, so would clearly seem to be prohibited under the new agreement.

The buzz on the monotouch IRC channel is that this restriction only applies to applications that are deployed the AppStore and not to enterprise deployment, which means that even if the restrictive clauses stay in place, MonoTouch might remain a good option for some.

This is a developing story and it doesn’t seem to make sense that Apple gains by limiting the universe of programmers for the iPhone. Prohibiting inefficient translation layers and libraries is one thing, prohibiting code generation and higher-level languages is another. One of the showcase apps for the iPad is “The Elements” whose media, page layout, and transitions were largely generated in Mathematica. Many games use higher-level languages such as Lua to script AI.

Logically, one would hope that the legal agreement would be clarified with some form of “…except for approved toolchains,” and figure out how to work with  Novell and Appcelerator and, yes, even Adobe.