Knowing .NET

At the recent DSL DevCon, Martin Fowler gave an excellent keynote speech discussing domain-specific languages, an important subject that seems certain to be the buzzword of the next year or two.

However, one slide of his got my goat, leading me to foam at the mouth, beat the table with my shoe, wag my finger angrily, and otherwise mix my metaphors. In that slide, he said something along the lines of “a DSL probably ought not to be Turing complete.”

I think this is terrible advice. First, I think it’s a pedagogical mistake. Second, I think it’s incorrect.

Proof

Q.E.D.

Discussion

Is SQL a less powerful programming language than the game of minesweeper?

To the extent that we want to broaden the number of people writing domain-specific languages, we can’t put that question on the entrance test.

Even if the percentage of people who know the answer can be made large, the percentage who comprehend it is quite a bit smaller, and the number who can apply it is a very small number. In a DSL, it’s smaller still, because you have to analyze your language in the context you’ve embedded it. You’ve climbed right up Bloom’s taxonomy! That’s why I think it’s a mistake pedagogically.

Why I think the advice is flat-out wrong is that DSLs are necessarily embedded in the context of a general-purpose language. Say I want a DSL that converted the numbers 1 through 256 into a set of CA rules following the obvious structure implied in:

A DSL with exactly one production ({byte} -> {CA rules}) is not Turing complete (duh). But the context in which this DSL is expressed is one that leads to a (possibly profound) discovery of Turing completeness. (In case it’s not obvious from the context — the illustrated ‘Rule 110′ CA is Turing complete.)

Fowler referred to this when he advised DSL writers to avoid “accidental” Turing completeness. That really set the hook deep in my mouth, because it has a whiff of this stuff. To raise the issue and then to hand-wave it away is leaving yourself one heck of a big escape hatch. The mention of ‘Turing completeness’ or ‘Turing equivalence’ begs attention; they’re either forbidden holy-words or they’re used precisely and must be discussed with precision. (Thus, O’Brien’s Corollary (1991) to Godwin’s Law: The first person to mention ‘Turing equivalence’ in a debate of programming languages loses.)

Solution

It’s only because Fowler generally gives such excellent advice that the “Turing complete” stuff provoked me so much. Sure enough, a moment after the slide I didn’t like, he came up with what seems like the answer. I’m not sure of his exact words, but it was along the lines of:

Start with a general language that you think is readable and take stuff out. If you can’t take out quite a bit, don’t write an external DSL.
 

Great advice.

ResolverOne is one of my favorite applications in the past few years. It’s a spreadsheet powered by IronPython. Spreadsheets are among the most powerful intellectual tools ever developed: if you can solve your problem with a spreadsheet, a spreadsheet is probably the fastest way to solve it. Yet there are certain things that spreadsheets don’t do well: recursion, branching, etc.

Python is a clean, modern programming language with a large and still-growing community. It’s a language which works well for writing 10 lines of code or 1,000 lines of code. (ResolverOne itself is more than 100K of Python, so I guess it works at that level, too!)

From now (Dec 2008) to May 2009, Resolver Systems is giving away $2K per month to the best spreadsheet built in ResolverOne. The best spreadsheet received during the competition gets the grand prize of an additional $15K.

Personally, it seems to me that the great advantage of the spreadsheet paradigm is a very screen-dense way of visualizing a large amount of data and very easy access to input parameters. Meanwhile, Python can be used to create arbitrarily-complex core algorithms. The combination seems ideal for tinkering in areas such as machine learning and simulation.

I try to do some recreational programming every year between Christmas and New Year. I’m not sure I’ll have the time this year, but if I do, I may well use ResolverOne and Python to do something.

 import sys import clr sys.path.append("c:\\program files\\Microsoft Research\\Infer.NET 2.2\\bin\\debug") clr.AddReferenceToFile("Infer.Compiler.dll") clr.AddReferenceToFile("Infer.Runtime.dll") from MicrosoftResearch.Infer import * from MicrosoftResearch.Infer.Models import * from MicrosoftResearch.Infer.Distributions import *  firstCoin = Variable[bool].Bernoulli(0.5) secondCoin = Variable[bool].Bernoulli(0.5) bothHeads = firstCoin & secondCoin ie = InferenceEngine() print ie.Infer(bothHeads) --> c:\Users\Larry O'Brien\Documents\Infer.NET 2.2>ipy InferNetTest1.py Compiling model...done. Initialising...done. Iterating: .........|.........|.........|.........|.........| 50 Bernoulli(0.25) 

Sweet

Some d00d named Leppie has written a Lisp for the DLR [via Jason Bock]

Silverlight steadfastly refuses to install on my development Tablet PC or in my “Orcas Beta” Virtual PC. Those are the only two machines on which I’ll put an under-development CLR, since experience has shown that it can be very difficult to cleanly uninstall anything in Microsoft’s browser-CLR-plugin axis. This is a shame, because I have some ambitious Silverlight projects.

Via  Rob Connery Interviewed by Miguel de Icaza, I took a quick look at SubSonic, which appears to be a good solution in the ASP.NET world for very rapidly generating Create-Retrieve-Update-Delete functionality pages that honor database foreign keys.

One of the dazzlers in the Ruby world is a library called ActiveRecord, which powers similar functionality within Ruby on Rails. If you’re lucky enough to be starting with a new database, you can generate “scaffolding” to edit your tables in a matter of quarter-hours. (As Steve Jones says in CRUD is Crap, that such functionality is considered dazzling in the year 2007 is an indictment of our toolsets, but nonetheless.)

Any client-facing application almost instantly moves beyond scaffolding, but I’ve been reminded recently of the need for rapid data-editing / cleaning in any large application but especially in SOAs, with data continuously flowing into the system boundaries. Garbage-in, garbage-out is a big problem in SOAs.

Two downsides of ActiveRecord are that:

• Foreign key relationships must be manually inserted, and
• It’s naming conventions can be tough to overcome when using a legacy DB

These seemingly minor issues can be significant when you’re dealing with an enterprise-sized database with several hundred tables. Whether SubSonice addresses these issues, I don’t know, but I look forward to adding it to my arsenal.

The source code to the Microsoft.Scripting namespace is available at http://www.codeplex.com/IronPython/Release/ProjectReleases.aspx?ReleaseId=438

A review of the IronPython codebase in the release makes the early-stage processing look pretty close to the (well-designed) IronPython 1.0 release: there’s a language-specific tokenizer, parser, AST generator and walker. Once you’ve transformed your AST into whatever-the-appropriate-form is, code generation looks insanely easy.

So the bad news is that there’s no compiler = new Language(BNF myGrammar); statement. The good news is that we already have such things: they’re called Lex and Yacc — or better yet, ANTLR. Although the sheer line-count of a language-specific compiler using Microsoft.Scripting seems large, such tools should be able to generate maybe 80-90% of it (I’m talking of the compiler, not the shell or runtime support, but that, too, looks generically handled by Microsoft.Scripting).

But this is one of those recursive things — you can’t generate a DLR-based compiler until someone writes a DLR target for the compiler-compiler. Interestingly, the definitive ANTLR Book will hit the shelves any day now. Hmmm … if I was a kid interested in getting a job at Microsoft, I know what I’d be doing this Summer …

That Microsoft was going to increase support for dynamic languages is no surprise: they’ve been talking about that since (at least) PDC ‘03 and various hires and initiatives have clearly been in the works. I haven’t seen the DLR yet, but my big question is: what version / runtime / patch level of the CLR  and libraries becomes the lowest-common denominator for Silverlight (i.e., cross-platform, in the browser)? Because for better or worse, that becomes the platform for dynamic languages in the .NET world.

I am surprised by the IronRuby announcement (and officially bestow the He-Man Programming Award to John Lam). I really thought we were going to see some form of Ruby#:Ruby::C#:Java. Although I’m happy (Ruby is now my #1 administrative programming language), I was actually hoping to see a new language. Ruby’s a fine language, but it doesn’t have a good story for concurrence, it has a boring model of XML (unlike VB), it has some unattractive Perl-isms. Most importantly, I think MS does a good job when they have the flexibility to evolve the language and, simultaneously, can devote the resources to developing the compilers, libraries, and toolsets.

Scott Guthrie just announced IronRuby! Yeehaw!!!!!!

—

Silverlight binding; demoing on Mac

—

Dynamic Language Runtime announced

—

“Shipping later this week” (I’m sure he means shipping a beta / CTP)

—

Uses Ruby naming scheme when programming .NET libraries (weird. Not sure I like that.)

—

“Switch the console into Python mode, Ruby mode, VB mode” (he means something called “Dynamic VB”)

—

www.silverlight.net is the commmunity site

(No sign of DLR / IronRuby / IronPython / Dynamic VB on that site as far as I can see)

—

“Componentize dispatching code, code generation, that sort of thing that compiler vendors do” … namechecks to Hugunin and Lam … we shipped this morning the DLR source code” on Codeplex …

(http://www.codeplex.com/Project/ProjectDirectory.aspx?ProjectSearchText=dynamic%20language%20runtime ? )

Naysawn has announced that the Visual Studio unit testing framework (MSTest) is moving into the Pro SKU of Visual Studio. That’s a good start, but I want to lobby for going the whole way:

Please move the unit testing framework into the .NET framework.

Source: MSTest Moving to VS Pro
Originally published on Wed, 28 Mar 2007 08:35:00 GMT by Brad Wilson

And there was much rejoicing…

Of course, Brad’s right that unit-testing libraries ought to move into the base library, but the VS tools are sophisticated enough to be implemented solely as VS add-ins.