The source code to the Microsoft.Scripting namespace is available at http://www.codeplex.com/IronPython/Release/ProjectReleases.aspx?ReleaseId=438

A review of the IronPython codebase in the release makes the early-stage processing look pretty close to the (well-designed) IronPython 1.0 release: there's a language-specific tokenizer, parser, AST generator and walker. Once you've transformed your AST into whatever-the-appropriate-form is, code generation looks insanely easy.

So the bad news is that there's no compiler = new Language(BNF myGrammar); statement. The good news is that we already have such things: they're called Lex and Yacc -- or better yet, ANTLR. Although the sheer line-count of a language-specific compiler using Microsoft.Scripting seems large, such tools should be able to generate maybe 80-90% of it (I'm talking of the compiler, not the shell or runtime support, but that, too, looks generically handled by Microsoft.Scripting).

But this is one of those recursive things -- you can't generate a DLR-based compiler until someone writes a DLR target for the compiler-compiler. Interestingly, the definitive ANTLR Book will hit the shelves any day now. Hmmm ... if I was a kid interested in getting a job at Microsoft, I know what I'd be doing this Summer ...