Wes Moise's musings on Supercompilation led me to this discussion of the the myth of the sufficiently smart compiler.
The "sufficiently smart compiler" is still trotted out regularly, even though the market has moved away from demanding even moderate attention to performance at the compiler level. Have you timed your rectangular arrays in C# lately? Or, to be inclusive, have you looked at what's (not) hoisted out of loops in Java?
The existence of the Iron* languages from Microsoft stems from Jim Hugunin's discovery that adding moderate smarts allowed dynamic languages to run fast on the CLR:
- Use native CLR constructs whenever possible. These constructs are all heavily optimized by the underlying runtime engine. Sometimes these constructs will have to be used creatively to properly match Python's dynamic semantics.
- Use code generation to produce fast-paths for common cases. This can either be development-time code generation with Python scripts creating C# code or run-time code generation using System.Reflection.Emit to produce IL on the fly.
- Include fall-backs to dynamic implementations for less common cases or for cases where Python's semantics requires a fully dynamic implementation.
- Finally, test the performance of every CLR construct used and find alternatives when performance is unacceptable. Type.InvokeMember is a great example a a function that must never be used in code that cares about performance.
That's hardly the stuff of PhD theses (don't misunderstand me: Hugunin's paper, which actually said something important, is more valuable than 99% of CS theses).
The point, though, is that we are in a time of high tension between what is possible and what is practiced. This gives me hope that we might see true breakthroughs in programming languages. Fred Brooks spoke of a silver bullet defined as a "single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity." [my emphasis]). I don't believe in silver bullets, but I think there's a possibility of shiny buckshot.
On the discouraging side, I think there are great difficulties to building such a system: the development of a shiny shotgun is, I think, the work of double-digit person-years. It's work that's too far over the horizon for VC funding, too pragmatic for grants, and too dependent on brilliant execution by a small, high-performance team for Open Source.