Archive for the ‘SD Tools’ Category.

Mars Climate Orbiter, Python, and Type Systems…

Faithful readers know that I’m learning Scala, somewhat reluctantly. A few weeks ago, I was reading New Scientist magazine and saw this writeup of Peter Norvig, Director of Research at Google, which mentioned that he was on the review board that studied the crash of the Mars Climate Orbiter. That sparked a question in me, because I’d recently watched Norvig’s talk “What To Demand From A Scientific Computing Language” which says “don’t spend too much time fretting about language choice,” but further goes on to argue that Python is a heckuva good scientific computing language. So I asked him:

The Mars Climate Orbiter disaster would seem to be a case study for a type system with more compile-time strictness. Obviously, an SI unit system is not part of Scala’s standard library, but I find it interesting that you have not cast all aside and picked up the banner of Haskell or what-have-you.

And he kindly responded with a typically pithy insight:

I don’t think the MCO incident has anything to say about language choice.  The problem involved reading data from a file and a miscommunication about what the numbers in the file were.  I don’t know of any language, no matter how type-strict, that forces you to tag the string “123.45″ in a file with the units of force (newtons vs foot-pounds), nor do I know of any language, no matter how type-loose, in which you could not impose such a convention if you wanted to.

This, for me, is the last word in the argument for and against manifest static typing: you can always get around it on the one hand, and you can always enforce preconditions on the other.

Although I have a marginal preference for explicit/manifest typing in function signatures because I think that it helps readability, it’s horses for courses — if you say you think explict typing hinders readability I can’t say you’re wrong (much less get worked up about it).

Regretfully, this is not to say that I don’t get worked up in type-system discussions: I do. But I get worked up by the over-simplifications that say ‘your way is fundamentally wrong.’ We’re so beyond the days when you could make that argument: there are large systems written in dynamic languages and type-inferred languages that minimize finger-typing in the mainstream (I’m looking at you, C#. No one’s calling you a Java clone now, are they?).

So, if the type system wasn’t at fault, what were the causes? Norvig explains:

Beyond the initial error, the reasons why the error proved fatal were more around organizational structure than around language choice:
(1) An anomaly was detected early on, but was not entered into an official issue-tracking database. Better practices would force all such things to be tracked.
(2) The team was separated between JPL in California and Lockheed-Martin in Colorado, so there were no lunvh-time discussions about “hey, did you get that anomaly straighten out?  No?  Well, let’s look into it more carefully…”
(3) The faulty code was not carefully code-reviewed, because of improper code re-use.  On the previous mission, this file was just a log file, not used during flight operations, and so was not subject to careful scrutiny.  In MCO, the file and surrounding code was re-used, but then at some point they promoted it to be a part of actual navigation, and unfortunately nobody went back and subjected the relevant code to careful review.
(4) Bad onboarding process of new engineers: The faulty code was written by a new engineer — first week (or maybe first month or so — on the job. This was deemed ok because originally it was “just a log file”, not mission-critical.

There you go: It’s not type-systems. It’s not where you put the brackets (Allman-style, all hail!). Defect management, team dynamics, code review, and onboarding. Now those things are worth getting worked up about.

Microsoft’s Intriguing Code Bubbles Projects Becomes Debugging Tool

I haven’t used this, but I talked briefly with the Code Bubbles developers about 18 months ago and thought that debugging would be the perfect fit. Seems like someone at MSR agreed:

Debugger Canvas – Microsoft Research.

Knowing Scala: Exercise 1

Taking my own 15 Exercises to Know A Programming Language as a starting point for my exploration of Scala…

The first set of exercises are right up the alley of a language with pattern-matching and list-processing:

Write a program that takes as its first argument one of the words ‘sum,’ ‘product,’ ‘mean,’ or ‘sqrt’ and for further arguments a series of numbers. The program applies the appropriate function to the series.

The first thing I did was write the functions:

val args = List(1, 2, 3, 4, 5)
args.foldLeft(0)(_+_) //Sum
args.foldLeft(1)(_*_) //Product
args.sort(_ < _)(if(args.length % 2 == 0) args.length / 2 - 1 else args.length / 2) //Median
args.map(Math.sqrt(_)) //Sqrt

Here the list-processing functions (highlighted in red) shine. The foldLeft() family of functions, also known as reduce, inject, or aggregate, applies a predicate to the elements of a data structure, accumulating a value. It’s hard to imagine cleaner code than what we have for the ‘sum’ and ‘product’ challenges. Similarly, the map() function is similarly clean-as-a-whistle for the ‘sqrt’ challenge. The ‘median’ challenge I’m less thrilled about: Passing in a custom comparator predicate to sort() is lovely, but my fingers typed in the C-language ternary operator :? and I was sorry to find that Scala doesn’t have it.

A big win for Scala is that I typed these in the Read-Evaluate-Print-Loop aka interactive console rather than having to deal with the overhead of a ‘real’ program and a edit-compile-test loop.

Moving on, the complete program follows:

object Main
{
	def main(args : Array[String])
	{
		val argList = List.fromArray(args)
		val op = opMatch(argList.head)
		val numList = argList.tail.map(java.lang.Double.parseDouble(_))
		val result = op(numList)
		print(result)
	}

	def sum(args : List[Double]) : Double = { args.foldLeft(0.0)(_+_) }

	def product(args : List[Double]) : Double = { args.foldLeft(1.0)(_*_) }

	def median(args : List[Double]) : Double =
	{
			args.sort(_ < _)(if(args.length % 2 == 0) args.length / 2 - 1 else args.length / 2)
	}

	def sqrt(args : List[Double]) : List[Double] = { args.map(Math.sqrt(_)) }

	def opMatch(arg : String) : (List[Double]) => Any =
	{
		arg match
		{
			case "sum" => return sum
			case "product" => return product
			case "median" => return median
			case "sqrt" => return sqrt
			_ => throw new java.lang.IllegalArgumentException("Unrecognized operator " + arg)
		}
	}
}

Let’s see… The translation of the REPL code into functions was very straightforward, although I have to admit to being somewhat disappointed that I could not use a type more abstract than Double. I don’t know if I could define a trait and then apply it retroactively to types in the java.lang package.

opMatch() shows some functional goodness — it’s declared as a function that takes a String and return a function that takes a List of Doubles and returns Any kind of object.

I’m not sure if I could do something more precise for my needs, like “returns a Double or List[Double]“. The pattern-matching in opMatch() is nice but not advanced.

If you have any observations to share, please comment below…

Cyclomatic Complexity Analyzer for Ruby

Saikuro : A Cyclomatic Complexity Analyzer.

Function complexity does not seem to be a great problem in the Ruby world, but tracking cyclomatic complexity is one of those things that can help pinpoint troubled modules.

ResolverOne: Best Spreadsheet Wins $17K

ResolverOne is one of my favorite applications in the past few years. It’s a spreadsheet powered by IronPython. Spreadsheets are among the most powerful intellectual tools ever developed: if you can solve your problem with a spreadsheet, a spreadsheet is probably the fastest way to solve it. Yet there are certain things that spreadsheets don’t do well: recursion, branching, etc.

Python is a clean, modern programming language with a large and still-growing community. It’s a language which works well for writing 10 lines of code or 1,000 lines of code. (ResolverOne itself is more than 100K of Python, so I guess it works at that level, too!)

From now (Dec 2008) to May 2009, Resolver Systems is giving away $2K per month to the best spreadsheet built in ResolverOne. The best spreadsheet received during the competition gets the grand prize of an additional $15K.

Personally, it seems to me that the great advantage of the spreadsheet paradigm is a very screen-dense way of visualizing a large amount of data and very easy access to input parameters. Meanwhile, Python can be used to create arbitrarily-complex core algorithms. The combination seems ideal for tinkering in areas such as machine learning and simulation.

I try to do some recreational programming every year between Christmas and New Year. I’m not sure I’ll have the time this year, but if I do, I may well use ResolverOne and Python to do something.

IronPython 2.0 & Microsoft Research Infer.NET 2.2

 import sys import clr sys.path.append("c:\\program files\\Microsoft Research\\Infer.NET 2.2\\bin\\debug") clr.AddReferenceToFile("Infer.Compiler.dll") clr.AddReferenceToFile("Infer.Runtime.dll") from MicrosoftResearch.Infer import * from MicrosoftResearch.Infer.Models import * from MicrosoftResearch.Infer.Distributions import *  firstCoin = Variable[bool].Bernoulli(0.5) secondCoin = Variable[bool].Bernoulli(0.5) bothHeads = firstCoin & secondCoin ie = InferenceEngine() print ie.Infer(bothHeads) --> c:\Users\Larry O'Brien\Documents\Infer.NET 2.2>ipy InferNetTest1.py Compiling model...done. Initialising...done. Iterating: .........|.........|.........|.........|.........| 50 Bernoulli(0.25) 

Sweet

IronRuby Previews Form Designer

From SapphireSteel, the makers of IronRuby, comes the first peek at an IronRuby visual design surface.

Open Komodo Now Available

Those without an investment in other editors would do well to investigate Open Komodo, a free as in speech editor based on the good as in good Komodo editor.

Bash Ups

With the release to public beta of Popfly, Microsoft’s mashup editor, I’ll reiterate my theory that mashups are the UNIX shell of the Internet. The corollary is that we need a suite of command equivalents:

Command Mashup Alternative
cd, mkdir, rmdir facilities for manipulating “current URI”; REST principles, etc.
mailx messaging transformations and transports: mail, IM, SMS, twitter, etc.
man  ?
jobs, ps, kill, sleep, etc. facilities for multiple mashup control
ls spidering facilities / robust HTML parsing, etc. “Get-ChildItem” in all its polymorphic complexity.
who FOAF
finger, chfn blogging
cat, sed, sort, grep, wc, tail, etc. All sorts of facilities for transformation of source to sink

Right now, everyone’s concentrating on what output the mashup editors can produce or what the component manipulation looks like. I think the winner of the mashup evolution will be the one that provides the most flexible suite of components.

F# To Become Product: Very Surprising

I’m tempted to label as “shocking” the announcement that F# will become a product fully integrated into Visual Studio, but I suppose it would be hard for anyone to ignore stuff as compelling as this.

F# is a derivative of OCaml and is a functional programming language. Those who delve into my language-related or concurrency-related posts will be familiar with the concept that one of the great advantages of functional languages are characteristics that lend themselves to automatic parallelization. Microsoft is making more and more noise about functional approaches (Eric Lippert’s “aside” that “Immutable data structures are the way of the future in C#.” is telling.) and this endorsement of F# is another sign that Redmond is throwing its weight pretty heavily behind this approach.

F# has been well-received among the hardcore language nerds but you have to give Microsoft credit for getting out ahead of the market on this one. F# is a very different beast than the Iron* languages (Python and Ruby). This isn’t Microsoft reacting to market demands, it’s Microsoft putting a not-at-all-well-known language into the spotlight.