Archive for November 2004

100GB HD for Your Tablet: $194

Possibilities for Intellisense(tm) in Pynk…

Julie points out that if it ain’t got Intellisense, it ain’t a productive programming environment. Now,

  • Pynk doesn’t even have cut-and-paste yet, so that comes first; and
  • It should be possible to use reflection to get, not quite Intellisense, but something like a list of properties and methods that might be applicable to the current object; but
  • We’re back to the PIP dilemma: I would need events firing as the user enters individual characters (at least, when they enter a ‘.’). Which you don’t get with the PIP, but which I might be able to get off the RecognizerContext object’s Recognize event. But, I don’t think I can actually get a reference to the PIP’s RecognizerContext, so I can’t use the PIP for input in this scenario. But I want to move towards free-form ink and/or syntax-directed UIs anyway…

P.S. I can’t do something Intellisense-like within Shark itself, since it runs from static configuration files and Intellisense is inherently dynamic

Syntax- directed shark

James Kendrick nailed it. You can edit the Shark keyboard to use full words:

Making a syntax-directed programming editor that much easier to contemplate.

Longhorn to be 64-bits only?!?!?!

Tech Blender points to this rumor that Longhorn will require 64-bit hardware. You know, one of the fairly easy lessons to draw from the 16- to 32-bit change is that it happened too late, that there was a great deal of pain because everyone tried to live with side-by-side APIs and “sort of” 32-bit versions of Windows etc. So, while I’d not heard this rumor before, it’s not inconceivable that it’s true.

TIP hooks would hook developers (and users)

What would you do if you had more programmatic access to the Tablet PC TIP? Via [Incremental Blogger]

I think the general answer to this is that I want to be able to control the TIP via something awfully close to Backus-Naur Form. Here’s a classic example of how math equation input might be defined:

input ::= ws expr ws eoi; 
expr ::= ws powterm [{ws '^' ws powterm}]; 
powterm ::= ws factor [{ws ('*'|'/') ws factor}]; 
factor ::= ws term [{ws ('+'|'-') ws term}]; 
term ::= '(' ws expr ws ')' | '-' ws expr | number; 
number ::= {dgt} ['.' {dgt}] [('e'|'E') ['-'] {dgt}]; 
dgt ::= '0'|'1'|'2'|'3'|'4'|'5'|'6'|'7'|'8'|'9'; 
ws ::= [{' '|'\t'|'\n'|'\r'}]; 

That’s really pretty darn comprehensible if you ask me.

Shark Programming?

Loren noted that what my video of Pynk programming really demonstrates is that it takes me 1:23 to handwrite “eval(io.Ink.Strokes.ToString())”. Too true. I’ve been thinking a lot about how to evolve Pynk and, although my first priority is making the “just plain text” interactive console fully functional, the ultimate goal is to make a viable ink-based programming environment, and that’s going to take some “outside the box” thinking.

This is what occurred to me on my walk this morning. Shark is an impressive new input technique from IBM:

This image shows some of the important features (short travel between related keys, button density) but not the most striking when it’s used: gesture-based recognition (:50 screencast).

I plan on training myself with Shark for several hours and then timing my input against the TabletPC TIP. We’ll see.

But it got me thinking of a Shark-like syntax-directed input pad for Pynk, combining keywords, in-scope variables, and perhaps even namespace-available classnames:

What do you think?

P.S. Iggy’s recent post that “It’s not about text input” is also apropos

Oh, _that’s_ why he doesn’t return my emails…

According to Steve Ballmer, Bill Gates receives 4,000,000 emails per day. (via Alice and Bill)

You know, somehow I think that after the fabulous spam-filtering technology of Microsoft Exchange and Outlook, there might, just might, be human eyes between billg@microsoft.com and the man himself.

Speech recognition circa 2004

If you’ve never tried dictation, you can get a sense of how it works by watching a video screencast I made shortly after I installed Version 8 of NaturallySpeaking. The out-of-the-box experience was dramatically better than before via [Jon's Radio]

Not long ago I had a chat with one of Microsoft’s recognition guys (J. Pittmann) and learned some interesting factoids about how voice recognition works. The most interesting is that they throw out tons of data and can’t take advantage of the added horsepower available in offline mode. You see, for me voice dictation would be much, much more useful if I could transcribe notes taken while driving or walking – when I’m at my computer in a quiet environment, I’d just as well use the keyboard or pen. The realtime aspect of recognition is not of much interest to me, but I would be thrilled if I could say “Take the next 18 hours to recognize what I babbled on my two-hour drive.”

Not with today’s algorithms. All speech recognizers began their lives several Moore’s generations ago and all use algorithms that began life in the pre-Pentium era, and were designed for very low sampling rates and bit size. (Today’s recognizers probably use higher-quality signals, but the point is that the algorithmic assumptions are what we’d today consider low-fidelity, low-memory, low-CPU.)

Basically, they quickly throw out everything to try to get to vectors representing sub-phonemes (“codes”) that they template-match to produce phonemes (). They try to match the stream of these to complete words and word-sequences, using probabilistic pattern matching and language models.

This bottom-up approach has gotten to the point where it works pretty well. But at every step up the abstraction chain, they abandon data (raw signal to code, code to phoneme, etc.) so that by the time they get to the language model, they don’t have the ability to revisit their initial data. For instance, if you use the words “C Plus Plus” in a document, what you’ll find is that you break the phrases used afterwards, because the phonemes of “++” are always screwing up the model. And once the language model is screwed up, you get this bizarre semi-phonetic mess that’s virtually impossible to work with. I’d probably prefer a pure phonetic output attached to the original note – the phonetic model would be indexable and searchable. It’s sort of like working with Ink – you learn that translating 100% of your handwriting into text is not necessary, so long as 99% of your handwriting is searchable.

By the way, one thing that really struck me about Jon’s screencast is how differently he approaches voice dictation. When composing text at my computer, words come out of me in a way that is entirely alien from my speech habits. My chief pitfall as a public speaker is that I talk fast. On the other hand, my professional writing comes out of me at the pace of about one phrase every 30 seconds, and that’s when I’m on a roll. One of the interesting things about Jon’s screencast is that he’s actually talking. When I’ve tried to use voice dictation, I’ve always basically voiced the word that I would otherwise type, which makes me sound like a robot with a run-down battery. “Voice dictation is…intriguing…to me…”

 

Screencast, Tablet PC, C#, Python, and Scoble Doesn’t Link To Me: What’s A Guy Got To Do?

The Da Vinci Coder

When you’re struck with writer’s block, you’re supposed to just write anything. This is what I just churned out instead of my next column:

I was awakened by my phone. You got used to that living on The Big Island. It’s already noon on Wall Street by the time the rosy fingers of dawn touch the wine-dark seas of Kona. Plus I have narcolepsy, a disease which tends to kick in at particularly inopportune times, as if advancing a plot that otherwise would flounder over flimsy connections . Narcolepsy is a crippling disease, but one which involves no physical deformations and yet is exotic enough so that most people have heard of it; I find it lends me an air of vulnerability that causes men to underestimate me and women to swoon.

“How may I be of service?” I said, unconsciously forming the words by which Parsifal proved himself worthy of receiving the Holy Grail from Anfortas, the Fisher King. Or so you’d believe from a mere surface reading of the medieval poet Wolfram von Eschenbach.

“Mr. O’Brien? Mr. Lawrence O’Brien?” Came the breathy voice of a woman — a voice that betrayed her as filled with worry and that further betrayed her as the owner of a cutting intellect that kept lesser men away. And that even further betrayed her as beautiful, a brunette, taller than average, with gray — no, green — eyes.

“Only my publisher calls me Lawrence,” I said, rising from bed and noting that it was a little after 5AM by the position of ‘Iwakeli’i, the male frigate bird, the W-shaped constellation known to the West as Casssiopeia, the queen whose insult of the Nereid sea nymphs dictates her constellation never touches the sea. Whose circumpolar orbit allows it, along with The Big Dipper, whose Hawaiian name is irrelevant but happens to be Na Hiku, to serve as utterly reliable timepieces for those who disdain — or distrust — the 32,768 oscillations per second of a quartz crystal exposed to an electrical current as well as for those whose quartz watches don’t have Indiglo and thus are quite difficult to read in the dark. “Call me Larry.”

“Well…Larry…I saw you on Larry King and–”

“Stop right there,” I said. I was in no mood to let her go further, beautiful leggy brunette or no. Programming legends like myself and, say, Don Box, draw beautiful women to us like moths to flame, which are drawn, not because of the heat, but because they are genetically programmed to maintain a steady orientation to the brightest light in the night sky — the moon, whose name is Luna and which is paired with Cassiopeia in the easily-confirmable-by-skeptical-readers Web site location  http://www.pbase.com/image/33397294. Except with us programming legends it wasn’t our simulacra of the reflection of the Sun to which beautiful women were compelled to maintain a steady orientation, but to our knowledge of esoterica, which is too close to an anagram of “eroticas” to be a coincidence.

Hmmm, 59,530 more words like that and you know Tom Hanks is on board. But maybe I should wait to see if National Treasure has legs…

I’m going to go for a swim…