Content-Aware Image Resizing For Photo Prints

I’ve got a few prototypes of content-aware image resizing running and the results are interesting, perhaps primarily because the algorithm works so well that usually you say “eh, what’s the big deal?” Unless there are faces in the photo, but more on that later…

Coincidentally, I ordered some prints to send to my in-laws yesterday and was reminded of the annoyance of crop differences between the standard snapshot aspect ratios. Ah hah, thinks I! I should do one of those “mashup” things the kids are all talking about!

Ideally, you’d have:

  • Easy access from your image software (“Oh, darn! Look at those default crop lines! Let me click on ‘Crop Perfect’!)
  • Integration into the printing process (see above.)
  • Preview
  • Maintain original file

Hmm…. There’s no Picasa plugin API (although there’s a Picasa Web Album API) and in my experience, all photo-printing places use an uploading plugin. Hmmm….

Any thoughts on strategy?

My Newbie Reaction to Halo 3 : Meh.

I’ve never played Halo nor Halo 2. I’ve now played the first level (“Groundfall” or somesuch) of Halo 3. Gotta’ tell you: my initial impression is that it’s not as entrancing as Gears of War or Call of Duty. The cutscenes make absolutely no sense to a newbie, the level design was nothing special (oh, the trees shake when you move past them. Cute.), the combat tactics weren’t clear, and the controls seemed sub-optimal (I read in Wired that you were supposed to always be switching between grenades, guns, and fists. But to use your fists, you have to take your thumb off the “look” control, so it’s hard to melee).

People talk about multiplayer, but let me clue you in to something about multiplayer: it’s offputting to normal people. Normal people don’t enjoy undergoing a series of losses before being buoyed up by the matchmaking system. Perhaps subsequent to the 10th match everything is wonderful as you slot into the ranks of your peers. But I’ll tell you what — I’ve never played 10 matches online in any game, because my first half-dozen matches have always been unpleasant. In racing games, people intentionally crash through the first corner; in first person shooters, you get picked off ten times in a single match.

Those pesky powers of 2

Excel 2007: The problem is that the 3rd column contains =a2*b2 … (the first 500 numbers for 2^32 pass, though…)


13 5041.154 100000
26 2520.577 100000
49 1337.449 100000
52 1260.288 100000
81 809.0741 100000
87 753.2759 100000
93 704.6774 100000
98 668.7245 100000
104 630.1442 100000
107 612.4766 100000
115 569.8696 100000
117 560.1282 100000
119 550.7143 100000
123 532.8049 100000
162 404.537 100000
169 387.7811 100000
173 378.815 100000
174 376.6379 100000
186 352.3387 100000

etc. etc.


Is there a pattern in there?

Magnifying Glass / Bug ID Expert System

I love this idea: a magnifying glass with a built-in expert system for identifying bugs. That’s a nice combination of form and function. My thrill was clouded upon reading the that it only identifies “more than 50 real live bugs;” if you’re going to do the hardware and firmware for this, it ought to be trivial to burn in the taxonomy for “thousands of real live bugs!” See, for instance, the remarkable 20Q orb.

Once upon a time, I fielded an expert system for identifying seabirds. One of the big problems was that interacting with the display necessarily meant taking your eyes off the seabird and, if you weren’t experienced, that often meant considerable time before re-acquiring. But if you were an experienced birder, you didn’t need the expert system. Plus display and battery life on the available portable computers of the time were problematic.

When the Palm came out, I investigated developing expert systems for them: the target niches were going to be birds, plants, and mushrooms. Too little market confidence and too few resources kept that one at the prototype stage.

Sadly, that “more than 50″ line and the lack of “I don’t know” button makes me wager that instead of having a real expert system, with inferencing and reasoning under uncertainty, the software is just a simple tree. Which, if I’m right, is pathetic — all of that work to develop a device and then to cripple it, for lack of a couple of hundred lines of code.

Writing A Technical Article, Pt. 3: Research Begins…

I’m not one of those people for whom the task of composition is a somewhat mechanical extension of increasingly detailed outlines. Nonetheless, research consumes between 80-90% of the time I spend on a technical article (note I said time and not effort — subjectively, writing text is two or three times harder than writing C++. You can satisfy a compiler.)

In this case, research will consist of:

  1. Integrating CodeAnalyst and Eclipse
  2. Profiling a simple application
  3. Implementing an image-processing application
  4. Profiling it to discover a performance problem
  5. Investigating that problem with CodeAnalyst
  6. Ameliorating it
  7. Evaluating the performance “win”
  8. If not “big enough” go to 3 

In this case, my target sample application has two algorithms that I think will be difficult:

  1. Seam-discovery (CPU intensive)
  2. Seam-removal (Memory-manipulation intensive)

With seam-discovery, I know that even if I implement a known-good algorithm (such as graph-cuts), I am guaranteed to peg the CPUs. My big gamble professionally is that I’m a good enough programmer to find a Step 6 “win” that I can pull off in a reasonable amount of time. A safer bet would be to choose an algorithm that would have performance problems if intentionally implemented in a naive manner (for instance, a filter that just iterated across rows and columns and, at each pixel, retrieved the neighboring pixels and averaged them. Pretty easy to tune that up!)

Now, a confession. By the time you read this, I’ve been somewhere not-Hawai’i for a week or so. I’ll carry print outs of this paper on content-aware image resizing and this paper on graph-cuts and study them on the plane tomorrow. I’ll have my laptop with me and may noodle around with source code, but it’s far more likely that I’ll be working with pen and paper to see if I can “get” the algorithm intuitively. Other than that, I’m going to do my darnedest not to spend too much time thinking about computers.

Coming September 19th or thereabouts: Part 4, Research Gets Underway…