iPower, my pwned host, boasts of being "Hacker safe"

I wasn’t going to name names, because I do not know the vulnerability which allowed a rootkit to be installed on my system. I may well have been the source of whatever vulnerability by which the system was compromised.

But this “Hacker Safe” blaze front and center on the iPower homepage is infuriating. The blaze is provided by way of ScanAlert. Judging from the logs that I saw before I was disconnected and lost the system, multiple machines within the same subnet as mine were compromised. The technical support from iPower was beyond unhelpful: the “live technical support” is provided by a call center that is not physically located with the data center. After the initial problems on Tuesday, we requested a local reboot and tighter reconfiguration. They couldn’t do it. Their only offer was to repave the machine and make it available to us over the Internet without any hardening of the attack surface! They couldn’t even activate a firewall for us or modify the ACL. We told them to go jump in a lake earlier today — four days after we discovered the compromise. Oh, and they’re not refunding us any of the $7,000 or so we paid to set up (multiple) servers with them.

iPower is “Hacker safe”? I suppose so, in a sense.

Highly not recommended.

I, Hacked

So one of my servers is totally compromised by a rootkit called Hacker Defender. I’ve spent the day trying to clean it off, and I think I just pronged it for good (cross my fingers). It’s funny how I discovered the problem (and by funny, I mean, there’s nothing funny about it this): my Tomcat-based task manager (Jira) stopped working. It threw a ClassNotFoundException when loading. That’s odd, thinks I, and switch to the relevant /classes directory and — sure enough — the /com folder leading to the classes is gone.

I sent out a message to users (“Who fracked with the server?”) and, after some stumbling around, explicitly unpacked the classes into that directory and … they disappear …

It turns out that the rootkit (or its payload) installs a significantly sized Java-based web service and then hides the .class files from Windows (I think the evil hidden process actively hooks kernel .dlls and hides .class files). Well, the same logic that hid the evil Java classes hid my good Java classes. And thus began my education.

So, long story short: I highly recommend Sophos Anti-Rootkit, which was able to diagnose the rootkit automatically and not-quite-automatically allowed me to locate and delete the critical initialization file that re-infects the system every time it is rebooted. (It is not enough to delete the driver!)

Now that the hidden files are visible, what does it turn out my system was doing? Trading movies. These guys f***ed with my system in orded to swap a cam of Norbit in German. F***ers.

Okay, so now I have logs of a whole bunch of machines, all presumably in the botnet. Do I send these to someone?

Update: OMFG. Someone from a British Telcom range just logged on (anonymously — how the f*** is he doing that? The guest account was never enabled, I’ve changed the name of the admin account, I’ve changed every password on the f***ing system), booted me off, and now I can’t access via Remote Desktop. Well, it was a nice server while it lasted…

2: The Prequel To 300

The weekend box office receipts aren’t yet published, but if Makalapua Cinemas in Kailua Kona are any benchmark of the American viewing population (and they aren’t), I predict that 300 will either set or approach record revenues.

We actually saw Zodiac, which was excellent (it’s ending challenges you to consider the gap between “preponderance of evidence” and “beyond a reasonable doubt,” and does so not from an authorially introduced ambiguity, but from the question of human obsession projecting patterns into mountains of circumstantial evidence), but there were huge lines for the multiple screens showing 300.

But the real joy was two boys in line locked in combat on the battleground of Thermopylae. 

Now, it may be that the history of the Greco-Persian wars are taught in some crusty ivy-shrouded prep school in Connecticut, but I’m quite certain that it’s not in any curriculum in Hawai’i. And, let’s be honest, it’s not in that category of “things a curious teenager might be expected to know.” That there was a city-state called Sparta — sure. That they were famous warriors — okay. Other than that, I insist that everything I heard was either made up on the spot or gleaned from Wikipedia in anticipation of the movie.

Which is fine. That’s how I learned about the battle of Thermopylae. But what was classic was that these two young men were trying to impress several lovely young women and, locked in intellectual battle, carried themselves well beyond their depth. And, having misremembered vital details of the Wikipedia article they found themselves not only (wrongly) explaining the outcome of the battle, they were asked by the young ladies what happened next in the war. Which they didn’t know. The one lad’s courage failed and he stammered something “Well, watch the movie,” while the other, bold warrior, took the chance and won the battle-for-fair-hearts by saying that the Persians retreated. And then, sadly, I had to go get my bucket of popcorn.

 Human drama. It never changes. He was a smart kid and I hope that he finesses his way out of his problem. He’d done the critical thing: getting the girls interested. Enthusiasm, confidence, a good story: Well done. But, my boy, you have to know “what happens next?”

Pragmatic Programmers To Publish Erlang Book

The Pragmatic Programmers have a very good sense of software developmet trends — they’re doing today what O’Reilly did in the early 90s. Coming in July from them is Joe Armstrong’s Programming Erlang. Erlang is seen by programming language mavens as one of the real contenders for the crown of “most practical language for writing concurrent programs.” I haven’t sensed any real groundswell for Erlang recently, but a great book on the language might well contribute to an uptick in popularity.

Assert(LOC(Test) ~= LOC(App))

Andrew Binstock discusses a talk with Agitar about “how many unit tests are enough?” The upshot is that if the amount of test code is roughly equal to the amount of application code, that generally translates into code coverage of around 70% and is generally “pretty good shape.”

I think that’s probably about right, although I’ll admit to rarely maintaining that level in a serious project — shame on me.